2023-10-06T20:40:27.123 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T20:40:27.124 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643 --name rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/orig.config.yaml 2023-10-06T20:40:27.202 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T20:40:27.296 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643 branch: rishabh-cephfs-mon description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: null first_in_suite: false job_id: '7414643' kernel: client: branch: testing flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-cephfs-mon ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_client_delegate_inos_pct: 50 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 14d8d032507701c76ca224e769ee65a96b660a8c subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 14d8d032507701c76ca224e769ee65a96b660a8c kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-cephfs-mon sha1: 14d8d032507701c76ca224e769ee65a96b660a8c owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7175 sha1: 14d8d032507701c76ca224e769ee65a96b660a8c sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: rishabh-cephfs-mon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 14d8d032507701c76ca224e769ee65a96b660a8c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-06_11:56:52 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T20:40:27.296 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa; will attempt to use it 2023-10-06T20:40:27.297 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks 2023-10-06T20:40:27.297 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T20:40:27.299 INFO:teuthology.task.internal:Checking packages... 2023-10-06T20:40:27.321 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '14d8d032507701c76ca224e769ee65a96b660a8c' 2023-10-06T20:40:27.322 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T20:40:27.322 INFO:teuthology.packaging:ref: None 2023-10-06T20:40:27.322 INFO:teuthology.packaging:tag: None 2023-10-06T20:40:27.322 INFO:teuthology.packaging:branch: rishabh-cephfs-mon 2023-10-06T20:40:27.322 INFO:teuthology.packaging:sha1: 14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:40:27.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-cephfs-mon 2023-10-06T20:40:27.593 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6616.g14d8d032 2023-10-06T20:40:27.593 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T20:40:27.601 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T20:40:27.601 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T20:40:27.606 INFO:teuthology.task.internal:Saving configuration 2023-10-06T20:40:27.629 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T20:40:27.635 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T20:40:27.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.094078', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:40:27.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.095683', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:40:27.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.092210', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T20:40:27.699 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T20:40:27.705 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-06T20:40:27.705 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-06T20:40:27.705 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-06T20:40:27.705 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T20:40:27.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23be71caf0>, signals=[15]) 2023-10-06T20:40:27.799 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T20:40:27.805 INFO:teuthology.task.internal:Opening connections... 2023-10-06T20:40:27.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T20:40:27.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:40:27.875 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2023-10-06T20:40:27.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:40:27.943 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T20:40:27.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:40:28.011 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T20:40:28.018 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2023-10-06T20:40:28.039 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2023-10-06T20:40:28.039 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2023-10-06T20:40:28.100 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:40:28.100 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:40:28.100 INFO:teuthology.orchestra.run.smithi026.stdout:ID="rhel" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="fedora" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8.6" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:40:28.101 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:40:28.102 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:40:28.103 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-10-06T20:40:28.128 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2023-10-06T20:40:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2023-10-06T20:40:28.149 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-10-06T20:40:28.209 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:40:28.209 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:40:28.209 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2023-10-06T20:40:28.209 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:40:28.210 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:40:28.211 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:40:28.211 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:40:28.211 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:40:28.211 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:40:28.211 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:40:28.212 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:40:28.212 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:40:28.212 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-10-06T20:40:28.237 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-10-06T20:40:28.257 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-10-06T20:40:28.258 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-10-06T20:40:28.318 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T20:40:28.318 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2023-10-06T20:40:28.318 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2023-10-06T20:40:28.318 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T20:40:28.319 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T20:40:28.320 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-06T20:40:28.321 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-10-06T20:40:28.345 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T20:40:28.355 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T20:40:28.361 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T20:40:28.361 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:40:28.364 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:40:28.367 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-10-06T20:40:28.385 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T20:40:28.391 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T20:40:28.392 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:40:28.424 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:40:28.427 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-10-06T20:40:28.456 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T20:40:28.530 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T20:40:28.549 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-06T20:40:28.550 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-06T20:40:28.550 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-06T20:40:28.550 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:40:28.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:40:28.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T20:40:28.725 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:40:28.726 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T20:40:28.726 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-10-06T20:40:28.746 INFO:teuthology.orchestra.run.smithi026.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-06T20:40:28.746 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:40:28.746 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T20:40:28.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum upgrade -y linux-firmware 2023-10-06T20:40:30.001 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-10-06T20:40:30.002 INFO:teuthology.orchestra.run.smithi026.stdout:Unable to read consumer identity 2023-10-06T20:40:31.601 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-10-06T20:40:35.409 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 16 MB 00:03 2023-10-06T20:40:39.093 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-10-06T20:40:39.918 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-10-06T20:40:39.919 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2023-10-06T20:40:39.920 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-10-06T20:40:40.062 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T20:40:40.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T20:40:40.289 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T20:40:40.290 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T20:40:40.292 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2023-10-06T20:40:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06 20:40:53 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T20:40:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2023-10-06 20:40:53-- 2023-10-06T20:40:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 13s 2023-10-06T20:40:53.110 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 363M in 13s (28.9 MB/s) 2023-10-06T20:40:53.111 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T20:40:53.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T20:40:54.777 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying... ######################################## 2023-10-06T20:40:54.843 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing... ######################################## 2023-10-06T20:40:54.929 INFO:teuthology.orchestra.run.smithi026.stdout:Updating / installing... 2023-10-06T20:41:35.504 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T20:41:59.543 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T20:41:59.543 DEBUG:teuthology.orchestra.run.smithi026:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T20:41:59.600 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T20:41:59.601 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T20:41:59.602 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T20:41:59.603 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T20:41:59.604 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T20:41:59.605 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T20:41:59.606 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T20:41:59.607 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T20:41:59.608 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T20:41:59.609 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T20:41:59.610 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T20:41:59.611 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T20:41:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T20:41:59.613 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T20:41:59.614 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T20:41:59.615 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T20:41:59.616 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T20:41:59.617 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T20:41:59.618 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T20:41:59.619 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T20:41:59.620 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T20:41:59.621 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T20:41:59.622 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T20:41:59.623 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T20:41:59.624 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T20:41:59.625 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T20:41:59.626 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T20:41:59.627 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T20:41:59.628 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T20:41:59.629 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T20:41:59.630 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T20:41:59.631 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T20:41:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T20:41:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T20:41:59.634 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T20:41:59.635 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T20:41:59.636 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T20:41:59.637 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T20:41:59.638 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T20:41:59.639 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T20:41:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T20:41:59.641 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T20:41:59.642 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T20:41:59.643 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T20:41:59.644 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T20:41:59.645 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T20:41:59.646 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T20:41:59.647 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T20:41:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T20:41:59.649 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T20:41:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T20:41:59.651 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T20:41:59.652 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T20:41:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T20:41:59.654 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T20:41:59.655 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T20:41:59.656 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T20:41:59.657 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T20:41:59.658 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T20:41:59.659 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T20:41:59.660 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T20:41:59.661 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T20:41:59.662 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T20:41:59.663 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T20:41:59.664 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T20:41:59.665 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T20:41:59.666 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T20:41:59.667 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T20:41:59.668 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T20:41:59.669 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T20:41:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T20:41:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T20:41:59.672 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T20:41:59.673 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T20:41:59.674 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T20:41:59.675 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T20:41:59.676 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T20:41:59.677 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T20:41:59.678 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T20:41:59.679 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T20:41:59.680 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T20:41:59.681 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T20:41:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T20:41:59.683 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T20:41:59.684 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T20:41:59.685 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T20:41:59.686 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T20:41:59.687 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T20:41:59.688 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T20:41:59.689 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T20:41:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T20:41:59.691 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T20:41:59.692 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T20:41:59.693 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T20:41:59.694 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T20:41:59.695 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T20:41:59.696 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T20:41:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T20:41:59.698 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T20:41:59.699 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T20:41:59.700 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T20:41:59.701 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T20:41:59.702 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T20:41:59.703 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T20:41:59.704 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T20:41:59.705 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T20:41:59.706 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T20:41:59.707 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T20:41:59.708 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T20:41:59.709 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T20:41:59.710 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T20:41:59.711 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T20:41:59.712 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T20:41:59.713 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T20:41:59.714 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T20:41:59.715 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T20:41:59.716 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T20:41:59.717 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T20:41:59.718 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T20:41:59.719 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T20:41:59.720 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T20:41:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T20:41:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T20:41:59.723 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T20:41:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T20:41:59.725 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T20:41:59.726 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T20:41:59.727 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T20:41:59.728 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T20:41:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T20:41:59.730 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T20:41:59.731 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T20:41:59.732 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T20:41:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T20:41:59.734 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T20:41:59.735 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T20:41:59.736 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T20:41:59.737 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T20:41:59.738 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T20:41:59.739 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T20:41:59.740 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T20:41:59.741 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T20:41:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T20:41:59.743 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T20:41:59.744 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T20:41:59.745 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T20:41:59.746 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T20:41:59.747 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T20:41:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T20:41:59.749 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T20:41:59.750 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T20:41:59.751 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T20:41:59.752 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T20:41:59.753 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T20:41:59.754 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T20:41:59.755 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T20:41:59.756 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T20:41:59.757 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T20:41:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T20:41:59.759 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T20:41:59.760 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T20:41:59.761 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T20:41:59.762 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.762 DEBUG:teuthology.orchestra.run.smithi026:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:41:59.788 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.789 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-10-06T20:41:59.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2023-10-06T20:41:59.868 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2023-10-06T20:41:59.868 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2023-10-06T20:41:59.868 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.02 2023-10-06T20:41:59.868 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 123.el8 2023-10-06T20:41:59.868 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Group : System Environment/Base 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 9265450 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-06T20:41:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Relocations : (not relocatable) 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : Red Hat, Inc. 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : Red Hat, Inc. 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T20:41:59.870 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T20:41:59.871 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2023-10-06T20:41:59.871 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:41:59.871 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T20:41:59.872 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T20:41:59.872 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:41:59.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T20:42:04.294 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2023-10-06T20:42:06.783 INFO:teuthology.orchestra.run.smithi026.stderr:done 2023-10-06T20:42:06.785 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2023-10-06T20:42:06.802 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.AojboHR7Bk 2023-10-06T20:42:06.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AojboHR7Bk 2023-10-06T20:42:06.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.AojboHR7Bk 2023-10-06T20:42:07.150 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.AojboHR7Bk is 6KB 2023-10-06T20:42:07.198 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.AojboHR7Bk 2023-10-06T20:42:07.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T20:42:07.296 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-06T20:42:07.296 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-06T20:42:07.296 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T20:42:07.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-10-06T20:42:07.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2023-10-06T20:42:37.469 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T20:42:37.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:42:37.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:42:55.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-10-06T20:43:04.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:43:05.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:43:23.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-10-06T20:43:35.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:43:35.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:43:36.983 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-10-06T20:43:37.416 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:43:37.416 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T20:43:37.416 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-10-06T20:43:37.458 INFO:teuthology.orchestra.run.smithi026.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T20:43:37.459 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T20:43:37.459 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T20:43:37.459 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T20:43:38.460 DEBUG:teuthology.parallel:result is None 2023-10-06T20:43:38.460 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T20:43:38.467 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T20:43:38.468 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:43:38.473 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:43:38.475 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T20:43:38.514 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T20:43:38.521 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T20:43:38.560 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T20:43:38.560 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:43:38.565 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:43:38.567 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T20:43:38.617 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T20:43:38.623 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T20:43:38.624 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:43:38.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:43:38.653 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T20:43:38.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:43:38.673 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:43:38.677 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:43:38.696 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T20:43:38.743 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.756 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.766 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.776 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.793 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.824 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T20:43:38.828 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T20:43:38.836 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T20:43:38.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:43:38.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:43:38.842 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T20:43:38.906 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T20:43:38.915 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T20:43:38.916 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:43:38.920 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:43:38.922 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T20:43:38.958 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.149 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.297 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:43:39.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:43:39.366 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.413 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.469 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.603 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:43:39.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:43:39.689 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T20:43:39.793 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.879 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T20:43:39.924 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:43:39.924 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T20:43:39.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2023-10-06T20:43:40.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2023-10-06T20:43:40.002 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-10-06T20:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:43:40.093 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:43:40.120 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T20:43:40.551 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T20:43:40.560 INFO:teuthology.task.internal:Starting timer... 2023-10-06T20:43:40.560 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T20:43:40.571 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T20:43:40.579 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T20:43:40.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2023-10-06T20:43:40.705 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2023-10-06T20:43:40.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2023-10-06T20:43:40.773 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2023-10-06T20:43:40.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2023-10-06T20:43:40.839 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2023-10-06T20:43:40.840 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T20:43:40.840 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2023-10-06T20:43:40.894 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2023-10-06T20:43:40.895 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2023-10-06T20:43:40.925 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2023-10-06T20:43:40.926 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2023-10-06T20:43:40.951 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2023-10-06T20:43:40.951 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2023-10-06T20:43:40.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:43:41.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:43:41.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:43:41.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:43:41.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T20:43:41.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T20:43:41.079 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T20:43:41.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:43:41.148 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:43:41.176 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2023-10-06T20:43:41.204 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T20:43:41.214 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T20:43:41.219 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T20:43:41.352 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T20:43:41.370 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T20:43:41.371 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi078.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T20:56:58.885 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-10-06T20:56:58.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:56:58.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:56:58.991 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-10-06T20:56:59.100 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-10-06T20:56:59.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-06T20:56:59.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:56:59.171 DEBUG:teuthology.orchestra.run.smithi078:> true 2023-10-06T20:56:59.243 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-06T20:56:59.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T20:56:59.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T20:56:59.311 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-06T20:56:59.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-06T20:56:59.386 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T20:56:59.401 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T20:56:59.401 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T20:56:59.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T20:56:59.406 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T20:56:59.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T20:56:59.409 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T20:56:59.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T20:56:59.443 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T20:56:59.456 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T20:56:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T20:56:59.471 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T20:56:59.478 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T20:56:59.514 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T20:56:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2023-10-06T20:56:59.536 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2023-10-06T20:56:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2023-10-06T20:56:59.548 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2023-10-06T20:56:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T20:56:59.562 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T20:56:59.576 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T20:56:59.576 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T20:56:59.610 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2023-10-06T20:56:59.640 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2023-10-06T20:56:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:bash: ntpq: command not found 2023-10-06T20:56:59.656 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-06T20:56:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T20:56:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2023-10-06T20:56:59.658 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.660 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T20:56:59.660 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-06T20:56:59.660 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.661 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.661 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.661 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T20:56:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T20:56:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.805 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T20:56:59.807 INFO:teuthology.run_tasks:Running task pexec... 2023-10-06T20:56:59.817 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-06T20:56:59.817 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T20:56:59.817 DEBUG:teuthology.orchestra.run.smithi026:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T20:56:59.818 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi078.front.sepia.ceph.com 2023-10-06T20:56:59.818 DEBUG:teuthology.orchestra.run.smithi078:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T20:56:59.818 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T20:56:59.819 DEBUG:teuthology.orchestra.run.smithi167:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-06T20:57:00.191 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:00.192 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:00.334 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:01.454 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:05:18 ago on Fri 06 Oct 2023 08:51:43 PM UTC. 2023-10-06T20:57:01.469 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:05:23 ago on Fri 06 Oct 2023 08:51:38 PM UTC. 2023-10-06T20:57:01.649 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:55 ago on Fri 06 Oct 2023 08:55:06 PM UTC. 2023-10-06T20:57:03.332 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-06T20:57:03.333 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-06T20:57:03.333 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-10-06T20:57:03.333 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-06T20:57:03.333 INFO:teuthology.orchestra.run.smithi167.stdout:Resetting modules: 2023-10-06T20:57:03.333 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools 2023-10-06T20:57:03.334 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:03.334 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-06T20:57:03.334 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-06T20:57:03.334 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:03.339 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2023-10-06T20:57:03.340 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout:Resetting modules: 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout: container-tools 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2023-10-06T20:57:03.341 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:03.629 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-10-06T20:57:03.630 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-10-06T20:57:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2023-10-06T20:57:03.630 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-10-06T20:57:03.630 INFO:teuthology.orchestra.run.smithi026.stdout:Resetting modules: 2023-10-06T20:57:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools 2023-10-06T20:57:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:03.635 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-10-06T20:57:03.635 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-10-06T20:57:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:04.307 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:05:20 ago on Fri 06 Oct 2023 08:51:43 PM UTC. 2023-10-06T20:57:04.340 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:05:25 ago on Fri 06 Oct 2023 08:51:38 PM UTC. 2023-10-06T20:57:04.931 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:58 ago on Fri 06 Oct 2023 08:55:06 PM UTC. 2023-10-06T20:57:08.253 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-06T20:57:08.424 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2023-10-06T20:57:08.686 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-10-06T20:57:08.725 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:08.889 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:09.315 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:09.738 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:05:26 ago on Fri 06 Oct 2023 08:51:43 PM UTC. 2023-10-06T20:57:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:05:31 ago on Fri 06 Oct 2023 08:51:38 PM UTC. 2023-10-06T20:57:10.349 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:03 ago on Fri 06 Oct 2023 08:55:06 PM UTC. 2023-10-06T20:57:11.439 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-06T20:57:11.534 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-10-06T20:57:11.534 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout:Installing group/module packages: 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T20:57:11.535 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T20:57:11.536 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T20:57:11.536 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T20:57:11.536 INFO:teuthology.orchestra.run.smithi167.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T20:57:11.536 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-06T20:57:11.536 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout:Installing module profiles: 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools/common 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:11.537 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout: container-tools rhel8 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================================================ 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout:Install 10 Packages 2023-10-06T20:57:11.538 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:11.539 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 18 M 2023-10-06T20:57:11.539 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 65 M 2023-10-06T20:57:11.540 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-06T20:57:11.633 INFO:teuthology.orchestra.run.smithi167.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 421 kB/s | 41 kB 00:00 2023-10-06T20:57:11.641 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2023-10-06T20:57:11.659 INFO:teuthology.orchestra.run.smithi167.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 203 kB/s | 25 kB 00:00 2023-10-06T20:57:11.684 INFO:teuthology.orchestra.run.smithi167.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 981 kB/s | 49 kB 00:00 2023-10-06T20:57:11.709 INFO:teuthology.orchestra.run.smithi167.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 543 kB/s | 94 kB 00:00 2023-10-06T20:57:11.733 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================================ 2023-10-06T20:57:11.733 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2023-10-06T20:57:11.733 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================================ 2023-10-06T20:57:11.733 INFO:teuthology.orchestra.run.smithi078.stdout:Installing group/module packages: 2023-10-06T20:57:11.733 INFO:teuthology.orchestra.run.smithi078.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout:Installing module profiles: 2023-10-06T20:57:11.734 INFO:teuthology.orchestra.run.smithi078.stdout: container-tools/common 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling module streams: 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout: container-tools rhel8 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================================ 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout:Install 10 Packages 2023-10-06T20:57:11.735 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:11.736 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 18 M 2023-10-06T20:57:11.736 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 65 M 2023-10-06T20:57:11.736 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2023-10-06T20:57:11.751 INFO:teuthology.orchestra.run.smithi167.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-10-06T20:57:11.835 INFO:teuthology.orchestra.run.smithi167.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-10-06T20:57:11.845 INFO:teuthology.orchestra.run.smithi078.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 858 kB/s | 94 kB 00:00 2023-10-06T20:57:11.860 INFO:teuthology.orchestra.run.smithi167.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.9 MB/s | 213 kB 00:00 2023-10-06T20:57:11.862 INFO:teuthology.orchestra.run.smithi078.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 196 kB/s | 25 kB 00:00 2023-10-06T20:57:11.879 INFO:teuthology.orchestra.run.smithi078.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 285 kB/s | 41 kB 00:00 2023-10-06T20:57:11.896 INFO:teuthology.orchestra.run.smithi078.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 978 kB/s | 49 kB 00:00 2023-10-06T20:57:11.946 INFO:teuthology.orchestra.run.smithi078.stdout:(5/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-10-06T20:57:11.980 INFO:teuthology.orchestra.run.smithi078.stdout:(6/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-10-06T20:57:11.986 INFO:teuthology.orchestra.run.smithi167.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-10-06T20:57:12.005 INFO:teuthology.orchestra.run.smithi078.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-10-06T20:57:12.052 INFO:teuthology.orchestra.run.smithi167.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-10-06T20:57:12.106 INFO:teuthology.orchestra.run.smithi078.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-10-06T20:57:12.144 INFO:teuthology.orchestra.run.smithi167.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 23 MB/s | 7.0 MB 00:00 2023-10-06T20:57:12.145 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:57:12.145 INFO:teuthology.orchestra.run.smithi167.stdout:Total 30 MB/s | 18 MB 00:00 2023-10-06T20:57:12.239 INFO:teuthology.orchestra.run.smithi078.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-10-06T20:57:12.331 INFO:teuthology.orchestra.run.smithi078.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-10-06T20:57:12.331 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:57:12.332 INFO:teuthology.orchestra.run.smithi078.stdout:Total 31 MB/s | 18 MB 00:00 2023-10-06T20:57:12.345 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-06T20:57:12.364 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-06T20:57:12.364 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-06T20:57:12.393 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-10-06T20:57:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-10-06T20:57:12.499 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-10-06T20:57:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-10-06T20:57:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Installing group/module packages: 2023-10-06T20:57:12.499 INFO:teuthology.orchestra.run.smithi026.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-06T20:57:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Installing module profiles: 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools/common 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools rhel8 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Install 10 Packages 2023-10-06T20:57:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:12.502 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 18 M 2023-10-06T20:57:12.502 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 65 M 2023-10-06T20:57:12.502 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-10-06T20:57:12.531 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2023-10-06T20:57:12.547 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-06T20:57:12.549 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2023-10-06T20:57:12.549 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2023-10-06T20:57:12.551 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-06T20:57:12.701 INFO:teuthology.orchestra.run.smithi026.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 125 kB/s | 25 kB 00:00 2023-10-06T20:57:12.726 INFO:teuthology.orchestra.run.smithi026.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 183 kB/s | 41 kB 00:00 2023-10-06T20:57:12.731 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2023-10-06T20:57:12.735 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2023-10-06T20:57:12.751 INFO:teuthology.orchestra.run.smithi026.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 378 kB/s | 94 kB 00:00 2023-10-06T20:57:12.851 INFO:teuthology.orchestra.run.smithi026.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 328 kB/s | 49 kB 00:00 2023-10-06T20:57:12.926 INFO:teuthology.orchestra.run.smithi026.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.6 MB/s | 524 kB 00:00 2023-10-06T20:57:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 896 kB/s | 149 kB 00:00 2023-10-06T20:57:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-10-06T20:57:13.072 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-06T20:57:13.194 INFO:teuthology.orchestra.run.smithi026.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-10-06T20:57:13.256 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T20:57:13.311 INFO:teuthology.orchestra.run.smithi026.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.5 MB/s | 2.2 MB 00:00 2023-10-06T20:57:13.385 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2023-10-06T20:57:13.424 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:13.427 INFO:teuthology.orchestra.run.smithi026.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-10-06T20:57:13.428 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:57:13.429 INFO:teuthology.orchestra.run.smithi026.stdout:Total 20 MB/s | 18 MB 00:00 2023-10-06T20:57:13.573 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T20:57:13.610 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T20:57:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:13.769 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-10-06T20:57:13.794 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-10-06T20:57:13.794 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-10-06T20:57:13.794 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T20:57:13.973 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T20:57:14.163 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T20:57:14.320 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-10-06T20:57:14.325 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-10-06T20:57:14.395 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T20:57:14.541 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T20:57:14.923 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-10-06T20:57:15.098 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-06T20:57:15.160 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T20:57:15.288 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:15.327 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T20:57:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-06T20:57:15.692 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-06T20:57:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T20:57:16.093 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-06T20:57:16.219 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T20:57:16.262 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T20:57:16.423 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T20:57:16.448 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T20:57:16.634 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:16.636 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T20:57:16.805 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:16.904 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T20:57:16.905 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T20:57:16.943 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T20:57:17.058 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T20:57:17.059 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T20:57:17.059 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T20:57:17.059 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T20:57:17.059 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T20:57:17.910 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-06T20:57:18.080 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T20:57:18.080 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:05:34 ago on Fri 06 Oct 2023 08:51:43 PM UTC. 2023-10-06T20:57:18.188 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-06T20:57:18.317 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T20:57:18.317 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:05:39 ago on Fri 06 Oct 2023 08:51:38 PM UTC. 2023-10-06T20:57:18.361 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-06T20:57:18.559 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:18.975 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-06T20:57:18.976 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-06T20:57:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-06T20:57:20.414 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:13 ago on Fri 06 Oct 2023 08:55:06 PM UTC. 2023-10-06T20:57:20.888 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T20:57:21.091 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:21.092 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-06T20:57:21.306 DEBUG:teuthology.parallel:result is None 2023-10-06T20:57:21.333 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2023-10-06T20:57:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:21.563 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2023-10-06T20:57:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:21.564 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2023-10-06T20:57:21.776 DEBUG:teuthology.parallel:result is None 2023-10-06T20:57:23.199 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:23.426 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-06T20:57:23.427 INFO:teuthology.orchestra.run.smithi026.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-06T20:57:23.427 INFO:teuthology.orchestra.run.smithi026.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-06T20:57:23.427 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:23.427 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-10-06T20:57:23.632 DEBUG:teuthology.parallel:result is None 2023-10-06T20:57:23.632 INFO:teuthology.run_tasks:Running task install... 2023-10-06T20:57:23.641 DEBUG:teuthology.task.install:project ceph 2023-10-06T20:57:23.641 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'}} 2023-10-06T20:57:23.641 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c'} 2023-10-06T20:57:23.642 INFO:teuthology.task.install:Using flavor: default 2023-10-06T20:57:23.647 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T20:57:23.647 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T20:57:23.648 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T20:57:23.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:57:23.650 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T20:57:23.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:57:23.651 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'tag': None, 'wait_for_package': False} 2023-10-06T20:57:23.651 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:57:23.791 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T20:57:23.791 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T20:57:23.869 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T20:57:23.870 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T20:57:23.878 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/ 2023-10-06T20:57:23.878 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6616.g14d8d032 2023-10-06T20:57:23.908 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T20:57:23.908 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:57:23.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T20:57:23.979 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T20:57:23.979 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:57:23.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T20:57:23.980 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T20:57:23.980 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T20:57:24.006 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T20:57:24.007 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T20:57:24.053 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T20:57:24.053 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:57:24.054 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T20:57:24.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T20:57:24.079 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-06T20:57:24.080 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/14d8d032507701c76ca224e769ee65a96b660a8c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T20:57:24.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T20:57:24.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T20:57:24.150 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T20:57:24.189 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T20:57:24.215 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2023-10-06T20:57:24.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2023-10-06T20:57:24.226 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T20:57:24.257 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2023-10-06T20:57:24.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2023-10-06T20:57:24.292 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2023-10-06T20:57:24.293 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2023-10-06T20:57:24.638 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:24.685 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:24.743 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:25.039 INFO:teuthology.orchestra.run.smithi078.stdout:44 files removed 2023-10-06T20:57:25.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T20:57:25.101 INFO:teuthology.orchestra.run.smithi167.stdout:44 files removed 2023-10-06T20:57:25.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T20:57:25.183 INFO:teuthology.orchestra.run.smithi026.stdout:44 files removed 2023-10-06T20:57:25.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-06T20:57:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:25.533 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:25.719 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-10-06T20:57:26.382 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 210 kB/s | 81 kB 00:00 2023-10-06T20:57:26.562 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 186 kB/s | 81 kB 00:00 2023-10-06T20:57:26.573 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 310 kB/s | 81 kB 00:00 2023-10-06T20:57:26.692 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-10-06T20:57:26.821 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2023-10-06T20:57:26.874 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2023-10-06T20:57:27.051 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-10-06T20:57:27.055 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 7.0 kB/s | 1.5 kB 00:00 2023-10-06T20:57:27.191 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-10-06T20:57:27.231 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-10-06T20:57:27.244 INFO:teuthology.orchestra.run.smithi078.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-10-06T20:57:27.359 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-06T20:57:27.992 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-10-06T20:57:28.301 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T20:57:29.057 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-10-06T20:57:31.641 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 404 kB/s | 24 kB 00:00 2023-10-06T20:57:32.634 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 491 kB/s | 24 kB 00:00 2023-10-06T20:57:32.646 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 MB/s | 47 MB 00:00 2023-10-06T20:57:32.771 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 627 kB/s | 24 kB 00:00 2023-10-06T20:57:33.995 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 MB/s | 47 MB 00:01 2023-10-06T20:57:34.026 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 47 MB 00:01 2023-10-06T20:57:40.948 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-10-06T20:57:42.335 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-10-06T20:57:43.312 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 MB/s | 53 MB 00:01 2023-10-06T20:57:52.205 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:52.206 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:52.220 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:52.221 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:52.222 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T20:57:52.223 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T20:57:52.225 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T20:57:52.228 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T20:57:52.229 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T20:57:52.360 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-06T20:57:52.372 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-10-06T20:57:52.372 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T20:57:52.373 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T20:57:52.374 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T20:57:52.375 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T20:57:52.376 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T20:57:52.377 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T20:57:52.378 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T20:57:52.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T20:57:52.380 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T20:57:52.381 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:52.382 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T20:57:52.383 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql 10 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:Install 128 Packages 2023-10-06T20:57:52.384 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 3 Packages 2023-10-06T20:57:52.385 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:57:52.385 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 208 M 2023-10-06T20:57:52.385 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-06T20:57:52.724 INFO:teuthology.orchestra.run.smithi167.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-10-06T20:57:52.999 INFO:teuthology.orchestra.run.smithi167.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 3.3 MB/s | 918 kB 00:00 2023-10-06T20:57:53.074 INFO:teuthology.orchestra.run.smithi167.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 2.2 MB/s | 167 kB 00:00 2023-10-06T20:57:53.612 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:53.613 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:53.628 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:53.629 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:53.630 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T20:57:53.631 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T20:57:53.633 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T20:57:53.636 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T20:57:53.637 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T20:57:53.683 INFO:teuthology.orchestra.run.smithi167.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 3.8 MB/s | 2.3 MB 00:00 2023-10-06T20:57:53.770 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2023-10-06T20:57:53.782 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================================================== 2023-10-06T20:57:53.782 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================================================== 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T20:57:53.783 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T20:57:53.784 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T20:57:53.785 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T20:57:53.786 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T20:57:53.786 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T20:57:53.786 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T20:57:53.786 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T20:57:53.787 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T20:57:53.787 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T20:57:53.787 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T20:57:53.787 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T20:57:53.787 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T20:57:53.788 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T20:57:53.789 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T20:57:53.790 INFO:teuthology.orchestra.run.smithi078.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T20:57:53.793 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T20:57:53.794 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T20:57:53.795 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T20:57:53.796 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T20:57:53.796 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T20:57:53.796 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T20:57:53.796 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:53.796 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T20:57:53.797 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T20:57:53.798 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T20:57:53.799 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling module streams: 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql 10 2023-10-06T20:57:53.800 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:53.801 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2023-10-06T20:57:53.801 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================================================== 2023-10-06T20:57:53.801 INFO:teuthology.orchestra.run.smithi078.stdout:Install 128 Packages 2023-10-06T20:57:53.801 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 3 Packages 2023-10-06T20:57:53.801 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:57:53.802 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 208 M 2023-10-06T20:57:53.802 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2023-10-06T20:57:54.012 INFO:teuthology.orchestra.run.smithi078.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-06T20:57:54.025 INFO:teuthology.orchestra.run.smithi167.stdout:(5/131): ceph-common-18.0.0-6616.g14d8d032.el8. 16 MB/s | 23 MB 00:01 2023-10-06T20:57:54.075 INFO:teuthology.orchestra.run.smithi167.stdout:(6/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 4.1 MB/s | 1.6 MB 00:00 2023-10-06T20:57:54.384 INFO:teuthology.orchestra.run.smithi167.stdout:(7/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 13 MB/s | 4.6 MB 00:00 2023-10-06T20:57:54.979 INFO:teuthology.orchestra.run.smithi078.stdout:(2/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 4.8 MB/s | 5.4 MB 00:01 2023-10-06T20:57:55.030 INFO:teuthology.orchestra.run.smithi078.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 3.3 MB/s | 167 kB 00:00 2023-10-06T20:57:55.051 INFO:teuthology.orchestra.run.smithi167.stdout:(8/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 2.2 MB/s | 5.4 MB 00:02 2023-10-06T20:57:55.142 INFO:teuthology.orchestra.run.smithi167.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 266 kB/s | 24 kB 00:00 2023-10-06T20:57:55.280 INFO:teuthology.orchestra.run.smithi078.stdout:(4/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 725 kB/s | 918 kB 00:01 2023-10-06T20:57:55.496 INFO:teuthology.orchestra.run.smithi078.stdout:(5/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 4.9 MB/s | 2.3 MB 00:00 2023-10-06T20:57:55.635 INFO:teuthology.orchestra.run.smithi167.stdout:(10/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 11 MB/s | 14 MB 00:01 2023-10-06T20:57:55.701 INFO:teuthology.orchestra.run.smithi167.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 3.8 MB/s | 260 kB 00:00 2023-10-06T20:57:55.756 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:55.757 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T20:57:55.760 INFO:teuthology.orchestra.run.smithi167.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 533 kB/s | 31 kB 00:00 2023-10-06T20:57:55.773 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:55.774 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-06T20:57:55.776 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-06T20:57:55.777 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T20:57:55.779 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T20:57:55.783 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T20:57:55.784 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T20:57:55.885 INFO:teuthology.orchestra.run.smithi167.stdout:(13/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 6.6 MB/s | 840 kB 00:00 2023-10-06T20:57:55.938 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.4 k 2023-10-06T20:57:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 5.4 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 918 k 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 167 k 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 1.6 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 132 k 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 1.8 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 7.4 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 47 k 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 14 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 49 M 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 261 k 2023-10-06T20:57:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 192 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 260 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 18 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 31 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 840 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 139 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-06T20:57:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 196 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 371 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 364 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 107 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 94 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.6 M 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 179 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-06T20:57:55.955 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 3.7 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 23 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 23 k 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 2.3 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 242 k 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 4.6 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 18 M 2023-10-06T20:57:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6616.g14d8d032.el8 ceph-noarch 15 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 24 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 183 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-06T20:57:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 518 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 6.9 M 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T20:57:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 45 k 2023-10-06T20:57:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.0.0-6616.g14d8d032.el8 ceph 126 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T20:57:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T20:57:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-06T20:57:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T20:57:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql 10 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-10-06T20:57:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-10-06T20:57:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:Install 128 Packages 2023-10-06T20:57:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 3 Packages 2023-10-06T20:57:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:57:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 208 M 2023-10-06T20:57:55.966 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-10-06T20:57:55.987 INFO:teuthology.orchestra.run.smithi167.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 1.8 MB/s | 183 kB 00:00 2023-10-06T20:57:56.061 INFO:teuthology.orchestra.run.smithi167.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 1.8 MB/s | 139 kB 00:00 2023-10-06T20:57:56.063 INFO:teuthology.orchestra.run.smithi078.stdout:(6/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 8.1 MB/s | 4.6 MB 00:00 2023-10-06T20:57:56.136 INFO:teuthology.orchestra.run.smithi167.stdout:(16/131): libradosstriper1-18.0.0-6616.g14d8d03 6.8 MB/s | 518 kB 00:00 2023-10-06T20:57:56.412 INFO:teuthology.orchestra.run.smithi167.stdout:(17/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 7.7 MB/s | 18 MB 00:02 2023-10-06T20:57:56.447 INFO:teuthology.orchestra.run.smithi078.stdout:(7/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 1.4 MB/s | 1.6 MB 00:01 2023-10-06T20:57:56.470 INFO:teuthology.orchestra.run.smithi167.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 771 kB/s | 45 kB 00:00 2023-10-06T20:57:56.479 INFO:teuthology.orchestra.run.smithi026.stdout:(1/131): ceph-18.0.0-6616.g14d8d032.el8.x86_64. 39 kB/s | 6.4 kB 00:00 2023-10-06T20:57:56.554 INFO:teuthology.orchestra.run.smithi167.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 1.5 MB/s | 126 kB 00:00 2023-10-06T20:57:56.638 INFO:teuthology.orchestra.run.smithi167.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 2.3 MB/s | 196 kB 00:00 2023-10-06T20:57:56.771 INFO:teuthology.orchestra.run.smithi167.stdout:(21/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 11 MB/s | 6.9 MB 00:00 2023-10-06T20:57:56.797 INFO:teuthology.orchestra.run.smithi167.stdout:(22/131): python3-rados-18.0.0-6616.g14d8d032.e 2.3 MB/s | 371 kB 00:00 2023-10-06T20:57:56.847 INFO:teuthology.orchestra.run.smithi167.stdout:(23/131): python3-rbd-18.0.0-6616.g14d8d032.el8 4.7 MB/s | 364 kB 00:00 2023-10-06T20:57:56.872 INFO:teuthology.orchestra.run.smithi167.stdout:(24/131): python3-rgw-18.0.0-6616.g14d8d032.el8 1.4 MB/s | 107 kB 00:00 2023-10-06T20:57:56.914 INFO:teuthology.orchestra.run.smithi167.stdout:(25/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-06T20:57:56.998 INFO:teuthology.orchestra.run.smithi167.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 2.1 MB/s | 179 kB 00:00 2023-10-06T20:57:57.046 INFO:teuthology.orchestra.run.smithi026.stdout:(2/131): ceph-fuse-18.0.0-6616.g14d8d032.el8.x8 1.6 MB/s | 918 kB 00:00 2023-10-06T20:57:57.081 INFO:teuthology.orchestra.run.smithi167.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 277 kB/s | 23 kB 00:00 2023-10-06T20:57:57.165 INFO:teuthology.orchestra.run.smithi167.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 1.5 MB/s | 132 kB 00:00 2023-10-06T20:57:57.197 INFO:teuthology.orchestra.run.smithi026.stdout:(3/131): ceph-immutable-object-cache-18.0.0-661 1.1 MB/s | 167 kB 00:00 2023-10-06T20:57:57.424 INFO:teuthology.orchestra.run.smithi167.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 6.8 MB/s | 1.8 MB 00:00 2023-10-06T20:57:57.483 INFO:teuthology.orchestra.run.smithi167.stdout:(30/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 5.9 MB/s | 3.6 MB 00:00 2023-10-06T20:57:57.574 INFO:teuthology.orchestra.run.smithi167.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 2.6 MB/s | 242 kB 00:00 2023-10-06T20:57:57.641 INFO:teuthology.orchestra.run.smithi167.stdout:(32/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 712 kB/s | 47 kB 00:00 2023-10-06T20:57:57.717 INFO:teuthology.orchestra.run.smithi167.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6616.g1 194 kB/s | 15 kB 00:00 2023-10-06T20:57:57.800 INFO:teuthology.orchestra.run.smithi167.stdout:(34/131): ceph-volume-18.0.0-6616.g14d8d032.el8 3.1 MB/s | 261 kB 00:00 2023-10-06T20:57:57.884 INFO:teuthology.orchestra.run.smithi167.stdout:(35/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 2.2 MB/s | 192 kB 00:00 2023-10-06T20:57:57.959 INFO:teuthology.orchestra.run.smithi167.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 242 kB/s | 18 kB 00:00 2023-10-06T20:57:57.997 INFO:teuthology.orchestra.run.smithi026.stdout:(4/131): ceph-mds-18.0.0-6616.g14d8d032.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-10-06T20:57:58.151 INFO:teuthology.orchestra.run.smithi167.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-10-06T20:57:58.185 INFO:teuthology.orchestra.run.smithi167.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-10-06T20:57:58.227 INFO:teuthology.orchestra.run.smithi167.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-10-06T20:57:58.388 INFO:teuthology.orchestra.run.smithi026.stdout:(5/131): ceph-mgr-18.0.0-6616.g14d8d032.el8.x86 4.2 MB/s | 1.6 MB 00:00 2023-10-06T20:57:58.444 INFO:teuthology.orchestra.run.smithi167.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 7.3 MB/s | 7.4 MB 00:01 2023-10-06T20:57:58.469 INFO:teuthology.orchestra.run.smithi167.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-10-06T20:57:58.536 INFO:teuthology.orchestra.run.smithi167.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-10-06T20:57:58.561 INFO:teuthology.orchestra.run.smithi167.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-10-06T20:57:58.587 INFO:teuthology.orchestra.run.smithi167.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-10-06T20:57:58.612 INFO:teuthology.orchestra.run.smithi167.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-10-06T20:57:58.662 INFO:teuthology.orchestra.run.smithi167.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-10-06T20:57:58.713 INFO:teuthology.orchestra.run.smithi167.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-10-06T20:57:58.755 INFO:teuthology.orchestra.run.smithi167.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.6 MB/s | 384 kB 00:00 2023-10-06T20:57:58.780 INFO:teuthology.orchestra.run.smithi167.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-10-06T20:57:58.806 INFO:teuthology.orchestra.run.smithi167.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2023-10-06T20:57:58.849 INFO:teuthology.orchestra.run.smithi026.stdout:(6/131): ceph-base-18.0.0-6616.g14d8d032.el8.x8 2.1 MB/s | 5.4 MB 00:02 2023-10-06T20:57:58.881 INFO:teuthology.orchestra.run.smithi167.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2023-10-06T20:57:58.923 INFO:teuthology.orchestra.run.smithi167.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-10-06T20:57:58.965 INFO:teuthology.orchestra.run.smithi167.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-10-06T20:57:59.040 INFO:teuthology.orchestra.run.smithi026.stdout:(7/131): ceph-mon-18.0.0-6616.g14d8d032.el8.x86 7.1 MB/s | 4.6 MB 00:00 2023-10-06T20:57:59.116 INFO:teuthology.orchestra.run.smithi167.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-10-06T20:57:59.165 INFO:teuthology.orchestra.run.smithi167.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 318 kB/s | 16 kB 00:00 2023-10-06T20:57:59.199 INFO:teuthology.orchestra.run.smithi167.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-10-06T20:57:59.214 INFO:teuthology.orchestra.run.smithi078.stdout:(8/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x86 5.7 MB/s | 18 MB 00:03 2023-10-06T20:57:59.224 INFO:teuthology.orchestra.run.smithi167.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 577 kB/s | 34 kB 00:00 2023-10-06T20:57:59.266 INFO:teuthology.orchestra.run.smithi167.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-10-06T20:57:59.289 INFO:teuthology.orchestra.run.smithi078.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 325 kB/s | 24 kB 00:00 2023-10-06T20:57:59.308 INFO:teuthology.orchestra.run.smithi167.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-10-06T20:57:59.333 INFO:teuthology.orchestra.run.smithi167.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-10-06T20:57:59.359 INFO:teuthology.orchestra.run.smithi167.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-10-06T20:57:59.384 INFO:teuthology.orchestra.run.smithi167.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-10-06T20:57:59.409 INFO:teuthology.orchestra.run.smithi167.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2023-10-06T20:57:59.510 INFO:teuthology.orchestra.run.smithi167.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-10-06T20:57:59.535 INFO:teuthology.orchestra.run.smithi167.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-10-06T20:57:59.569 INFO:teuthology.orchestra.run.smithi167.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-06T20:57:59.594 INFO:teuthology.orchestra.run.smithi167.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-10-06T20:57:59.620 INFO:teuthology.orchestra.run.smithi167.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2023-10-06T20:57:59.644 INFO:teuthology.orchestra.run.smithi167.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-06T20:57:59.987 INFO:teuthology.orchestra.run.smithi167.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 40 MB/s | 14 MB 00:00 2023-10-06T20:58:00.037 INFO:teuthology.orchestra.run.smithi167.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-10-06T20:58:00.087 INFO:teuthology.orchestra.run.smithi167.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-06T20:58:00.154 INFO:teuthology.orchestra.run.smithi167.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-10-06T20:58:00.205 INFO:teuthology.orchestra.run.smithi167.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-10-06T20:58:00.246 INFO:teuthology.orchestra.run.smithi167.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-10-06T20:58:00.281 INFO:teuthology.orchestra.run.smithi167.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 2.5 MB/s | 1.7 MB 00:00 2023-10-06T20:58:00.306 INFO:teuthology.orchestra.run.smithi026.stdout:(8/131): ceph-radosgw-18.0.0-6616.g14d8d032.el8 11 MB/s | 14 MB 00:01 2023-10-06T20:58:00.322 INFO:teuthology.orchestra.run.smithi167.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-10-06T20:58:00.365 INFO:teuthology.orchestra.run.smithi026.stdout:(9/131): ceph-selinux-18.0.0-6616.g14d8d032.el8 419 kB/s | 24 kB 00:00 2023-10-06T20:58:00.381 INFO:teuthology.orchestra.run.smithi167.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.9 MB/s | 157 kB 00:00 2023-10-06T20:58:00.414 INFO:teuthology.orchestra.run.smithi167.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.9 MB/s | 538 kB 00:00 2023-10-06T20:58:00.440 INFO:teuthology.orchestra.run.smithi167.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-06T20:58:00.473 INFO:teuthology.orchestra.run.smithi167.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-06T20:58:00.524 INFO:teuthology.orchestra.run.smithi167.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 530 kB/s | 27 kB 00:00 2023-10-06T20:58:00.607 INFO:teuthology.orchestra.run.smithi167.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-10-06T20:58:00.633 INFO:teuthology.orchestra.run.smithi167.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-10-06T20:58:00.658 INFO:teuthology.orchestra.run.smithi167.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 876 kB/s | 44 kB 00:00 2023-10-06T20:58:00.717 INFO:teuthology.orchestra.run.smithi167.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-10-06T20:58:00.784 INFO:teuthology.orchestra.run.smithi167.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-10-06T20:58:00.809 INFO:teuthology.orchestra.run.smithi167.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-10-06T20:58:00.834 INFO:teuthology.orchestra.run.smithi167.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-06T20:58:00.851 INFO:teuthology.orchestra.run.smithi167.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-10-06T20:58:00.885 INFO:teuthology.orchestra.run.smithi167.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-10-06T20:58:00.910 INFO:teuthology.orchestra.run.smithi167.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-10-06T20:58:00.952 INFO:teuthology.orchestra.run.smithi167.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-06T20:58:01.002 INFO:teuthology.orchestra.run.smithi167.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-10-06T20:58:01.313 INFO:teuthology.orchestra.run.smithi167.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-10-06T20:58:01.370 INFO:teuthology.orchestra.run.smithi167.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.5 MB/s | 202 kB 00:00 2023-10-06T20:58:01.453 INFO:teuthology.orchestra.run.smithi167.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2023-10-06T20:58:01.479 INFO:teuthology.orchestra.run.smithi167.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-10-06T20:58:01.537 INFO:teuthology.orchestra.run.smithi167.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-10-06T20:58:01.621 INFO:teuthology.orchestra.run.smithi167.stdout:(100/131): postgresql-server-10.21-2.module+el8 30 MB/s | 5.1 MB 00:00 2023-10-06T20:58:01.654 INFO:teuthology.orchestra.run.smithi167.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-10-06T20:58:01.696 INFO:teuthology.orchestra.run.smithi167.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-10-06T20:58:01.722 INFO:teuthology.orchestra.run.smithi167.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-10-06T20:58:01.739 INFO:teuthology.orchestra.run.smithi167.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-06T20:58:01.766 INFO:teuthology.orchestra.run.smithi167.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-10-06T20:58:01.781 INFO:teuthology.orchestra.run.smithi167.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-10-06T20:58:01.807 INFO:teuthology.orchestra.run.smithi167.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-10-06T20:58:01.824 INFO:teuthology.orchestra.run.smithi167.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-10-06T20:58:01.849 INFO:teuthology.orchestra.run.smithi167.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-10-06T20:58:01.874 INFO:teuthology.orchestra.run.smithi167.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-06T20:58:01.908 INFO:teuthology.orchestra.run.smithi167.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-10-06T20:58:01.933 INFO:teuthology.orchestra.run.smithi167.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-10-06T20:58:01.958 INFO:teuthology.orchestra.run.smithi167.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-10-06T20:58:01.992 INFO:teuthology.orchestra.run.smithi167.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-10-06T20:58:02.017 INFO:teuthology.orchestra.run.smithi167.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-10-06T20:58:02.051 INFO:teuthology.orchestra.run.smithi167.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-10-06T20:58:02.076 INFO:teuthology.orchestra.run.smithi167.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-06T20:58:02.110 INFO:teuthology.orchestra.run.smithi167.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-10-06T20:58:02.160 INFO:teuthology.orchestra.run.smithi167.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 21 MB/s | 2.3 MB 00:00 2023-10-06T20:58:02.177 INFO:teuthology.orchestra.run.smithi167.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-10-06T20:58:02.203 INFO:teuthology.orchestra.run.smithi167.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-10-06T20:58:02.228 INFO:teuthology.orchestra.run.smithi167.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-06T20:58:02.253 INFO:teuthology.orchestra.run.smithi167.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-06T20:58:02.299 INFO:teuthology.orchestra.run.smithi167.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 543 kB/s | 39 kB 00:00 2023-10-06T20:58:02.345 INFO:teuthology.orchestra.run.smithi167.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2023-10-06T20:58:02.370 INFO:teuthology.orchestra.run.smithi167.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.3 MB/s | 171 kB 00:00 2023-10-06T20:58:02.396 INFO:teuthology.orchestra.run.smithi167.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-06T20:58:03.540 INFO:teuthology.orchestra.run.smithi026.stdout:(10/131): ceph-osd-18.0.0-6616.g14d8d032.el8.x8 3.8 MB/s | 18 MB 00:04 2023-10-06T20:58:03.681 INFO:teuthology.orchestra.run.smithi026.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 1.8 MB/s | 260 kB 00:00 2023-10-06T20:58:04.038 INFO:teuthology.orchestra.run.smithi167.stdout:(128/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 2.3 MB/s | 3.7 MB 00:01 2023-10-06T20:58:04.088 INFO:teuthology.orchestra.run.smithi167.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-06T20:58:04.157 INFO:teuthology.orchestra.run.smithi026.stdout:(12/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 13 MB/s | 49 MB 00:03 2023-10-06T20:58:04.182 INFO:teuthology.orchestra.run.smithi026.stdout:(13/131): libcephfs-devel-18.0.0-6616.g14d8d032 62 kB/s | 31 kB 00:00 2023-10-06T20:58:04.223 INFO:teuthology.orchestra.run.smithi078.stdout:(10/131): ceph-radosgw-18.0.0-6616.g14d8d032.el 1.8 MB/s | 14 MB 00:07 2023-10-06T20:58:04.314 INFO:teuthology.orchestra.run.smithi078.stdout:(11/131): cephfs-mirror-18.0.0-6616.g14d8d032.e 2.8 MB/s | 260 kB 00:00 2023-10-06T20:58:04.356 INFO:teuthology.orchestra.run.smithi078.stdout:(12/131): libcephfs-devel-18.0.0-6616.g14d8d032 747 kB/s | 31 kB 00:00 2023-10-06T20:58:04.472 INFO:teuthology.orchestra.run.smithi167.stdout:(130/131): librados2-18.0.0-6616.g14d8d032.el8. 1.8 MB/s | 3.7 MB 00:02 2023-10-06T20:58:04.557 INFO:teuthology.orchestra.run.smithi078.stdout:(13/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 4.1 MB/s | 840 kB 00:00 2023-10-06T20:58:04.582 INFO:teuthology.orchestra.run.smithi026.stdout:(14/131): ceph-common-18.0.0-6616.g14d8d032.el8 2.7 MB/s | 23 MB 00:08 2023-10-06T20:58:04.608 INFO:teuthology.orchestra.run.smithi026.stdout:(15/131): libcephsqlite-18.0.0-6616.g14d8d032.e 432 kB/s | 183 kB 00:00 2023-10-06T20:58:04.641 INFO:teuthology.orchestra.run.smithi026.stdout:(16/131): libcephfs2-18.0.0-6616.g14d8d032.el8. 1.7 MB/s | 840 kB 00:00 2023-10-06T20:58:04.667 INFO:teuthology.orchestra.run.smithi026.stdout:(17/131): librados-devel-18.0.0-6616.g14d8d032. 1.6 MB/s | 139 kB 00:00 2023-10-06T20:58:04.734 INFO:teuthology.orchestra.run.smithi026.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 670 kB/s | 45 kB 00:00 2023-10-06T20:58:04.792 INFO:teuthology.orchestra.run.smithi026.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 2.1 MB/s | 126 kB 00:00 2023-10-06T20:58:04.824 INFO:teuthology.orchestra.run.smithi078.stdout:(14/131): libcephsqlite-18.0.0-6616.g14d8d032.e 689 kB/s | 183 kB 00:00 2023-10-06T20:58:04.901 INFO:teuthology.orchestra.run.smithi026.stdout:(20/131): libradosstriper1-18.0.0-6616.g14d8d03 1.7 MB/s | 518 kB 00:00 2023-10-06T20:58:04.926 INFO:teuthology.orchestra.run.smithi026.stdout:(21/131): python3-cephfs-18.0.0-6616.g14d8d032. 1.4 MB/s | 196 kB 00:00 2023-10-06T20:58:04.949 INFO:teuthology.orchestra.run.smithi078.stdout:(15/131): librados-devel-18.0.0-6616.g14d8d032. 1.1 MB/s | 139 kB 00:00 2023-10-06T20:58:05.143 INFO:teuthology.orchestra.run.smithi026.stdout:(22/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 14 MB/s | 6.9 MB 00:00 2023-10-06T20:58:05.150 INFO:teuthology.orchestra.run.smithi078.stdout:(16/131): libradosstriper1-18.0.0-6616.g14d8d03 2.5 MB/s | 518 kB 00:00 2023-10-06T20:58:05.177 INFO:teuthology.orchestra.run.smithi026.stdout:(23/131): python3-rbd-18.0.0-6616.g14d8d032.el8 1.4 MB/s | 364 kB 00:00 2023-10-06T20:58:05.202 INFO:teuthology.orchestra.run.smithi026.stdout:(24/131): python3-rados-18.0.0-6616.g14d8d032.e 1.2 MB/s | 371 kB 00:00 2023-10-06T20:58:05.227 INFO:teuthology.orchestra.run.smithi026.stdout:(25/131): python3-rgw-18.0.0-6616.g14d8d032.el8 1.2 MB/s | 107 kB 00:00 2023-10-06T20:58:05.244 INFO:teuthology.orchestra.run.smithi026.stdout:(26/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-06T20:58:05.279 INFO:teuthology.orchestra.run.smithi026.stdout:(27/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 3.5 MB/s | 179 kB 00:00 2023-10-06T20:58:05.295 INFO:teuthology.orchestra.run.smithi026.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6616.g 461 kB/s | 23 kB 00:00 2023-10-06T20:58:05.337 INFO:teuthology.orchestra.run.smithi026.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 2.2 MB/s | 132 kB 00:00 2023-10-06T20:58:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-10-06T20:58:06.046 INFO:teuthology.orchestra.run.smithi026.stdout:(31/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 2.3 MB/s | 1.8 MB 00:00 2023-10-06T20:58:06.071 INFO:teuthology.orchestra.run.smithi026.stdout:(32/131): ceph-mgr-modules-core-18.0.0-6616.g14 2.6 MB/s | 242 kB 00:00 2023-10-06T20:58:06.096 INFO:teuthology.orchestra.run.smithi026.stdout:(33/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 953 kB/s | 47 kB 00:00 2023-10-06T20:58:06.121 INFO:teuthology.orchestra.run.smithi026.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6616.g1 291 kB/s | 15 kB 00:00 2023-10-06T20:58:06.147 INFO:teuthology.orchestra.run.smithi026.stdout:(35/131): ceph-volume-18.0.0-6616.g14d8d032.el8 5.1 MB/s | 261 kB 00:00 2023-10-06T20:58:06.197 INFO:teuthology.orchestra.run.smithi026.stdout:(36/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 363 kB/s | 18 kB 00:00 2023-10-06T20:58:06.222 INFO:teuthology.orchestra.run.smithi026.stdout:(37/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 1.9 MB/s | 192 kB 00:00 2023-10-06T20:58:06.322 INFO:teuthology.orchestra.run.smithi026.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 592 kB/s | 59 kB 00:00 2023-10-06T20:58:06.365 INFO:teuthology.orchestra.run.smithi026.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-10-06T20:58:06.390 INFO:teuthology.orchestra.run.smithi026.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-06T20:58:06.432 INFO:teuthology.orchestra.run.smithi026.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.0 MB/s | 476 kB 00:00 2023-10-06T20:58:06.457 INFO:teuthology.orchestra.run.smithi026.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 653 kB/s | 44 kB 00:00 2023-10-06T20:58:06.516 INFO:teuthology.orchestra.run.smithi026.stdout:(43/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 2.7 MB/s | 3.6 MB 00:01 2023-10-06T20:58:06.558 INFO:teuthology.orchestra.run.smithi026.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-10-06T20:58:06.616 INFO:teuthology.orchestra.run.smithi026.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-10-06T20:58:06.641 INFO:teuthology.orchestra.run.smithi026.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-10-06T20:58:06.667 INFO:teuthology.orchestra.run.smithi026.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-10-06T20:58:06.692 INFO:teuthology.orchestra.run.smithi026.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-10-06T20:58:06.718 INFO:teuthology.orchestra.run.smithi026.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-10-06T20:58:06.733 INFO:teuthology.orchestra.run.smithi078.stdout:(17/131): librgw2-18.0.0-6616.g14d8d032.el8.x86 4.4 MB/s | 6.9 MB 00:01 2023-10-06T20:58:06.743 INFO:teuthology.orchestra.run.smithi026.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-10-06T20:58:06.768 INFO:teuthology.orchestra.run.smithi026.stdout:(51/131): python3-google-auth-1.1.1-10.el8.noar 644 kB/s | 81 kB 00:00 2023-10-06T20:58:06.775 INFO:teuthology.orchestra.run.smithi078.stdout:(18/131): python3-ceph-argparse-18.0.0-6616.g14 1.1 MB/s | 45 kB 00:00 2023-10-06T20:58:06.794 INFO:teuthology.orchestra.run.smithi026.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 972 kB/s | 49 kB 00:00 2023-10-06T20:58:06.817 INFO:teuthology.orchestra.run.smithi078.stdout:(19/131): python3-ceph-common-18.0.0-6616.g14d8 2.9 MB/s | 126 kB 00:00 2023-10-06T20:58:06.819 INFO:teuthology.orchestra.run.smithi026.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-10-06T20:58:06.844 INFO:teuthology.orchestra.run.smithi026.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-06T20:58:06.884 INFO:teuthology.orchestra.run.smithi078.stdout:(20/131): python3-cephfs-18.0.0-6616.g14d8d032. 2.9 MB/s | 196 kB 00:00 2023-10-06T20:58:06.895 INFO:teuthology.orchestra.run.smithi026.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-10-06T20:58:06.920 INFO:teuthology.orchestra.run.smithi026.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-10-06T20:58:06.945 INFO:teuthology.orchestra.run.smithi026.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-10-06T20:58:06.970 INFO:teuthology.orchestra.run.smithi026.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-10-06T20:58:06.984 INFO:teuthology.orchestra.run.smithi078.stdout:(21/131): python3-rados-18.0.0-6616.g14d8d032.e 3.6 MB/s | 371 kB 00:00 2023-10-06T20:58:06.995 INFO:teuthology.orchestra.run.smithi026.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-10-06T20:58:07.021 INFO:teuthology.orchestra.run.smithi026.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-10-06T20:58:07.037 INFO:teuthology.orchestra.run.smithi026.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-10-06T20:58:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-10-06T20:58:07.071 INFO:teuthology.orchestra.run.smithi026.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 878 kB/s | 29 kB 00:00 2023-10-06T20:58:07.084 INFO:teuthology.orchestra.run.smithi078.stdout:(22/131): python3-rbd-18.0.0-6616.g14d8d032.el8 3.6 MB/s | 364 kB 00:00 2023-10-06T20:58:07.088 INFO:teuthology.orchestra.run.smithi026.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 551 kB/s | 27 kB 00:00 2023-10-06T20:58:07.114 INFO:teuthology.orchestra.run.smithi026.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2023-10-06T20:58:07.131 INFO:teuthology.orchestra.run.smithi026.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-10-06T20:58:07.189 INFO:teuthology.orchestra.run.smithi026.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-10-06T20:58:07.206 INFO:teuthology.orchestra.run.smithi026.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-10-06T20:58:07.231 INFO:teuthology.orchestra.run.smithi026.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 728 kB/s | 85 kB 00:00 2023-10-06T20:58:07.257 INFO:teuthology.orchestra.run.smithi026.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-10-06T20:58:07.293 INFO:teuthology.orchestra.run.smithi078.stdout:(23/131): ceph-common-18.0.0-6616.g14d8d032.el8 1.7 MB/s | 23 MB 00:13 2023-10-06T20:58:07.315 INFO:teuthology.orchestra.run.smithi026.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-10-06T20:58:07.338 INFO:teuthology.orchestra.run.smithi078.stdout:(24/131): python3-rgw-18.0.0-6616.g14d8d032.el8 420 kB/s | 107 kB 00:00 2023-10-06T20:58:07.365 INFO:teuthology.orchestra.run.smithi026.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-10-06T20:58:07.369 INFO:teuthology.orchestra.run.smithi078.stdout:(25/131): rbd-fuse-18.0.0-6616.g14d8d032.el8.x8 1.2 MB/s | 94 kB 00:00 2023-10-06T20:58:07.391 INFO:teuthology.orchestra.run.smithi026.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 816 kB/s | 54 kB 00:00 2023-10-06T20:58:07.424 INFO:teuthology.orchestra.run.smithi026.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-06T20:58:07.444 INFO:teuthology.orchestra.run.smithi078.stdout:(26/131): rbd-nbd-18.0.0-6616.g14d8d032.el8.x86 2.3 MB/s | 179 kB 00:00 2023-10-06T20:58:07.458 INFO:teuthology.orchestra.run.smithi026.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-06T20:58:07.475 INFO:teuthology.orchestra.run.smithi026.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 776 kB/s | 39 kB 00:00 2023-10-06T20:58:07.525 INFO:teuthology.orchestra.run.smithi026.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-06T20:58:07.527 INFO:teuthology.orchestra.run.smithi078.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6616.g 277 kB/s | 23 kB 00:00 2023-10-06T20:58:07.567 INFO:teuthology.orchestra.run.smithi026.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.7 MB/s | 538 kB 00:00 2023-10-06T20:58:07.592 INFO:teuthology.orchestra.run.smithi026.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-10-06T20:58:07.603 INFO:teuthology.orchestra.run.smithi078.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6616.g14d8d03 1.7 MB/s | 132 kB 00:00 2023-10-06T20:58:07.626 INFO:teuthology.orchestra.run.smithi026.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-10-06T20:58:07.685 INFO:teuthology.orchestra.run.smithi026.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-10-06T20:58:07.769 INFO:teuthology.orchestra.run.smithi026.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 318 kB/s | 27 kB 00:00 2023-10-06T20:58:07.819 INFO:teuthology.orchestra.run.smithi026.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-06T20:58:07.861 INFO:teuthology.orchestra.run.smithi026.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-10-06T20:58:07.953 INFO:teuthology.orchestra.run.smithi078.stdout:(29/131): rbd-mirror-18.0.0-6616.g14d8d032.el8. 5.8 MB/s | 3.6 MB 00:00 2023-10-06T20:58:07.995 INFO:teuthology.orchestra.run.smithi078.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6616.g14d8d 4.5 MB/s | 1.8 MB 00:00 2023-10-06T20:58:08.011 INFO:teuthology.orchestra.run.smithi026.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-10-06T20:58:08.061 INFO:teuthology.orchestra.run.smithi026.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-10-06T20:58:08.079 INFO:teuthology.orchestra.run.smithi078.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6616.g14 2.8 MB/s | 242 kB 00:00 2023-10-06T20:58:08.137 INFO:teuthology.orchestra.run.smithi026.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-10-06T20:58:08.162 INFO:teuthology.orchestra.run.smithi026.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-10-06T20:58:08.187 INFO:teuthology.orchestra.run.smithi026.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-10-06T20:58:08.213 INFO:teuthology.orchestra.run.smithi026.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-06T20:58:08.237 INFO:teuthology.orchestra.run.smithi026.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 400 kB/s | 20 kB 00:00 2023-10-06T20:58:08.263 INFO:teuthology.orchestra.run.smithi026.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-06T20:58:08.344 INFO:teuthology.orchestra.run.smithi026.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 6.5 MB/s | 709 kB 00:00 2023-10-06T20:58:08.397 INFO:teuthology.orchestra.run.smithi026.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.7 MB/s | 198 kB 00:00 2023-10-06T20:58:08.571 INFO:teuthology.orchestra.run.smithi078.stdout:(32/131): ceph-test-18.0.0-6616.g14d8d032.el8.x 5.3 MB/s | 49 MB 00:09 2023-10-06T20:58:08.596 INFO:teuthology.orchestra.run.smithi078.stdout:(33/131): ceph-mgr-rook-18.0.0-6616.g14d8d032.e 92 kB/s | 47 kB 00:00 2023-10-06T20:58:08.597 INFO:teuthology.orchestra.run.smithi026.stdout:(95/131): python3-scipy-1.0.0-21.module+el8.5.0 10 MB/s | 14 MB 00:01 2023-10-06T20:58:08.621 INFO:teuthology.orchestra.run.smithi078.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6616.g1 291 kB/s | 15 kB 00:00 2023-10-06T20:58:08.639 INFO:teuthology.orchestra.run.smithi026.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-06T20:58:08.689 INFO:teuthology.orchestra.run.smithi078.stdout:(35/131): ceph-volume-18.0.0-6616.g14d8d032.el8 2.8 MB/s | 261 kB 00:00 2023-10-06T20:58:08.689 INFO:teuthology.orchestra.run.smithi026.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-10-06T20:58:08.730 INFO:teuthology.orchestra.run.smithi078.stdout:(36/131): cephadm-18.0.0-6616.g14d8d032.el8.noa 1.7 MB/s | 192 kB 00:00 2023-10-06T20:58:08.756 INFO:teuthology.orchestra.run.smithi078.stdout:(37/131): cephfs-top-18.0.0-6616.g14d8d032.el8. 271 kB/s | 18 kB 00:00 2023-10-06T20:58:08.764 INFO:teuthology.orchestra.run.smithi026.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 9.7 MB/s | 4.9 MB 00:00 2023-10-06T20:58:08.865 INFO:teuthology.orchestra.run.smithi026.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2023-10-06T20:58:08.873 INFO:teuthology.orchestra.run.smithi078.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 504 kB/s | 59 kB 00:00 2023-10-06T20:58:08.940 INFO:teuthology.orchestra.run.smithi026.stdout:(100/131): postgresql-contrib-10.21-2.module+el 4.5 MB/s | 808 kB 00:00 2023-10-06T20:58:08.955 INFO:teuthology.orchestra.run.smithi078.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-10-06T20:58:08.982 INFO:teuthology.orchestra.run.smithi078.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 691 kB/s | 75 kB 00:00 2023-10-06T20:58:08.982 INFO:teuthology.orchestra.run.smithi026.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-10-06T20:58:09.024 INFO:teuthology.orchestra.run.smithi026.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-10-06T20:58:09.032 INFO:teuthology.orchestra.run.smithi078.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 6.1 MB/s | 476 kB 00:00 2023-10-06T20:58:09.049 INFO:teuthology.orchestra.run.smithi078.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 651 kB/s | 44 kB 00:00 2023-10-06T20:58:09.066 INFO:teuthology.orchestra.run.smithi026.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 7.5 MB/s | 1.5 MB 00:00 2023-10-06T20:58:09.083 INFO:teuthology.orchestra.run.smithi078.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-10-06T20:58:09.091 INFO:teuthology.orchestra.run.smithi026.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2023-10-06T20:58:09.099 INFO:teuthology.orchestra.run.smithi078.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 655 kB/s | 33 kB 00:00 2023-10-06T20:58:09.116 INFO:teuthology.orchestra.run.smithi078.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 487 kB/s | 16 kB 00:00 2023-10-06T20:58:09.125 INFO:teuthology.orchestra.run.smithi026.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-10-06T20:58:09.150 INFO:teuthology.orchestra.run.smithi026.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 823 kB/s | 48 kB 00:00 2023-10-06T20:58:09.167 INFO:teuthology.orchestra.run.smithi078.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-10-06T20:58:09.175 INFO:teuthology.orchestra.run.smithi026.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-06T20:58:09.201 INFO:teuthology.orchestra.run.smithi078.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-10-06T20:58:09.201 INFO:teuthology.orchestra.run.smithi026.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 847 kB/s | 43 kB 00:00 2023-10-06T20:58:09.226 INFO:teuthology.orchestra.run.smithi078.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-06T20:58:09.226 INFO:teuthology.orchestra.run.smithi026.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-06T20:58:09.251 INFO:teuthology.orchestra.run.smithi078.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-06T20:58:09.259 INFO:teuthology.orchestra.run.smithi026.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-10-06T20:58:09.276 INFO:teuthology.orchestra.run.smithi078.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-10-06T20:58:09.285 INFO:teuthology.orchestra.run.smithi026.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 758 kB/s | 19 kB 00:00 2023-10-06T20:58:09.327 INFO:teuthology.orchestra.run.smithi026.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.4 MB/s | 560 kB 00:00 2023-10-06T20:58:09.350 INFO:teuthology.orchestra.run.smithi078.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 661 kB/s | 49 kB 00:00 2023-10-06T20:58:09.352 INFO:teuthology.orchestra.run.smithi026.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.2 MB/s | 155 kB 00:00 2023-10-06T20:58:09.386 INFO:teuthology.orchestra.run.smithi026.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 541 kB/s | 18 kB 00:00 2023-10-06T20:58:09.402 INFO:teuthology.orchestra.run.smithi078.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 7.7 MB/s | 1.2 MB 00:00 2023-10-06T20:58:09.419 INFO:teuthology.orchestra.run.smithi026.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 5.9 MB/s | 559 kB 00:00 2023-10-06T20:58:09.427 INFO:teuthology.orchestra.run.smithi078.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 767 kB/s | 59 kB 00:00 2023-10-06T20:58:09.528 INFO:teuthology.orchestra.run.smithi078.stdout:(54/131): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-10-06T20:58:09.553 INFO:teuthology.orchestra.run.smithi078.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 400 kB/s | 60 kB 00:00 2023-10-06T20:58:09.587 INFO:teuthology.orchestra.run.smithi078.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-10-06T20:58:09.612 INFO:teuthology.orchestra.run.smithi078.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-10-06T20:58:09.637 INFO:teuthology.orchestra.run.smithi078.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-06T20:58:09.662 INFO:teuthology.orchestra.run.smithi078.stdout:(59/131): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2023-10-06T20:58:09.688 INFO:teuthology.orchestra.run.smithi078.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-10-06T20:58:09.702 INFO:teuthology.orchestra.run.smithi026.stdout:(116/131): postgresql-server-10.21-2.module+el8 5.0 MB/s | 5.1 MB 00:01 2023-10-06T20:58:09.729 INFO:teuthology.orchestra.run.smithi026.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 770 kB/s | 238 kB 00:00 2023-10-06T20:58:09.746 INFO:teuthology.orchestra.run.smithi078.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 481 kB/s | 64 kB 00:00 2023-10-06T20:58:09.762 INFO:teuthology.orchestra.run.smithi026.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.6 MB/s | 605 kB 00:00 2023-10-06T20:58:09.787 INFO:teuthology.orchestra.run.smithi026.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-10-06T20:58:09.788 INFO:teuthology.orchestra.run.smithi078.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 190 kB/s | 24 kB 00:00 2023-10-06T20:58:09.813 INFO:teuthology.orchestra.run.smithi026.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-06T20:58:09.813 INFO:teuthology.orchestra.run.smithi078.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-10-06T20:58:09.838 INFO:teuthology.orchestra.run.smithi026.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-06T20:58:09.839 INFO:teuthology.orchestra.run.smithi078.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 296 kB/s | 27 kB 00:00 2023-10-06T20:58:09.863 INFO:teuthology.orchestra.run.smithi026.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-06T20:58:09.873 INFO:teuthology.orchestra.run.smithi078.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-06T20:58:09.888 INFO:teuthology.orchestra.run.smithi026.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-06T20:58:09.898 INFO:teuthology.orchestra.run.smithi078.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-10-06T20:58:09.914 INFO:teuthology.orchestra.run.smithi026.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-10-06T20:58:09.923 INFO:teuthology.orchestra.run.smithi078.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 732 kB/s | 61 kB 00:00 2023-10-06T20:58:09.949 INFO:teuthology.orchestra.run.smithi078.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-10-06T20:58:09.964 INFO:teuthology.orchestra.run.smithi026.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-10-06T20:58:09.965 INFO:teuthology.orchestra.run.smithi078.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 924 kB/s | 85 kB 00:00 2023-10-06T20:58:09.982 INFO:teuthology.orchestra.run.smithi078.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-10-06T20:58:09.989 INFO:teuthology.orchestra.run.smithi026.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 171 kB 00:00 2023-10-06T20:58:10.014 INFO:teuthology.orchestra.run.smithi026.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-06T20:58:10.033 INFO:teuthology.orchestra.run.smithi078.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.4 MB/s | 110 kB 00:00 2023-10-06T20:58:10.067 INFO:teuthology.orchestra.run.smithi078.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-06T20:58:10.081 INFO:teuthology.orchestra.run.smithi026.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-10-06T20:58:10.132 INFO:teuthology.orchestra.run.smithi026.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-06T20:58:10.133 INFO:teuthology.orchestra.run.smithi078.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2023-10-06T20:58:10.167 INFO:teuthology.orchestra.run.smithi078.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2023-10-06T20:58:10.217 INFO:teuthology.orchestra.run.smithi078.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 874 kB/s | 39 kB 00:00 2023-10-06T20:58:10.317 INFO:teuthology.orchestra.run.smithi078.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2023-10-06T20:58:10.365 INFO:teuthology.orchestra.run.smithi026.stdout:(130/131): librados2-18.0.0-6616.g14d8d032.el8. 10 MB/s | 3.7 MB 00:00 2023-10-06T20:58:10.576 INFO:teuthology.orchestra.run.smithi078.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 1.9 MB/s | 688 kB 00:00 2023-10-06T20:58:10.610 INFO:teuthology.orchestra.run.smithi078.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 1.8 MB/s | 538 kB 00:00 2023-10-06T20:58:10.635 INFO:teuthology.orchestra.run.smithi078.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-06T20:58:10.660 INFO:teuthology.orchestra.run.smithi078.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-06T20:58:10.702 INFO:teuthology.orchestra.run.smithi078.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-06T20:58:10.728 INFO:teuthology.orchestra.run.smithi078.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-10-06T20:58:10.770 INFO:teuthology.orchestra.run.smithi078.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-06T20:58:10.803 INFO:teuthology.orchestra.run.smithi078.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-06T20:58:10.924 INFO:teuthology.orchestra.run.smithi078.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-10-06T20:58:10.979 INFO:teuthology.orchestra.run.smithi078.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.2 MB/s | 457 kB 00:00 2023-10-06T20:58:11.049 INFO:teuthology.orchestra.run.smithi026.stdout:(131/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 3.6 MB/s | 3.7 MB 00:01 2023-10-06T20:58:11.052 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:58:11.052 INFO:teuthology.orchestra.run.smithi026.stdout:Total 14 MB/s | 208 MB 00:15 2023-10-06T20:58:11.121 INFO:teuthology.orchestra.run.smithi078.stdout:(87/131): python3-scipy-1.0.0-21.module+el8.5.0 12 MB/s | 14 MB 00:01 2023-10-06T20:58:11.146 INFO:teuthology.orchestra.run.smithi078.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-10-06T20:58:11.172 INFO:teuthology.orchestra.run.smithi078.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-06T20:58:11.197 INFO:teuthology.orchestra.run.smithi078.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-06T20:58:11.230 INFO:teuthology.orchestra.run.smithi078.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 341 kB/s | 20 kB 00:00 2023-10-06T20:58:11.256 INFO:teuthology.orchestra.run.smithi078.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-10-06T20:58:11.356 INFO:teuthology.orchestra.run.smithi078.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 6.8 MB/s | 3.7 MB 00:00 2023-10-06T20:58:11.390 INFO:teuthology.orchestra.run.smithi078.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2023-10-06T20:58:11.415 INFO:teuthology.orchestra.run.smithi078.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-10-06T20:58:11.457 INFO:teuthology.orchestra.run.smithi078.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-06T20:58:11.499 INFO:teuthology.orchestra.run.smithi078.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-06T20:58:11.532 INFO:teuthology.orchestra.run.smithi167.stdout:(131/131): ceph-test-18.0.0-6616.g14d8d032.el8. 3.0 MB/s | 49 MB 00:16 2023-10-06T20:58:11.535 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:58:11.536 INFO:teuthology.orchestra.run.smithi167.stdout:Total 11 MB/s | 208 MB 00:19 2023-10-06T20:58:11.599 INFO:teuthology.orchestra.run.smithi078.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-10-06T20:58:11.683 INFO:teuthology.orchestra.run.smithi078.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-10-06T20:58:11.741 INFO:teuthology.orchestra.run.smithi078.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-10-06T20:58:11.766 INFO:teuthology.orchestra.run.smithi078.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-10-06T20:58:11.809 INFO:teuthology.orchestra.run.smithi078.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-10-06T20:58:11.834 INFO:teuthology.orchestra.run.smithi078.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-10-06T20:58:11.859 INFO:teuthology.orchestra.run.smithi078.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-10-06T20:58:11.885 INFO:teuthology.orchestra.run.smithi078.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-06T20:58:11.910 INFO:teuthology.orchestra.run.smithi078.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-10-06T20:58:11.935 INFO:teuthology.orchestra.run.smithi078.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-06T20:58:11.960 INFO:teuthology.orchestra.run.smithi078.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-10-06T20:58:11.986 INFO:teuthology.orchestra.run.smithi078.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-06T20:58:12.053 INFO:teuthology.orchestra.run.smithi078.stdout:(110/131): postgresql-server-10.21-2.module+el8 9.1 MB/s | 5.1 MB 00:00 2023-10-06T20:58:12.110 INFO:teuthology.orchestra.run.smithi078.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-10-06T20:58:12.137 INFO:teuthology.orchestra.run.smithi078.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 224 kB/s | 19 kB 00:00 2023-10-06T20:58:12.162 INFO:teuthology.orchestra.run.smithi078.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.9 MB/s | 155 kB 00:00 2023-10-06T20:58:12.196 INFO:teuthology.orchestra.run.smithi078.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.6 MB/s | 560 kB 00:00 2023-10-06T20:58:12.221 INFO:teuthology.orchestra.run.smithi078.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-10-06T20:58:12.255 INFO:teuthology.orchestra.run.smithi078.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.6 MB/s | 559 kB 00:00 2023-10-06T20:58:12.280 INFO:teuthology.orchestra.run.smithi078.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-06T20:58:12.314 INFO:teuthology.orchestra.run.smithi078.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-10-06T20:58:12.347 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-06T20:58:12.347 INFO:teuthology.orchestra.run.smithi078.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.1 MB/s | 72 kB 00:00 2023-10-06T20:58:12.372 INFO:teuthology.orchestra.run.smithi078.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-10-06T20:58:12.398 INFO:teuthology.orchestra.run.smithi078.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-06T20:58:12.423 INFO:teuthology.orchestra.run.smithi078.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-06T20:58:12.448 INFO:teuthology.orchestra.run.smithi078.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-06T20:58:12.466 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-06T20:58:12.466 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-06T20:58:12.497 INFO:teuthology.orchestra.run.smithi078.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 527 kB/s | 39 kB 00:00 2023-10-06T20:58:12.541 INFO:teuthology.orchestra.run.smithi078.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.9 MB/s | 644 kB 00:00 2023-10-06T20:58:12.566 INFO:teuthology.orchestra.run.smithi078.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.4 MB/s | 171 kB 00:00 2023-10-06T20:58:12.591 INFO:teuthology.orchestra.run.smithi078.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-06T20:58:12.641 INFO:teuthology.orchestra.run.smithi078.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-10-06T20:58:12.700 INFO:teuthology.orchestra.run.smithi078.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-06T20:58:12.726 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-10-06T20:58:12.871 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-10-06T20:58:12.871 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-10-06T20:58:13.042 INFO:teuthology.orchestra.run.smithi078.stdout:(130/131): librbd1-18.0.0-6616.g14d8d032.el8.x8 8.2 MB/s | 3.7 MB 00:00 2023-10-06T20:58:13.150 INFO:teuthology.orchestra.run.smithi078.stdout:(131/131): librados2-18.0.0-6616.g14d8d032.el8. 6.4 MB/s | 3.7 MB 00:00 2023-10-06T20:58:13.154 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2023-10-06T20:58:13.154 INFO:teuthology.orchestra.run.smithi078.stdout:Total 11 MB/s | 208 MB 00:19 2023-10-06T20:58:13.586 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-06T20:58:13.590 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-06T20:58:13.991 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2023-10-06T20:58:14.110 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2023-10-06T20:58:14.110 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2023-10-06T20:58:14.487 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-10-06T20:58:14.494 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-10-06T20:58:15.242 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2023-10-06T20:58:15.246 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2023-10-06T20:58:18.593 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-06T20:58:18.801 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T20:58:18.943 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T20:58:19.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-06T20:58:19.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:19.406 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:19.661 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:19.866 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-10-06T20:58:20.067 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:20.116 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T20:58:20.260 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:20.281 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T20:58:20.332 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2023-10-06T20:58:20.504 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-06T20:58:20.538 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-06T20:58:20.653 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:20.658 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:20.699 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-06T20:58:20.794 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:20.844 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-06T20:58:20.977 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:21.033 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:21.127 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:21.227 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:21.386 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:58:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-06T20:58:21.523 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:21.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T20:58:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:21.783 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:21.847 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T20:58:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-06T20:58:22.268 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:22.282 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:22.337 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T20:58:22.416 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:22.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T20:58:22.574 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:22.681 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:58:22.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:22.802 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:22.907 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:22.911 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:22.960 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:23.023 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:58:23.041 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:23.094 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:58:23.168 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:23.190 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:23.304 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:23.315 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:58:23.319 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T20:58:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:23.492 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:58:23.530 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T20:58:23.744 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-06T20:58:23.943 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-06T20:58:23.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T20:58:24.055 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:24.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T20:58:24.244 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:24.316 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:24.362 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-06T20:58:24.431 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:24.630 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-06T20:58:24.677 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:24.765 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:24.807 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:24.839 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:24.916 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 14/134 2023-10-06T20:58:25.048 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:25.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:25.142 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 15/134 2023-10-06T20:58:25.162 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:25.298 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:25.307 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:25.444 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-06T20:58:25.632 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:25.752 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:26.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-06T20:58:26.036 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:26.237 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-06T20:58:26.253 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-06T20:58:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:26.472 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-06T20:58:26.632 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-06T20:58:26.832 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-06T20:58:26.933 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-06T20:58:27.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T20:58:27.115 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:27.233 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T20:58:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:27.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T20:58:27.923 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:28.009 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-06T20:58:28.230 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-06T20:58:28.240 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T20:58:28.265 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-06T20:58:28.484 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-06T20:58:28.514 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-06T20:58:28.550 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-06T20:58:28.702 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-06T20:58:28.748 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-06T20:58:28.837 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-06T20:58:28.904 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-06T20:58:28.942 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-06T20:58:29.112 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-06T20:58:29.114 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-06T20:58:29.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-06T20:58:29.309 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:29.350 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T20:58:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-06T20:58:29.564 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T20:58:29.571 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 26/134 2023-10-06T20:58:29.718 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T20:58:29.721 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 27/134 2023-10-06T20:58:30.604 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T20:58:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 28/134 2023-10-06T20:58:30.954 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-06T20:58:31.065 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:31.199 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-06T20:58:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-06T20:58:31.385 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-06T20:58:31.460 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-06T20:58:31.506 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-06T20:58:31.579 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T20:58:31.706 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:31.750 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-06T20:58:31.786 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T20:58:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:31.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T20:58:32.146 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T20:58:32.316 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-06T20:58:32.477 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-06T20:58:32.653 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-06T20:58:32.804 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-06T20:58:33.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-06T20:58:33.231 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-06T20:58:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:33.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-06T20:58:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-06T20:58:33.845 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-06T20:58:34.052 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-06T20:58:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T20:58:34.156 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-06T20:58:34.249 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T20:58:34.288 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-06T20:58:34.428 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T20:58:34.508 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-06T20:58:34.513 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-06T20:58:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T20:58:34.713 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-06T20:58:34.779 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-06T20:58:34.809 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-06T20:58:34.932 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-06T20:58:34.949 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-06T20:58:35.016 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-06T20:58:35.131 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 36/134 2023-10-06T20:58:35.183 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-06T20:58:35.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-06T20:58:35.326 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-06T20:58:35.345 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 37/134 2023-10-06T20:58:35.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-06T20:58:35.524 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-06T20:58:35.566 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-06T20:58:35.587 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-06T20:58:35.710 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-06T20:58:35.762 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-06T20:58:35.762 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-06T20:58:35.867 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-06T20:58:36.013 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-06T20:58:36.041 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-06T20:58:36.150 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-06T20:58:36.263 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-06T20:58:36.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-06T20:58:36.458 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-06T20:58:36.473 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-06T20:58:36.538 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-06T20:58:36.682 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-06T20:58:36.748 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-06T20:58:36.892 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-06T20:58:36.898 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-06T20:58:36.938 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-06T20:58:37.043 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-06T20:58:37.250 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:37.265 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-06T20:58:37.317 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:37.418 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-06T20:58:37.447 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-06T20:58:37.563 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-06T20:58:37.680 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-06T20:58:37.753 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-06T20:58:37.780 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-06T20:58:37.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:37.968 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-06T20:58:37.978 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:38.004 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-06T20:58:38.179 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-06T20:58:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-06T20:58:38.299 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-06T20:58:38.375 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-06T20:58:38.418 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-06T20:58:38.524 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T20:58:38.549 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-06T20:58:38.583 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-06T20:58:38.686 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T20:58:38.726 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-06T20:58:38.774 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-06T20:58:38.942 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T20:58:38.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-06T20:58:39.059 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-06T20:58:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-06T20:58:39.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T20:58:39.304 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-06T20:58:39.309 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-06T20:58:39.328 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T20:58:39.485 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T20:58:39.601 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-06T20:58:39.660 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-06T20:58:39.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T20:58:39.863 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-06T20:58:39.927 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T20:58:39.965 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-06T20:58:40.037 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-06T20:58:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T20:58:40.173 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-06T20:58:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-06T20:58:40.335 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T20:58:40.379 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:40.449 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:40.463 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T20:58:40.464 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-06T20:58:40.680 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T20:58:40.681 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-06T20:58:40.697 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-06T20:58:40.860 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-06T20:58:40.875 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-06T20:58:40.886 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T20:58:41.055 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:41.063 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T20:58:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:41.155 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-06T20:58:41.306 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T20:58:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-06T20:58:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-06T20:58:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-06T20:58:41.531 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T20:58:41.646 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-06T20:58:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T20:58:41.723 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T20:58:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T20:58:41.932 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T20:58:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-06T20:58:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T20:58:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-06T20:58:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-06T20:58:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T20:58:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-06T20:58:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T20:58:42.699 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-06T20:58:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T20:58:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T20:58:42.972 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-06T20:58:43.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T20:58:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T20:58:43.172 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-06T20:58:43.310 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-06T20:58:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T20:58:43.368 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T20:58:43.453 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-06T20:58:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T20:58:43.607 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T20:58:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T20:58:43.708 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-06T20:58:43.909 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-06T20:58:43.909 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T20:58:43.910 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T20:58:44.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-06T20:58:44.133 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T20:58:44.171 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T20:58:44.317 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-06T20:58:44.352 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T20:58:44.356 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T20:58:44.438 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-06T20:58:44.592 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T20:58:44.670 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-06T20:58:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T20:58:44.879 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-06T20:58:44.970 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T20:58:45.071 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-06T20:58:45.156 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T20:58:45.212 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T20:58:45.315 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:45.339 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-06T20:58:45.414 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:45.557 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-06T20:58:45.582 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T20:58:45.718 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-06T20:58:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-06T20:58:46.319 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T20:58:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T20:58:46.821 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T20:58:47.131 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T20:58:47.364 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-06T20:58:47.394 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T20:58:47.629 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T20:58:47.703 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-06T20:58:47.962 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-06T20:58:48.490 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-06T20:58:48.494 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T20:58:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:48.761 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-06T20:58:48.773 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:48.929 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T20:58:48.989 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-06T20:58:49.272 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:49.523 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:49.887 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-06T20:58:50.070 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:50.132 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:50.152 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 92/134 2023-10-06T20:58:50.322 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 93/134 2023-10-06T20:58:50.390 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:50.477 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:50.640 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:58:52.633 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:53.254 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:53.858 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:54.124 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:54.195 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:58:55.613 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:55.820 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:56.974 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 94/134 2023-10-06T20:58:57.306 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:57.457 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 95/134 2023-10-06T20:58:57.586 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:59:09.792 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:59:09.792 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2023-10-06T20:59:09.792 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2023-10-06T20:59:09.792 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-06T20:59:09.793 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:59:10.129 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:11.725 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:11.895 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:12.271 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:12.724 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:12.870 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:12.887 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:13.519 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:59:13.519 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2023-10-06T20:59:13.520 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2023-10-06T20:59:13.520 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2023-10-06T20:59:13.520 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /run 2023-10-06T20:59:13.520 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2023-10-06T20:59:13.520 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2023-10-06T20:59:13.521 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2023-10-06T20:59:13.521 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2023-10-06T20:59:13.521 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2023-10-06T20:59:13.521 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:59:13.884 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:14.560 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:14.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:14.926 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T20:59:15.103 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T20:59:15.269 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:15.484 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:15.489 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:15.655 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:15.794 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:15.877 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:16.051 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:16.504 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:16.709 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:16.726 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:16.758 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:16.961 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:17.052 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:17.413 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T20:59:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:17.654 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:17.852 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:17.863 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 96/134 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /run 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2023-10-06T20:59:18.726 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2023-10-06T20:59:18.727 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2023-10-06T20:59:18.727 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-10-06T20:59:18.727 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:59:18.765 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:18.881 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T20:59:19.091 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T20:59:19.150 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:19.232 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T20:59:19.258 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:19.416 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:19.449 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:19.477 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:19.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:19.625 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:19.704 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:19.794 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:20.152 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:20.336 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:20.662 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:21.030 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 97/134 2023-10-06T20:59:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T20:59:21.495 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 98/134 2023-10-06T20:59:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:21.666 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:22.226 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 99/134 2023-10-06T20:59:22.391 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 100/134 2023-10-06T20:59:23.122 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T20:59:23.352 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:23.387 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:23.541 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:23.593 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:23.783 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:24.145 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:24.324 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:24.994 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:25.296 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T20:59:25.372 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:25.443 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 101/134 2023-10-06T20:59:25.641 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:25.713 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 102/134 2023-10-06T20:59:25.906 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 103/134 2023-10-06T20:59:26.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 104/134 2023-10-06T20:59:26.515 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-06T20:59:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:26.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-06T20:59:27.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 119/134 2023-10-06T20:59:27.205 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:27.380 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T20:59:27.527 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 105/134 2023-10-06T20:59:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:28.075 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 106/134 2023-10-06T20:59:28.444 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 107/134 2023-10-06T20:59:28.654 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:28.722 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 108/134 2023-10-06T20:59:28.939 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:28.955 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:28.968 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 109/134 2023-10-06T20:59:29.252 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T20:59:29.398 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:29.597 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:29.763 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:30.034 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:30.194 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T20:59:30.483 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T20:59:30.502 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 110/134 2023-10-06T20:59:30.509 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-06T20:59:30.726 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:30.798 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 111/134 2023-10-06T20:59:30.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T20:59:30.827 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-06T20:59:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:31.004 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 112/134 2023-10-06T20:59:31.009 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 119/134 2023-10-06T20:59:31.119 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T20:59:31.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:31.251 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:31.259 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:31.377 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T20:59:31.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-06T20:59:31.524 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:31.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:31.614 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 113/134 2023-10-06T20:59:31.692 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:31.798 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:31.892 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:31.904 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:31.904 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:32.114 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:32.114 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:32.330 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:32.524 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:33.040 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:33.040 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T20:59:33.040 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T20:59:33.040 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T20:59:33.040 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T20:59:33.041 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T20:59:33.042 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T20:59:33.043 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T20:59:33.044 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T20:59:33.045 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-06T20:59:33.046 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-06T20:59:33.047 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-06T20:59:33.048 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-06T20:59:33.049 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-06T20:59:33.050 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T20:59:33.051 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:33.052 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T20:59:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:34.081 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:34.099 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:34.240 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T20:59:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T20:59:34.848 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T20:59:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T20:59:35.281 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:35.289 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:35.450 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-06T20:59:35.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:35.452 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:01:55 ago on Fri 06 Oct 2023 08:57:40 PM UTC. 2023-10-06T20:59:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:35.744 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:35.957 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:35.970 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:35.970 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:36.151 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:36.151 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:36.394 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:36.621 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:59:37.116 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T20:59:37.117 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T20:59:37.118 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T20:59:37.127 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T20:59:37.127 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T20:59:37.127 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T20:59:37.127 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T20:59:37.128 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T20:59:37.129 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T20:59:37.130 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T20:59:37.131 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-06T20:59:37.132 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-06T20:59:37.133 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-06T20:59:37.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-06T20:59:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T20:59:37.137 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:37.138 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T20:59:37.263 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 114/134 2023-10-06T20:59:37.711 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 115/134 2023-10-06T20:59:37.854 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:37.987 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2023-10-06T20:59:38.123 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-06T20:59:38.200 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.201 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.202 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:38.203 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-3.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.204 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T20:59:38.205 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T20:59:38.206 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T20:59:38.207 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T20:59:38.208 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:38.209 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.210 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T20:59:38.211 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T20:59:38.212 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T20:59:38.213 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T20:59:38.213 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T20:59:38.213 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T20:59:38.213 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-06T20:59:38.393 DEBUG:teuthology.parallel:result is None 2023-10-06T20:59:39.427 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-06T20:59:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-06T20:59:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:39.769 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:57 ago on Fri 06 Oct 2023 08:57:42 PM UTC. 2023-10-06T20:59:39.978 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 119/134 2023-10-06T20:59:40.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:40.360 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 121/134 2023-10-06T20:59:40.547 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:42.470 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2023-10-06T20:59:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T20:59:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: jq-1.6-3.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T20:59:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T20:59:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T20:59:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T20:59:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T20:59:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T20:59:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-06T20:59:42.696 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2023-10-06T20:59:42.868 DEBUG:teuthology.parallel:result is None 2023-10-06T20:59:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:44.354 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:44.391 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-06T20:59:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 124/134 2023-10-06T20:59:44.931 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 125/134 2023-10-06T20:59:45.264 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 126/134 2023-10-06T20:59:45.572 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-06T20:59:45.778 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:45.802 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:46.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-06T20:59:46.236 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:46.354 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-06T20:59:46.553 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:46.582 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-06T20:59:46.582 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:46.760 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:46.760 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:47.036 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-06T20:59:47.220 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:47.484 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:47.485 INFO:teuthology.orchestra.run.smithi026.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T20:59:47.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:59:48.117 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 1/134 2023-10-06T20:59:48.117 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 3/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 4/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6616.g14d8d 5/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 6/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 7/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 8/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 9/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 10/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 11/134 2023-10-06T20:59:48.118 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 12/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 13/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_ 14/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 15/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 16/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_6 17/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86 18/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 19/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el 20/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.0.0-6616.g14d8d032.el8. 21/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_6 22/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 23/134 2023-10-06T20:59:48.119 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 24/134 2023-10-06T20:59:48.120 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 25/134 2023-10-06T20:59:48.120 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 26/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 27/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 28/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032. 29/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noa 30/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.n 31/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d 32/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el 33/134 2023-10-06T20:59:48.129 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 34/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.e 35/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 36/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 37/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 38/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-06T20:59:48.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-06T20:59:48.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-06T20:59:48.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-06T20:59:48.133 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-06T20:59:48.134 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-06T20:59:48.135 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-06T20:59:48.136 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-06T20:59:48.137 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 129/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 131/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-06T20:59:48.138 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-06T20:59:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-06T20:59:50.957 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:07 ago on Fri 06 Oct 2023 08:57:43 PM UTC. 2023-10-06T20:59:53.774 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-10-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2023-10-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.019 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.020 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.021 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:18.0.0-6616.g14d8d032.el8.noarch 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-3.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-06T20:59:54.022 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libpq-13.5-1.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.023 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-06T20:59:54.024 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T20:59:54.025 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T20:59:54.026 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T20:59:54.027 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T20:59:54.028 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T20:59:54.029 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T20:59:54.030 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.0.0-6616.g14d8d032.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-06T20:59:54.031 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T20:59:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-06T20:59:54.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T20:59:54.032 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-10-06T20:59:54.357 DEBUG:teuthology.parallel:result is None 2023-10-06T20:59:54.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:54.505 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T20:59:54.561 INFO:teuthology.orchestra.run.smithi026.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.561 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.561 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T20:59:54.562 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:54.706 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T20:59:54.735 INFO:teuthology.orchestra.run.smithi078.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.735 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.735 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T20:59:54.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:54.882 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T20:59:54.913 INFO:teuthology.orchestra.run.smithi167.stdout:18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.914 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6616.g14d8d032.el8 2023-10-06T20:59:54.914 INFO:teuthology.task.install:The correct ceph version 18.0.0-6616.g14d8d032 is installed. 2023-10-06T20:59:54.914 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T20:59:54.915 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:59:54.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T20:59:55.008 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:59:55.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T20:59:55.057 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:59:55.057 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T20:59:55.105 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T20:59:55.106 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:59:55.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T20:59:55.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T20:59:55.291 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:59:55.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T20:59:55.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T20:59:55.392 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:59:55.393 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T20:59:55.425 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T20:59:55.495 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T20:59:55.495 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:59:55.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T20:59:55.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T20:59:55.681 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:59:55.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T20:59:55.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T20:59:55.782 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:59:55.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T20:59:55.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T20:59:55.884 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T20:59:55.884 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T20:59:55.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T20:59:55.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T20:59:56.065 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T20:59:56.066 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T20:59:56.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T20:59:56.170 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T20:59:56.170 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T20:59:56.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T20:59:56.272 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-06T20:59:56.352 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_client_delegate_inos_pct': 50, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T20:59:56.352 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:56.353 INFO:tasks.cephadm:Cluster fsid is 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T20:59:56.353 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-06T20:59:56.353 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.78', 'mon.c': '172.21.15.167'} 2023-10-06T20:59:56.353 INFO:tasks.cephadm:First mon is mon.a on smithi026 2023-10-06T20:59:56.354 INFO:tasks.cephadm:First mgr is x 2023-10-06T20:59:56.354 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-06T20:59:56.354 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2023-10-06T20:59:56.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2023-10-06T20:59:56.462 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2023-10-06T20:59:56.494 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-06T20:59:56.494 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:56.635 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '14d8d032507701c76ca224e769ee65a96b660a8c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73047/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6616-g14d8d032', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6616.g14d8d032'}, 'url': 'https://1.chacra.ceph.com/r/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-06 10:46:00.441906', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-cephfs-mon', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-06T20:59:56.775 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref rishabh-cephfs-mon, sha1 14d8d032507701c76ca224e769ee65a96b660a8c from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=14d8d032507701c76ca224e769ee65a96b660a8c 2023-10-06T20:59:56.775 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm 2023-10-06T20:59:56.776 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 20:59 /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:57.199 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:57.637 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 20:59 /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:57.637 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-cephfs-mon/14d8d032507701c76ca224e769ee65a96b660a8c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:58.192 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 6 20:59 /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:58.192 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:58.244 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:58.266 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-06T20:59:58.304 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c on all hosts... 2023-10-06T20:59:58.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T20:59:58.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T20:59:58.310 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c pull 2023-10-06T21:00:00.754 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:00:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:00:03.068 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:00:29.432 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2023-10-06T21:00:29.433 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-06T21:00:30.838 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2023-10-06T21:00:30.839 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev)", 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14", 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:80f40c42893bd196d52fcd70174d853855d3acb22cdc6bd7eee5f6d87be734f7" 2023-10-06T21:00:35.550 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:00:35.551 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:00:35.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2023-10-06T21:00:35.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2023-10-06T21:00:35.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2023-10-06T21:00:35.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2023-10-06T21:00:35.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2023-10-06T21:00:35.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2023-10-06T21:00:35.859 INFO:tasks.cephadm:Writing seed config... 2023-10-06T21:00:35.860 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-06T21:00:35.861 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] mds_client_delegate_inos_pct = 50 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-06T21:00:35.862 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-10-06T21:00:35.863 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-06T21:00:35.864 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-06T21:00:35.865 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-06T21:00:35.865 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-06T21:00:35.865 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:00:35.866 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-06T21:00:35.909 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4dcc50a6-648b-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_client_delegate_inos_pct = 50 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-06T21:00:35.909 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-06T21:00:35.954 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-06T21:00:35.999 INFO:tasks.cephadm:Bootstrapping... 2023-10-06T21:00:35.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c -v bootstrap --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:00:36.023 INFO:journalctl@ceph.mon.a.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:00:36.068 INFO:journalctl@ceph.mgr.x.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:00:36.236 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-10-06T21:00:36.236 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c', '-v', 'bootstrap', '--fsid', '4dcc50a6-648b-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2023-10-06T21:00:36.408 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:00:36.409 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-06T21:00:36.409 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2023-10-06T21:00:36.581 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:00:36.581 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2023-10-06T21:00:36.581 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2023-10-06T21:00:36.596 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T21:00:36.596 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T21:00:36.609 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T21:00:36.609 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:00:36.623 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2023-10-06T21:00:36.636 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2023-10-06T21:00:36.636 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2023-10-06T21:00:36.636 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2023-10-06T21:00:36.807 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-10-06T21:00:36.807 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-06T21:00:36.807 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2023-10-06T21:00:36.807 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2023-10-06T21:00:36.821 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-06T21:00:36.821 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-06T21:00:36.834 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-06T21:00:36.834 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:00:36.847 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2023-10-06T21:00:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2023-10-06T21:00:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2023-10-06T21:00:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2023-10-06T21:00:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:36.861 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 140143256545768 on /run/cephadm/4dcc50a6-648b-11ee-8db5-212e2dc638e7.lock 2023-10-06T21:00:36.861 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 140143256545768 acquired on /run/cephadm/4dcc50a6-648b-11ee-8db5-212e2dc638e7.lock 2023-10-06T21:00:36.861 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2023-10-06T21:00:36.862 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2023-10-06T21:00:36.862 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2023-10-06T21:00:36.869 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-06T21:00:36.870 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2023-10-06T21:00:36.877 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-06T21:00:36.877 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-06T21:00:36.877 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2023-10-06T21:00:36.883 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-06T21:00:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-06T21:00:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T21:00:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-06T21:00:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-10-06T21:00:36.884 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-06T21:00:36.885 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2023-10-06T21:00:36.885 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2023-10-06T21:00:36.885 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-06T21:00:36.886 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-06T21:00:36.887 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:00:37.359 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c... 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:a951a90ba2bd822dba777faf493a1625bb5a2f5c5042f0fdc315d633d2c600c8 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:c754feb42fd5218bf53a2fe4f07ffd844c4388a7db5210910b097caa02dbfd14 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-06T21:00:37.360 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Storing signatures 2023-10-06T21:00:38.334 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:00:38.334 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:00:38.335 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2023-10-06T21:00:39.106 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2023-10-06T21:00:39.106 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2023-10-06T21:00:39.882 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQB3dSBlM+TtHhAAlAeMEWowZDoUwHZcNx+Wcg== 2023-10-06T21:00:41.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQB4dSBlp4H+DhAAras9v+jQ1HDlhCEtL1tH9w== 2023-10-06T21:00:41.837 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQB5dSBlnmbkGhAA3lEGZVFNsmLNIzvgyi+43Q== 2023-10-06T21:00:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = pacific 2023-10-06T21:00:42.714 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:42.715 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-06T21:00:42.715 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:00:42.715 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2023-10-06T21:00:43.489 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2023-10-06T21:00:43.737 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-06T21:00:43.945 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-06T21:00:44.196 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target. 2023-10-06T21:00:44.196 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target. 2023-10-06T21:00:44.570 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T21:00:44.570 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-06T21:00:44.771 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target.wants/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@.service. 2023-10-06T21:00:45.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:44 smithi026 systemd[1]: Starting Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:00:45.555 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:45 smithi026 podman[97944]: 2023-10-06T21:00:45.555 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:45 smithi026 bash[97944]: 5a0bf2da0762db7faace932011e89b73c37239c1b9057711b09832a02104d4d4 2023-10-06T21:00:45.587 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:00:45.587 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-10-06T21:00:45.604 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:00:45.604 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:00:45.604 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-10-06T21:00:45.604 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T21:00:45.605 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2023-10-06T21:00:45.605 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2023-10-06T21:00:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:45 smithi026 ceph-mon[97984]: mkfs 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:45 smithi026 ceph-mon[97984]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:00:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:45 smithi026 systemd[1]: Started Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:00:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:46 smithi026 ceph-mon[97984]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:00:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:46 smithi026 ceph-mon[97984]: monmap e1: 1 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0]} removed_ranks: {} 2023-10-06T21:00:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:46 smithi026 ceph-mon[97984]: fsmap 2023-10-06T21:00:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:46 smithi026 ceph-mon[97984]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T21:00:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:46 smithi026 ceph-mon[97984]: mgrmap e1: no daemons active 2023-10-06T21:00:48.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:47 smithi026 ceph-mon[97984]: from='client.? 172.21.15.26:0/2257981893' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 2s) 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-06T21:00:48.097 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.503+0000 7fd170ac7700 1 Processor -- start 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.503+0000 7fd170ac7700 1 -- start start 2023-10-06T21:00:48.098 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.503+0000 7fd170ac7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.503+0000 7fd170ac7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c07a720 con 0x7fd16c07bcf0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.504+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.504+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38260/0 (socket says 172.21.15.26:38260) 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.504+0000 7fd16a59c700 1 -- 172.21.15.26:0/1766131498 learned_addr learned my addr 172.21.15.26:0/1766131498 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.505+0000 7fd16a59c700 1 -- 172.21.15.26:0/1766131498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c07a860 con 0x7fd16c07bcf0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.505+0000 7fd16a59c700 1 --2- 172.21.15.26:0/1766131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0098a0 tx=0x7fd15c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5babd14c6bc60a3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.505+0000 7fd16959a700 1 -- 172.21.15.26:0/1766131498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd15c004160 con 0x7fd16c07bcf0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.505+0000 7fd16959a700 1 -- 172.21.15.26:0/1766131498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c0042c0 con 0x7fd16c07bcf0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.505+0000 7fd16959a700 1 -- 172.21.15.26:0/1766131498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd15c0045a0 con 0x7fd16c07bcf0 2023-10-06T21:00:48.099 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 -- 172.21.15.26:0/1766131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 msgr2=0x7fd16c07a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 --2- 172.21.15.26:0/1766131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0098a0 tx=0x7fd15c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 -- 172.21.15.26:0/1766131498 shutdown_connections 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 --2- 172.21.15.26:0/1766131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c07a150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 -- 172.21.15.26:0/1766131498 >> 172.21.15.26:0/1766131498 conn(0x7fd16c101ee0 msgr2=0x7fd16c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 -- 172.21.15.26:0/1766131498 shutdown_connections 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.506+0000 7fd170ac7700 1 -- 172.21.15.26:0/1766131498 wait complete. 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.507+0000 7fd170ac7700 1 Processor -- start 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.507+0000 7fd170ac7700 1 -- start start 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd170ac7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:48.100 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd170ac7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c19c470 con 0x7fd16c07bcf0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd16a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38266/0 (socket says 172.21.15.26:38266) 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd16a59c700 1 -- 172.21.15.26:0/2257981893 learned_addr learned my addr 172.21.15.26:0/2257981893 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd16a59c700 1 -- 172.21.15.26:0/2257981893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c009550 con 0x7fd16c07bcf0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.508+0000 7fd16a59c700 1 --2- 172.21.15.26:0/2257981893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd16c07b1c0 tx=0x7fd15c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd15c011630 con 0x7fd16c07bcf0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c011790 con 0x7fd16c07bcf0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd15c011a30 con 0x7fd16c07bcf0 2023-10-06T21:00:48.101 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c19c670 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c19cb10 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd15c022070 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.509+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd15c01c450 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.510+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c195eb0 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.513+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd15c01b450 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.589+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd16c07abd0 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.589+0000 7fd15b7fe700 1 -- 172.21.15.26:0/2257981893 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+310 (secure 0 0 0) 0x7fd15c01b5f0 con 0x7fd16c07bcf0 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 msgr2=0x7fd16c19bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 --2- 172.21.15.26:0/2257981893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd16c07b1c0 tx=0x7fd15c004750 comp rx=0 tx=0).stop 2023-10-06T21:00:48.102 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 shutdown_connections 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 --2- 172.21.15.26:0/2257981893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd16c07bcf0 0x7fd16c19bf30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 >> 172.21.15.26:0/2257981893 conn(0x7fd16c101ee0 msgr2=0x7fd16c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 shutdown_connections 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:47.590+0000 7fd170ac7700 1 -- 172.21.15.26:0/2257981893 wait complete. 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2023-10-06T21:00:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2023-10-06T21:00:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:48 smithi026 ceph-mon[97984]: from='client.? 172.21.15.26:0/573061239' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T21:00:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:48 smithi026 ceph-mon[97984]: from='client.? 172.21.15.26:0/573061239' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-06T21:00:49.175 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:49.182 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.625+0000 7f4629321700 1 Processor -- start 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.625+0000 7f4629321700 1 -- start start 2023-10-06T21:00:49.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.626+0000 7f4629321700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.626+0000 7f4629321700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462407a720 con 0x7f462407bcf0 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.626+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.626+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.626+0000 7f4622ffd700 1 -- 172.21.15.26:0/3254550576 learned_addr learned my addr 172.21.15.26:0/3254550576 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.627+0000 7f4622ffd700 1 -- 172.21.15.26:0/3254550576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462407a860 con 0x7f462407bcf0 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.627+0000 7f4622ffd700 1 --2- 172.21.15.26:0/3254550576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcd78034dceea18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.627+0000 7f4621ffb700 1 -- 172.21.15.26:0/3254550576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614004160 con 0x7f462407bcf0 2023-10-06T21:00:49.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.627+0000 7f4621ffb700 1 -- 172.21.15.26:0/3254550576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46140042c0 con 0x7f462407bcf0 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.627+0000 7f4621ffb700 1 -- 172.21.15.26:0/3254550576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46140045a0 con 0x7f462407bcf0 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 -- 172.21.15.26:0/3254550576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 msgr2=0x7f462407a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 --2- 172.21.15.26:0/3254550576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).stop 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 -- 172.21.15.26:0/3254550576 shutdown_connections 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 --2- 172.21.15.26:0/3254550576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462407a150 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 -- 172.21.15.26:0/3254550576 >> 172.21.15.26:0/3254550576 conn(0x7f4624101ee0 msgr2=0x7f4624104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 -- 172.21.15.26:0/3254550576 shutdown_connections 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.628+0000 7f4629321700 1 -- 172.21.15.26:0/3254550576 wait complete. 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.629+0000 7f4629321700 1 Processor -- start 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.629+0000 7f4629321700 1 -- start start 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.629+0000 7f4629321700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:49.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.629+0000 7f4629321700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462419c470 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38284/0 (socket says 172.21.15.26:38284) 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4622ffd700 1 -- 172.21.15.26:0/573061239 learned_addr learned my addr 172.21.15.26:0/573061239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4622ffd700 1 -- 172.21.15.26:0/573061239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009550 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4622ffd700 1 --2- 172.21.15.26:0/573061239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f462407b1c0 tx=0x7f4614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614011630 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614011790 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614011a30 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.630+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462419c670 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.631+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462419cb10 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.631+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4614022070 con 0x7f462407bcf0 2023-10-06T21:00:49.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.631+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f461401c450 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.632+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624195eb0 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.634+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f461401b450 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.707+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f462407ae80 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.711+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f461401b5f0 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.712+0000 7f460bfff700 1 -- 172.21.15.26:0/573061239 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f461401b790 con 0x7f462407bcf0 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.712+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 msgr2=0x7f462419bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.712+0000 7f4629321700 1 --2- 172.21.15.26:0/573061239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f462407b1c0 tx=0x7f4614004750 comp rx=0 tx=0).stop 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.712+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 shutdown_connections 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.712+0000 7f4629321700 1 --2- 172.21.15.26:0/573061239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462407bcf0 0x7f462419bf30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.713+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 >> 172.21.15.26:0/573061239 conn(0x7f4624101ee0 msgr2=0x7f46241037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.713+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 shutdown_connections 2023-10-06T21:00:49.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:48.713+0000 7f4629321700 1 -- 172.21.15.26:0/573061239 wait complete. 2023-10-06T21:00:49.188 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2023-10-06T21:00:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:49 smithi026 ceph-mon[97984]: from='client.? 172.21.15.26:0/2680932896' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:00:50.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.705+0000 7fc4569d9700 1 Processor -- start 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc4569d9700 1 -- start start 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc4569d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc4569d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45007a740 con 0x7fc45007bd10 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38300/0 (socket says 172.21.15.26:38300) 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.706+0000 7fc44ffff700 1 -- 172.21.15.26:0/343074623 learned_addr learned my addr 172.21.15.26:0/343074623 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.707+0000 7fc44ffff700 1 -- 172.21.15.26:0/343074623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45007a880 con 0x7fc45007bd10 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.707+0000 7fc44ffff700 1 --2- 172.21.15.26:0/343074623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4400098a0 tx=0x7fc440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc3e134c2de9d0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.707+0000 7fc44effd700 1 -- 172.21.15.26:0/343074623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc440004160 con 0x7fc45007bd10 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.707+0000 7fc44effd700 1 -- 172.21.15.26:0/343074623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4400042c0 con 0x7fc45007bd10 2023-10-06T21:00:50.161 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.707+0000 7fc44effd700 1 -- 172.21.15.26:0/343074623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc440011a00 con 0x7fc45007bd10 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 -- 172.21.15.26:0/343074623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 msgr2=0x7fc45007a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 --2- 172.21.15.26:0/343074623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc4400098a0 tx=0x7fc440009bb0 comp rx=0 tx=0).stop 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 -- 172.21.15.26:0/343074623 shutdown_connections 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 --2- 172.21.15.26:0/343074623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45007a170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 -- 172.21.15.26:0/343074623 >> 172.21.15.26:0/343074623 conn(0x7fc450101ee0 msgr2=0x7fc450104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 -- 172.21.15.26:0/343074623 shutdown_connections 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.708+0000 7fc4569d9700 1 -- 172.21.15.26:0/343074623 wait complete. 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.709+0000 7fc4569d9700 1 Processor -- start 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.709+0000 7fc4569d9700 1 -- start start 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.709+0000 7fc4569d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:50.162 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.709+0000 7fc4569d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45019cca0 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.710+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.710+0000 7fc44ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38302/0 (socket says 172.21.15.26:38302) 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.710+0000 7fc44ffff700 1 -- 172.21.15.26:0/2680932896 learned_addr learned my addr 172.21.15.26:0/2680932896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.710+0000 7fc44ffff700 1 -- 172.21.15.26:0/2680932896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440009550 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44ffff700 1 --2- 172.21.15.26:0/2680932896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc440009130 tx=0x7fc440011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc440003b70 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc440003cd0 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4400197d0 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45019cea0 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45019d340 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc4400199c0 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.711+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc44000b720 con 0x7fc45007bd10 2023-10-06T21:00:50.163 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.712+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4500624e0 con 0x7fc45007bd10 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.715+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc44002a060 con 0x7fc45007bd10 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.789+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc45010cf50 con 0x7fc45007bd10 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.789+0000 7fc44d7fa700 1 -- 172.21.15.26:0/2680932896 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fc440019d10 con 0x7fc45007bd10 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 msgr2=0x7fc45019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 --2- 172.21.15.26:0/2680932896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc440009130 tx=0x7fc440011ad0 comp rx=0 tx=0).stop 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 shutdown_connections 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 --2- 172.21.15.26:0/2680932896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45007bd10 0x7fc45019c760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 >> 172.21.15.26:0/2680932896 conn(0x7fc450101ee0 msgr2=0x7fc450102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 shutdown_connections 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:49.790+0000 7fc4569d9700 1 -- 172.21.15.26:0/2680932896 wait complete. 2023-10-06T21:00:50.164 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2023-10-06T21:00:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:50 smithi026 systemd[1]: Stopping Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:00:50.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:50 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a[97960]: 2023-10-06T21:00:50.624+0000 7f71c5376700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:00:50.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:50 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a[97960]: 2023-10-06T21:00:50.624+0000 7f71c5376700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-06T21:00:51.242 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:50 smithi026 bash[98323]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T21:00:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:51 smithi026 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-06T21:00:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:51 smithi026 systemd[1]: Stopped Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:00:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:51 smithi026 systemd[1]: Starting Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:00:52.191 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:51 smithi026 podman[98450]: 2023-10-06T21:00:52.191 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 bash[98450]: 3b01bf7adf1bf069c6d83158537c9a1d8d011736ac95f5008b4a0e224813868b 2023-10-06T21:00:52.191 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T21:00:52.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T21:00:52.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: pidfile_write: ignore empty --pid-file 2023-10-06T21:00:52.216 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: load: jerasure load: lrc 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: RocksDB version: 7.9.2 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Git sha 0 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: DB SUMMARY 2023-10-06T21:00:52.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: DB Session ID: TDPBTKYPOC1Q1LCYJJLM 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: CURRENT file: CURRENT 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101473 ; 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.error_if_exists: 0 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.create_if_missing: 0 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.paranoid_checks: 1 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T21:00:52.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.env: 0x55e364fa1da0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.info_log: 0x55e3661f95c0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.statistics: (nil) 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.use_fsync: 0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_log_file_size: 0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_fallocate: 1 2023-10-06T21:00:52.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.use_direct_reads: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.db_log_dir: 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.wal_dir: 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T21:00:52.525 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.write_buffer_manager: 0x55e3661b1360 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T21:00:52.526 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.unordered_write: 0 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.row_cache: None 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.wal_filter: None 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.two_write_queues: 0 2023-10-06T21:00:52.527 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.wal_compression: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.atomic_flush: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.log_readahead_size: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T21:00:52.528 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_background_jobs: 2 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_background_compactions: -1 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_subcompactions: 1 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T21:00:52.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_open_files: -1 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_background_flushes: -1 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Compression algorithms supported: 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kZSTD supported: 0 2023-10-06T21:00:52.530 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kXpressCompression supported: 0 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kZlibCompression supported: 1 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kSnappyCompression supported: 1 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kLZ4Compression supported: 1 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: kBZip2Compression supported: 0 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.merge_operator: 2023-10-06T21:00:52.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_filter: None 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e3661f96c0) 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2023-10-06T21:00:52.532 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x55e36624f1f0 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2023-10-06T21:00:52.533 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2023-10-06T21:00:52.534 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression: NoCompression 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T21:00:52.535 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.num_levels: 7 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T21:00:52.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:00:52.537 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:00:52.537 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T21:00:52.537 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T21:00:52.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T21:00:52.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T21:00:52.540 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.table_properties_collectors: 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.inplace_update_support: 0 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.bloom_locality: 0 2023-10-06T21:00:52.541 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.max_successive_merges: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.ttl: 2592000 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enable_blob_files: false 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.min_blob_size: 0 2023-10-06T21:00:52.542 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0b09f212-9f55-4b82-9884-6eaab9b13b8d 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626052199803, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-06T21:00:52.543 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626052201756, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 337, "table_properties": {"data_size": 94110, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14172, "raw_average_key_size": 49, "raw_value_size": 86407, "raw_average_value_size": 304, "num_data_blocks": 11, "num_entries": 284, "num_filter_entries": 284, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696626052, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0b09f212-9f55-4b82-9884-6eaab9b13b8d", "db_session_id": "TDPBTKYPOC1Q1LCYJJLM", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626052201908, "job": 1, "event": "recovery_finished"} 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e3662ec000 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: DB pointer 0x55e3662d8000 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T21:00:52.544 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 95.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 95.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.545 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T21:00:52.546 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 11.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 11.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x55e36624f1f0#7 capacity: 512.00 MB usage: 1.48 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???) e1 preinit fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).mds e1 new map 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).mds e1 print_map 2023-10-06T21:00:52.547 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expand map: {default=false} 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta from 'false' to 'false' 2023-10-06T21:00:52.548 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expanded map: {default=false} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expand map: {default=info} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta from 'info' to 'info' 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expanded map: {default=info} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expand map: {default=daemon} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expand map: {default=debug} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: expand_channel_meta expanded map: {default=debug} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 systemd[1]: Started Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: monmap e1: 1 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0]} removed_ranks: {} 2023-10-06T21:00:52.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: fsmap 2023-10-06T21:00:52.550 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: osdmap e1: 0 total, 0 up, 0 in 2023-10-06T21:00:52.550 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:52 smithi026 ceph-mon[98490]: mgrmap e1: no daemons active 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.787+0000 7f11dc8db700 1 Processor -- start 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.788+0000 7f11dc8db700 1 -- start start 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.788+0000 7f11dc8db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.788+0000 7f11dc8db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d807a740 con 0x7f11d807bd10 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.789+0000 7f11d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:53.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.789+0000 7f11d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50114/0 (socket says 172.21.15.26:50114) 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.789+0000 7f11d659c700 1 -- 172.21.15.26:0/4139745500 learned_addr learned my addr 172.21.15.26:0/4139745500 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.789+0000 7f11d659c700 1 -- 172.21.15.26:0/4139745500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d807a880 con 0x7f11d807bd10 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.789+0000 7f11d659c700 1 --2- 172.21.15.26:0/4139745500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11c80098a0 tx=0x7f11c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5206736503234c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11d559a700 1 -- 172.21.15.26:0/4139745500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c8004160 con 0x7f11d807bd10 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11d559a700 1 -- 172.21.15.26:0/4139745500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11c80042c0 con 0x7f11d807bd10 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11d559a700 1 -- 172.21.15.26:0/4139745500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c8011a00 con 0x7f11d807bd10 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 -- 172.21.15.26:0/4139745500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 msgr2=0x7f11d807a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 --2- 172.21.15.26:0/4139745500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11c80098a0 tx=0x7f11c8009bb0 comp rx=0 tx=0).stop 2023-10-06T21:00:53.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 -- 172.21.15.26:0/4139745500 shutdown_connections 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 --2- 172.21.15.26:0/4139745500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d807a170 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 -- 172.21.15.26:0/4139745500 >> 172.21.15.26:0/4139745500 conn(0x7f11d8101ec0 msgr2=0x7f11d81042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 -- 172.21.15.26:0/4139745500 shutdown_connections 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.790+0000 7f11dc8db700 1 -- 172.21.15.26:0/4139745500 wait complete. 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.791+0000 7f11dc8db700 1 Processor -- start 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11dc8db700 1 -- start start 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11dc8db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11dc8db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d819cce0 con 0x7f11d807bd10 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50126/0 (socket says 172.21.15.26:50126) 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.792+0000 7f11d659c700 1 -- 172.21.15.26:0/2145033571 learned_addr learned my addr 172.21.15.26:0/2145033571 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11d659c700 1 -- 172.21.15.26:0/2145033571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8009550 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11d659c700 1 --2- 172.21.15.26:0/2145033571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009fb0 tx=0x7f11c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c8003a40 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11c8003ba0 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d819cee0 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11c801a750 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.793+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d819d380 con 0x7f11d807bd10 2023-10-06T21:00:53.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.794+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f11c801ac80 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.794+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f11c80193c0 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.794+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d80624e0 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.797+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f11c802de20 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.870+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f11d810ccf0 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.872+0000 7f11c77fe700 1 -- 172.21.15.26:0/2145033571 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f11c8024b60 con 0x7f11d807bd10 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 msgr2=0x7f11d819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 --2- 172.21.15.26:0/2145033571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009fb0 tx=0x7f11c8011ad0 comp rx=0 tx=0).stop 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 shutdown_connections 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 --2- 172.21.15.26:0/2145033571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11d807bd10 0x7f11d819c7a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:53.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 >> 172.21.15.26:0/2145033571 conn(0x7f11d8101ec0 msgr2=0x7f11d8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:53.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 shutdown_connections 2023-10-06T21:00:53.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:52.873+0000 7f11dc8db700 1 -- 172.21.15.26:0/2145033571 wait complete. 2023-10-06T21:00:53.206 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-06T21:00:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:00:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2023-10-06T21:00:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-06T21:00:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-06T21:00:53.591 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T21:00:53.591 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-06T21:00:53.806 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7.target.wants/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@.service. 2023-10-06T21:00:53.874 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:53 smithi026 systemd[1]: Starting Ceph mgr.x for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:00:54.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:53 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2145033571' entity='client.admin' 2023-10-06T21:00:54.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 podman[98718]: 2023-10-06T21:00:54.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 bash[98718]: 57773bc11cd692b2b64f66f18f0b17e07c3cd37b67dfd97ef01d6617df1fd510 2023-10-06T21:00:54.616 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:00:54.617 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-10-06T21:00:54.634 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:00:54.634 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:00:54.634 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-10-06T21:00:54.634 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2023-10-06T21:00:54.652 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:00:54.652 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2023-10-06T21:00:54.669 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2023-10-06T21:00:54.943 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 systemd[1]: Started Ceph mgr.x for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:00:54.944 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:54.707+0000 7fab1115f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:00:54.944 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:54.821+0000 7fab1115f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:00:55.321 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:54 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:54.940+0000 7fab1115f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:00:55.321 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:55.076+0000 7fab1115f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:00:55.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:00:55 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1609388730' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T21:00:55.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:55.318+0000 7fab1115f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:00:55.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T21:00:55.863 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2023-10-06T21:00:55.864 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:00:55.864 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T21:00:55.864 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T21:00:55.864 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:00:55.864 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T21:00:55.865 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.866 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:00:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T21:00:45.550532+0000", 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.269+0000 7f7ec37d2700 1 Processor -- start 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.269+0000 7f7ec37d2700 1 -- start start 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.270+0000 7f7ec37d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:55.868 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.270+0000 7f7ec37d2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc107310 con 0x7f7ebc106920 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.270+0000 7f7ec156e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7ec156e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50156/0 (socket says 172.21.15.26:50156) 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7ec156e700 1 -- 172.21.15.26:0/3887711318 learned_addr learned my addr 172.21.15.26:0/3887711318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7ec156e700 1 -- 172.21.15.26:0/3887711318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc107b20 con 0x7f7ebc106920 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7ec156e700 1 --2- 172.21.15.26:0/3887711318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80098a0 tx=0x7f7eb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c06a5b4a0f6031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7eb3fff700 1 -- 172.21.15.26:0/3887711318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb8004160 con 0x7f7ebc106920 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7eb3fff700 1 -- 172.21.15.26:0/3887711318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7eb80042c0 con 0x7f7ebc106920 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.271+0000 7f7eb3fff700 1 -- 172.21.15.26:0/3887711318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb8011a00 con 0x7f7ebc106920 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 -- 172.21.15.26:0/3887711318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 msgr2=0x7f7ebc106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 --2- 172.21.15.26:0/3887711318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80098a0 tx=0x7f7eb8009bb0 comp rx=0 tx=0).stop 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 -- 172.21.15.26:0/3887711318 shutdown_connections 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 --2- 172.21.15.26:0/3887711318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc106d40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:55.869 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 -- 172.21.15.26:0/3887711318 >> 172.21.15.26:0/3887711318 conn(0x7f7ebc101ee0 msgr2=0x7f7ebc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 -- 172.21.15.26:0/3887711318 shutdown_connections 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.272+0000 7f7ec37d2700 1 -- 172.21.15.26:0/3887711318 wait complete. 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.273+0000 7f7ec37d2700 1 Processor -- start 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec37d2700 1 -- start start 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec37d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec37d2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc19cca0 con 0x7f7ebc106920 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec156e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec156e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50168/0 (socket says 172.21.15.26:50168) 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.274+0000 7f7ec156e700 1 -- 172.21.15.26:0/1609388730 learned_addr learned my addr 172.21.15.26:0/1609388730 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7ec156e700 1 -- 172.21.15.26:0/1609388730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8009550 con 0x7f7ebc106920 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7ec156e700 1 --2- 172.21.15.26:0/1609388730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009130 tx=0x7f7eb8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:00:55.870 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb8003ab0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7eb8003c10 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7eb80247d0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc19cea0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.275+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc19d340 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.276+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7eb8024d00 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.276+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7eb80193c0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.276+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc0624e0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.279+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f7eb801abb0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.357+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7ebc10adf0 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.358+0000 7f7eb27fc700 1 -- 172.21.15.26:0/1609388730 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f7eb8005a20 con 0x7f7ebc106920 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 msgr2=0x7f7ebc19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:00:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 --2- 172.21.15.26:0/1609388730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009130 tx=0x7f7eb8011ad0 comp rx=0 tx=0).stop 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 shutdown_connections 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 --2- 172.21.15.26:0/1609388730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ebc106920 0x7f7ebc19c760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 >> 172.21.15.26:0/1609388730 conn(0x7f7ebc101ee0 msgr2=0x7f7ebc102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 shutdown_connections 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:00:55.359+0000 7f7ec37d2700 1 -- 172.21.15.26:0/1609388730 wait complete. 2023-10-06T21:00:55.872 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2023-10-06T21:00:55.893 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:55.813+0000 7fab1115f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:00:56.236 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:55.987+0000 7fab1115f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:00:56.237 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:56 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:56.106+0000 7fab1115f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:00:56.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:56 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:56.234+0000 7fab1115f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:00:57.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:56 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:56.794+0000 7fab1115f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:00:58.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:57 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:57.708+0000 7fab1115f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:00:58.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:57 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:57.817+0000 7fab1115f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:00:59.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:58 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:58.817+0000 7fab1115f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:00:59.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:58 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:58.940+0000 7fab1115f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:00:59.550 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:59.171+0000 7fab1115f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:00:59.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:59.547+0000 7fab1115f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:01:00.323 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:00:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:00:59.828+0000 7fab1115f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:01:00.323 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:00.027+0000 7fab1115f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:01:00.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:00.382+0000 7fab1115f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:01:01.783 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:01.345+0000 7fab1115f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:01:01.783 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:01.524+0000 7fab1115f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:01:02.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:01.779+0000 7fab1115f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:01:02.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:01.930+0000 7fab1115f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:01:02.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:02.218+0000 7fab1115f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:01:03.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:03.335+0000 7fab1115f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:01:03.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: Activating manager daemon x 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: mgrmap e2: x(active, starting, since 0.00402396s) 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: Manager daemon x is now available 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:01:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' 2023-10-06T21:01:03.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:03 smithi026 ceph-mon[98490]: from='mgr.14100 172.21.15.26:0/611186099' entity='mgr.x' 2023-10-06T21:01:05.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:05 smithi026 ceph-mon[98490]: mgrmap e3: x(active, since 1.01047s) 2023-10-06T21:01:05.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-06T21:01:05.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-10-06T21:01:05.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:01:05.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-06T21:01:05.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-06T21:01:05.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-06T21:01:05.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:01:05.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2023-10-06T21:01:05.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2023-10-06T21:00:45.550532+0000", 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-06T21:01:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.149+0000 7f98b668e700 1 Processor -- start 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.149+0000 7f98b668e700 1 -- start start 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.149+0000 7f98b668e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.149+0000 7f98b668e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b007a780 con 0x7f98b007bd50 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.150+0000 7f98b568c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.150+0000 7f98b568c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40630/0 (socket says 172.21.15.26:40630) 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.150+0000 7f98b568c700 1 -- 172.21.15.26:0/2885842238 learned_addr learned my addr 172.21.15.26:0/2885842238 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.150+0000 7f98b568c700 1 -- 172.21.15.26:0/2885842238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b007a8c0 con 0x7f98b007bd50 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.150+0000 7f98b568c700 1 --2- 172.21.15.26:0/2885842238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098a0 tx=0x7f98ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754faa392d2461b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:05.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98a7fff700 1 -- 172.21.15.26:0/2885842238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac004160 con 0x7f98b007bd50 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98a7fff700 1 -- 172.21.15.26:0/2885842238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98ac0042c0 con 0x7f98b007bd50 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98a7fff700 1 -- 172.21.15.26:0/2885842238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac011a00 con 0x7f98b007bd50 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 -- 172.21.15.26:0/2885842238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 msgr2=0x7f98b007a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 --2- 172.21.15.26:0/2885842238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098a0 tx=0x7f98ac009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 -- 172.21.15.26:0/2885842238 shutdown_connections 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 --2- 172.21.15.26:0/2885842238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b007a1b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 -- 172.21.15.26:0/2885842238 >> 172.21.15.26:0/2885842238 conn(0x7f98b0101cf0 msgr2=0x7f98b0104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 -- 172.21.15.26:0/2885842238 shutdown_connections 2023-10-06T21:01:05.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.151+0000 7f98b668e700 1 -- 172.21.15.26:0/2885842238 wait complete. 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.152+0000 7f98b668e700 1 Processor -- start 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b668e700 1 -- start start 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b668e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b668e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019cb30 con 0x7f98b007bd50 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b568c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b568c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40644/0 (socket says 172.21.15.26:40644) 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b568c700 1 -- 172.21.15.26:0/3680670867 learned_addr learned my addr 172.21.15.26:0/3680670867 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.153+0000 7f98b568c700 1 -- 172.21.15.26:0/3680670867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac009550 con 0x7f98b007bd50 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98b568c700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009130 tx=0x7f98ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:05.978 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac003ab0 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98ac003c10 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac0247d0 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b019cd30 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.154+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b019d1d0 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.155+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 3) v1 ==== 45462+0+0 (secure 0 0 0) 0x7f98ac01abb0 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.155+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b010cae0 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.156+0000 7f98a67fc700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 0x7f989c03abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.156+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98ac04c340 con 0x7f98b007bd50 2023-10-06T21:01:05.979 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.156+0000 7f98b4e8b700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 0x7f989c03abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.157+0000 7f98b4e8b700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 0x7f989c03abb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.162+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f98ac018710 con 0x7f98b007bd50 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.347+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f98ac02d770 con 0x7f98b007bd50 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.453+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f98b010cd50 con 0x7f98b007bd50 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.454+0000 7f98a67fc700 1 -- 172.21.15.26:0/3680670867 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f98ac0188f0 con 0x7f98b007bd50 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 msgr2=0x7f989c03abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 0x7f989c03abb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 msgr2=0x7f98b019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009130 tx=0x7f98ac011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 shutdown_connections 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f989c0386f0 0x7f989c03abb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 --2- 172.21.15.26:0/3680670867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98b007bd50 0x7f98b019c5f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 >> 172.21.15.26:0/3680670867 conn(0x7f98b0101cf0 msgr2=0x7f98b0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 shutdown_connections 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:05.456+0000 7f98b668e700 1 -- 172.21.15.26:0/3680670867 wait complete. 2023-10-06T21:01:05.981 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2023-10-06T21:01:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:06 smithi026 ceph-mon[98490]: mgrmap e4: x(active, since 2s) 2023-10-06T21:01:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:06 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3680670867' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-06T21:01:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.522+0000 7f6e8aea8700 1 Processor -- start 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.522+0000 7f6e8aea8700 1 -- start start 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.522+0000 7f6e8aea8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.522+0000 7f6e8aea8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8407a720 con 0x7f6e8407bcf0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.523+0000 7f6e88c44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.523+0000 7f6e88c44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40658/0 (socket says 172.21.15.26:40658) 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.523+0000 7f6e88c44700 1 -- 172.21.15.26:0/2822633358 learned_addr learned my addr 172.21.15.26:0/2822633358 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.523+0000 7f6e88c44700 1 -- 172.21.15.26:0/2822633358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8407a860 con 0x7f6e8407bcf0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.523+0000 7f6e88c44700 1 --2- 172.21.15.26:0/2822633358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e780098a0 tx=0x7f6e78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8165027a17b742d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e837fe700 1 -- 172.21.15.26:0/2822633358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e78004160 con 0x7f6e8407bcf0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e837fe700 1 -- 172.21.15.26:0/2822633358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e780042c0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e837fe700 1 -- 172.21.15.26:0/2822633358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e78011a00 con 0x7f6e8407bcf0 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2822633358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 msgr2=0x7f6e8407a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2822633358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6e780098a0 tx=0x7f6e78009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:07.150 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2822633358 shutdown_connections 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2822633358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8407a150 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2822633358 >> 172.21.15.26:0/2822633358 conn(0x7f6e84101ee0 msgr2=0x7f6e84104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.524+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2822633358 shutdown_connections 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.525+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2822633358 wait complete. 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.525+0000 7f6e8aea8700 1 Processor -- start 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e8aea8700 1 -- start start 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e8aea8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e8aea8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8419cca0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e88c44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e88c44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40666/0 (socket says 172.21.15.26:40666) 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.526+0000 7f6e88c44700 1 -- 172.21.15.26:0/2517905286 learned_addr learned my addr 172.21.15.26:0/2517905286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e88c44700 1 -- 172.21.15.26:0/2517905286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78009550 con 0x7f6e8407bcf0 2023-10-06T21:01:07.151 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e88c44700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009130 tx=0x7f6e78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e78003b70 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e78003cd0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e780197d0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8419cea0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.527+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8419d340 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.528+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f6e7801abb0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.528+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8410ccd0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.529+0000 7f6e81ffb700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 0x7f6e7403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.529+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e7804c9c0 con 0x7f6e8407bcf0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.529+0000 7f6e83fff700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 0x7f6e7403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:07.152 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.530+0000 7f6e83fff700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 0x7f6e7403ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006fd0 tx=0x7f6e70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.534+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e7801a430 con 0x7f6e8407bcf0 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.743+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6e8410cf10 con 0x7f6e8407bcf0 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.744+0000 7f6e81ffb700 1 -- 172.21.15.26:0/2517905286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f6e78023370 con 0x7f6e8407bcf0 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 msgr2=0x7f6e7403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 0x7f6e7403ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006fd0 tx=0x7f6e70006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 msgr2=0x7f6e8419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009130 tx=0x7f6e78011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 shutdown_connections 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f6e74038820 0x7f6e7403ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 --2- 172.21.15.26:0/2517905286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8407bcf0 0x7f6e8419c760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 >> 172.21.15.26:0/2517905286 conn(0x7f6e84101ee0 msgr2=0x7f6e84102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 shutdown_connections 2023-10-06T21:01:07.153 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:06.746+0000 7f6e8aea8700 1 -- 172.21.15.26:0/2517905286 wait complete. 2023-10-06T21:01:07.154 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2023-10-06T21:01:07.431 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:07 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2517905286' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-06T21:01:08.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2435685079' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-06T21:01:08.618 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setuser ceph since I am not root 2023-10-06T21:01:08.618 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setgroup ceph since I am not root 2023-10-06T21:01:08.827 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.688+0000 7f41ff30e700 1 Processor -- start 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.688+0000 7f41ff30e700 1 -- start start 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.688+0000 7f41ff30e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.688+0000 7f41ff30e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f81072f0 con 0x7f41f8106900 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.689+0000 7f41fd0aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.689+0000 7f41fd0aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40682/0 (socket says 172.21.15.26:40682) 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.689+0000 7f41fd0aa700 1 -- 172.21.15.26:0/3655149603 learned_addr learned my addr 172.21.15.26:0/3655149603 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.689+0000 7f41fd0aa700 1 -- 172.21.15.26:0/3655149603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f8107b00 con 0x7f41f8106900 2023-10-06T21:01:08.832 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41fd0aa700 1 --2- 172.21.15.26:0/3655149603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41f40098a0 tx=0x7f41f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a38131fe6009c73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41effff700 1 -- 172.21.15.26:0/3655149603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41f4004160 con 0x7f41f8106900 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41effff700 1 -- 172.21.15.26:0/3655149603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41f40042c0 con 0x7f41f8106900 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41effff700 1 -- 172.21.15.26:0/3655149603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41f4011a00 con 0x7f41f8106900 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41ff30e700 1 -- 172.21.15.26:0/3655149603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 msgr2=0x7f41f8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41ff30e700 1 --2- 172.21.15.26:0/3655149603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41f40098a0 tx=0x7f41f4009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41ff30e700 1 -- 172.21.15.26:0/3655149603 shutdown_connections 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41ff30e700 1 --2- 172.21.15.26:0/3655149603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f8106d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.690+0000 7f41ff30e700 1 -- 172.21.15.26:0/3655149603 >> 172.21.15.26:0/3655149603 conn(0x7f41f8101ec0 msgr2=0x7f41f81042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.691+0000 7f41ff30e700 1 -- 172.21.15.26:0/3655149603 shutdown_connections 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.691+0000 7f41ff30e700 1 -- 172.21.15.26:0/3655149603 wait complete. 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41ff30e700 1 Processor -- start 2023-10-06T21:01:08.833 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41ff30e700 1 -- start start 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41ff30e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41ff30e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f819aaf0 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41fd0aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41fd0aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40692/0 (socket says 172.21.15.26:40692) 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.692+0000 7f41fd0aa700 1 -- 172.21.15.26:0/2435685079 learned_addr learned my addr 172.21.15.26:0/2435685079 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41fd0aa700 1 -- 172.21.15.26:0/2435685079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009550 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41fd0aa700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009130 tx=0x7f41f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41f4003ab0 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41f4003c10 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41f40247d0 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f819acf0 con 0x7f41f8106900 2023-10-06T21:01:08.834 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.693+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f819b190 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.695+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f80624e0 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.695+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f41f401abb0 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.695+0000 7f41ee7fc700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 0x7f41e403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.695+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f41f404c540 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.695+0000 7f41fc8a9700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 0x7f41e403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.696+0000 7f41fc8a9700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 0x7f41e403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.700+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41f4018710 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:07.969+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f41f810ad00 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.351+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f41f4018350 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.353+0000 7f41ee7fc700 1 -- 172.21.15.26:0/2435685079 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f41f402d770 con 0x7f41f8106900 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 msgr2=0x7f41e403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:08.835 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 0x7f41e403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006fd0 tx=0x7f41e8006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 msgr2=0x7f41f819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009130 tx=0x7f41f4011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 shutdown_connections 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f41e4038820 0x7f41e403ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 --2- 172.21.15.26:0/2435685079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f8106900 0x7f41f819a5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 >> 172.21.15.26:0/2435685079 conn(0x7f41f8101ec0 msgr2=0x7f41f8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 shutdown_connections 2023-10-06T21:01:08.836 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:08.355+0000 7f41ff30e700 1 -- 172.21.15.26:0/2435685079 wait complete. 2023-10-06T21:01:08.981 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:08.615+0000 7fbfe2535200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:01:08.981 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:08.723+0000 7fbfe2535200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:01:08.981 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:08.847+0000 7fbfe2535200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:01:08.981 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:08.978+0000 7fbfe2535200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:01:09.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:09.243+0000 7fbfe2535200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:01:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2435685079' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-06T21:01:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:09 smithi026 ceph-mon[98490]: mgrmap e5: x(active, since 5s) 2023-10-06T21:01:10.026 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:09.727+0000 7fbfe2535200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:01:10.026 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:09.901+0000 7fbfe2535200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:01:10.026 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:10.022+0000 7fbfe2535200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:01:10.117 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.404+0000 7f123b13a700 1 Processor -- start 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f123b13a700 1 -- start start 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f123b13a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f123b13a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123407a740 con 0x7f123407bd10 2023-10-06T21:01:10.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f1238ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f1238ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40724/0 (socket says 172.21.15.26:40724) 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.405+0000 7f1238ed6700 1 -- 172.21.15.26:0/1734586076 learned_addr learned my addr 172.21.15.26:0/1734586076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.406+0000 7f1238ed6700 1 -- 172.21.15.26:0/1734586076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123407a880 con 0x7f123407bd10 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.406+0000 7f1238ed6700 1 --2- 172.21.15.26:0/1734586076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12280098a0 tx=0x7f1228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54312ae97f66f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.406+0000 7f12337fe700 1 -- 172.21.15.26:0/1734586076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1228004160 con 0x7f123407bd10 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.406+0000 7f12337fe700 1 -- 172.21.15.26:0/1734586076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12280042c0 con 0x7f123407bd10 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.406+0000 7f12337fe700 1 -- 172.21.15.26:0/1734586076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1228011a00 con 0x7f123407bd10 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 -- 172.21.15.26:0/1734586076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 msgr2=0x7f123407a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 --2- 172.21.15.26:0/1734586076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12280098a0 tx=0x7f1228009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 -- 172.21.15.26:0/1734586076 shutdown_connections 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 --2- 172.21.15.26:0/1734586076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123407a170 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 -- 172.21.15.26:0/1734586076 >> 172.21.15.26:0/1734586076 conn(0x7f1234101ee0 msgr2=0x7f1234104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 -- 172.21.15.26:0/1734586076 shutdown_connections 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.407+0000 7f123b13a700 1 -- 172.21.15.26:0/1734586076 wait complete. 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.408+0000 7f123b13a700 1 Processor -- start 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f123b13a700 1 -- start start 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f123b13a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f123b13a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419cca0 con 0x7f123407bd10 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f1238ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f1238ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40736/0 (socket says 172.21.15.26:40736) 2023-10-06T21:01:10.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f1238ed6700 1 -- 172.21.15.26:0/3359056188 learned_addr learned my addr 172.21.15.26:0/3359056188 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.409+0000 7f1238ed6700 1 -- 172.21.15.26:0/3359056188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228009550 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f1238ed6700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1228009fb0 tx=0x7f1228011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1228003a40 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1228003ba0 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f122801a750 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123419cea0 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.410+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123419d340 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.411+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f1228024b60 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.412+0000 7f1231ffb700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 0x7f122403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.412+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f122804c4b0 con 0x7f123407bd10 2023-10-06T21:01:10.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.412+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123410cd10 con 0x7f123407bd10 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.412+0000 7f1233fff700 1 -- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 msgr2=0x7f122403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.412+0000 7f1233fff700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 0x7f122403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.417+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f122802d3f0 con 0x7f123407bd10 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.613+0000 7f1233fff700 1 -- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 msgr2=0x7f122403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.613+0000 7f1233fff700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 0x7f122403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.686+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f123419fd00 con 0x7f123407bd10 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.687+0000 7f1231ffb700 1 -- 172.21.15.26:0/3359056188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1228024430 con 0x7f123407bd10 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.689+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 msgr2=0x7f122403ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.689+0000 7f123b13a700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 0x7f122403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.689+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 msgr2=0x7f123419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:10.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.689+0000 7f123b13a700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1228009fb0 tx=0x7f1228011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.689+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 shutdown_connections 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.690+0000 7f123b13a700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7f1224038820 0x7f122403ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.690+0000 7f123b13a700 1 --2- 172.21.15.26:0/3359056188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123407bd10 0x7f123419c760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.690+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 >> 172.21.15.26:0/3359056188 conn(0x7f1234101ee0 msgr2=0x7f1234102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.690+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 shutdown_connections 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:09.690+0000 7f123b13a700 1 -- 172.21.15.26:0/3359056188 wait complete. 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2023-10-06T21:01:10.123 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2023-10-06T21:01:10.323 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:10.157+0000 7fbfe2535200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:01:10.740 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3359056188' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T21:01:11.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:10.737+0000 7fbfe2535200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:01:12.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:11.614+0000 7fbfe2535200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:01:12.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:11.723+0000 7fbfe2535200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:01:13.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:12.714+0000 7fbfe2535200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:01:13.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:12.839+0000 7fbfe2535200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:01:13.450 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:13.071+0000 7fbfe2535200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:01:13.732 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:13.447+0000 7fbfe2535200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:01:14.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:13.729+0000 7fbfe2535200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:01:14.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:13.930+0000 7fbfe2535200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:01:14.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:14.287+0000 7fbfe2535200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:01:15.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:15.253+0000 7fbfe2535200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:01:15.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:15.433+0000 7fbfe2535200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:01:16.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:15.691+0000 7fbfe2535200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:01:16.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:15.843+0000 7fbfe2535200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:01:16.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:16.132+0000 7fbfe2535200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:01:17.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:17.248+0000 7fbfe2535200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:01:18.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: Active manager daemon x restarted 2023-10-06T21:01:18.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: Activating manager daemon x 2023-10-06T21:01:18.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: osdmap e2: 0 total, 0 up, 0 in 2023-10-06T21:01:18.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: mgrmap e6: x(active, starting, since 0.00392594s) 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: Manager daemon x is now available 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: Found migration_current of "None". Setting to last migration. 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:18.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:18.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:01:18.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:01:18.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:18.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:18.489 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:18 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.647+0000 7fb2bf8ff700 1 Processor -- start 2023-10-06T21:01:18.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bf8ff700 1 -- start start 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bf8ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bf8ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8107310 con 0x7fb2b8106920 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bd69b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bd69b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40742/0 (socket says 172.21.15.26:40742) 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.648+0000 7fb2bd69b700 1 -- 172.21.15.26:0/3357168705 learned_addr learned my addr 172.21.15.26:0/3357168705 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.649+0000 7fb2bd69b700 1 -- 172.21.15.26:0/3357168705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8107b20 con 0x7fb2b8106920 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.649+0000 7fb2bd69b700 1 --2- 172.21.15.26:0/3357168705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40098a0 tx=0x7fb2b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a9f1a72fcda65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.649+0000 7fb2affff700 1 -- 172.21.15.26:0/3357168705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4004160 con 0x7fb2b8106920 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.649+0000 7fb2affff700 1 -- 172.21.15.26:0/3357168705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2b40042c0 con 0x7fb2b8106920 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.649+0000 7fb2affff700 1 -- 172.21.15.26:0/3357168705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4011a00 con 0x7fb2b8106920 2023-10-06T21:01:18.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/3357168705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 msgr2=0x7fb2b8106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/3357168705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40098a0 tx=0x7fb2b4009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/3357168705 shutdown_connections 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/3357168705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b8106d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/3357168705 >> 172.21.15.26:0/3357168705 conn(0x7fb2b8101ee0 msgr2=0x7fb2b8104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/3357168705 shutdown_connections 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.650+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/3357168705 wait complete. 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.651+0000 7fb2bf8ff700 1 Processor -- start 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.651+0000 7fb2bf8ff700 1 -- start start 2023-10-06T21:01:18.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bf8ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bf8ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b807c020 con 0x7fb2b8106920 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bd69b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bd69b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40752/0 (socket says 172.21.15.26:40752) 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bd69b700 1 -- 172.21.15.26:0/628861101 learned_addr learned my addr 172.21.15.26:0/628861101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.652+0000 7fb2bd69b700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009550 con 0x7fb2b8106920 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2bd69b700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009130 tx=0x7fb2b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4003ab0 con 0x7fb2b8106920 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2b4003c10 con 0x7fb2b8106920 2023-10-06T21:01:18.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2b40247d0 con 0x7fb2b8106920 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b807a130 con 0x7fb2b8106920 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.653+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b807a5d0 con 0x7fb2b8106920 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.654+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fb2b401a430 con 0x7fb2b8106920 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.655+0000 7fb2ae7fc700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.655+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2a403b440 con 0x7fb2a4038870 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.655+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb2b404c5a0 con 0x7fb2b8106920 2023-10-06T21:01:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.655+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.722 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.655+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:01:18.722 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.856+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.722 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:10.856+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:01:18.722 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:11.257+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.722 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:11.257+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:12.058+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:12.058+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:13.659+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:13.659+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:16.863+0000 7fb2bce9a700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3098538275 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:16.863+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:17.253+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) v1 ==== 45346+0+0 (secure 0 0 0) 0x7fb2b401abb0 con 0x7fb2b8106920 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:17.253+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 msgr2=0x7fb2a403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:17.253+0000 7fb2ae7fc700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.255+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7fb2b4017d90 con 0x7fb2b8106920 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.255+0000 7fb2ae7fc700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 0x7fb2a403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:18.723 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.255+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2a403b440 con 0x7fb2a403bfd0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.256+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 0x7fb2a403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.257+0000 7fb2bce9a700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 0x7fb2a403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8003a10 tx=0x7fb2a80092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.257+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fb2a403b440 con 0x7fb2a403bfd0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.264+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb2b807b680 con 0x7fb2a403bfd0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.264+0000 7fb2ae7fc700 1 -- 172.21.15.26:0/628861101 <== mgr.14116 v2:172.21.15.26:6800/118366491 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb2b807b680 con 0x7fb2a403bfd0 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 msgr2=0x7fb2a403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 0x7fb2a403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8003a10 tx=0x7fb2a80092b0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 msgr2=0x7fb2b807bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009130 tx=0x7fb2b4011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 shutdown_connections 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fb2a403bfd0 0x7fb2a403e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.724 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2b8106920 0x7fb2b807bae0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 --2- 172.21.15.26:0/628861101 >> [v2:172.21.15.26:6800/3098538275,v1:172.21.15.26:6801/3098538275] conn(0x7fb2a4038870 0x7fb2a403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 >> 172.21.15.26:0/628861101 conn(0x7fb2b8101ee0 msgr2=0x7fb2b8102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 shutdown_connections 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:18.265+0000 7fb2bf8ff700 1 -- 172.21.15.26:0/628861101 wait complete. 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2023-10-06T21:01:18.725 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2023-10-06T21:01:19.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:17] ENGINE Bus STARTING 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:18] ENGINE Serving on https://172.21.15.26:7150 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:18] ENGINE Serving on http://172.21.15.26:8765 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:18] ENGINE Bus STARTED 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: mgrmap e7: x(active, since 1.00742s) 2023-10-06T21:01:19.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:19 smithi026 ceph-mon[98490]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T21:01:19.991 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.236+0000 7f2beb51b700 1 Processor -- start 2023-10-06T21:01:19.991 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2beb51b700 1 -- start start 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2beb51b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2beb51b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be407a720 con 0x7f2be407bcf0 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2be92b7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2be92b7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56226/0 (socket says 172.21.15.26:56226) 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.237+0000 7f2be92b7700 1 -- 172.21.15.26:0/1473884105 learned_addr learned my addr 172.21.15.26:0/1473884105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.238+0000 7f2be92b7700 1 -- 172.21.15.26:0/1473884105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be407a860 con 0x7f2be407bcf0 2023-10-06T21:01:19.992 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.238+0000 7f2be92b7700 1 --2- 172.21.15.26:0/1473884105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2be00098a0 tx=0x7f2be0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cffda2feea6120 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.238+0000 7f2bdbfff700 1 -- 172.21.15.26:0/1473884105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be0004160 con 0x7f2be407bcf0 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.238+0000 7f2bdbfff700 1 -- 172.21.15.26:0/1473884105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2be00042c0 con 0x7f2be407bcf0 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.238+0000 7f2bdbfff700 1 -- 172.21.15.26:0/1473884105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be0011a00 con 0x7f2be407bcf0 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 -- 172.21.15.26:0/1473884105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 msgr2=0x7f2be407a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1473884105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2be00098a0 tx=0x7f2be0009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 -- 172.21.15.26:0/1473884105 shutdown_connections 2023-10-06T21:01:19.993 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1473884105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be407a150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 -- 172.21.15.26:0/1473884105 >> 172.21.15.26:0/1473884105 conn(0x7f2be4101ec0 msgr2=0x7f2be41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 -- 172.21.15.26:0/1473884105 shutdown_connections 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.239+0000 7f2beb51b700 1 -- 172.21.15.26:0/1473884105 wait complete. 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.240+0000 7f2beb51b700 1 Processor -- start 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.240+0000 7f2beb51b700 1 -- start start 2023-10-06T21:01:19.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.240+0000 7f2beb51b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.240+0000 7f2beb51b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be419cce0 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2be92b7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2be92b7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56236/0 (socket says 172.21.15.26:56236) 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2be92b7700 1 -- 172.21.15.26:0/1128400741 learned_addr learned my addr 172.21.15.26:0/1128400741 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2be92b7700 1 -- 172.21.15.26:0/1128400741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009550 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2be92b7700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009fb0 tx=0x7f2be0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.241+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be0003a40 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.242+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be419cee0 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.242+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2be0003ba0 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.242+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be001a750 con 0x7f2be407bcf0 2023-10-06T21:01:19.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.242+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be419d380 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.243+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7f2be0024b60 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.243+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be410ccb0 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.243+0000 7f2bda7fc700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 0x7f2bd003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.243+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2be004c6c0 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.243+0000 7f2be8ab6700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 0x7f2bd003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.244+0000 7f2be8ab6700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 0x7f2bd003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4006fd0 tx=0x7f2bd4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.249+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2be0024430 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.259+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f2be002d770 con 0x7f2be407bcf0 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.473+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2be4104180 con 0x7f2bd0038700 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.480+0000 7f2bda7fc700 1 -- 172.21.15.26:0/1128400741 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2be4104180 con 0x7f2bd0038700 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.482+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 msgr2=0x7f2bd003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.482+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 0x7f2bd003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4006fd0 tx=0x7f2bd4006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:19.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.482+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 msgr2=0x7f2be419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.482+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009fb0 tx=0x7f2be0011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 shutdown_connections 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f2bd0038700 0x7f2bd003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 --2- 172.21.15.26:0/1128400741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2be407bcf0 0x7f2be419c7a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 >> 172.21.15.26:0/1128400741 conn(0x7f2be4101ec0 msgr2=0x7f2be4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 shutdown_connections 2023-10-06T21:01:20.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:19.483+0000 7f2beb51b700 1 -- 172.21.15.26:0/1128400741 wait complete. 2023-10-06T21:01:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:20 smithi026 ceph-mon[98490]: mgrmap e8: x(active, since 2s) 2023-10-06T21:01:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:20 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:20 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:21.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-06T21:01:21.171 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.519+0000 7fa7a7588700 1 Processor -- start 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a7588700 1 -- start start 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a7588700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a7588700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a007a740 con 0x7fa7a007bd10 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a5324700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a5324700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56250/0 (socket says 172.21.15.26:56250) 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.520+0000 7fa7a5324700 1 -- 172.21.15.26:0/2873493448 learned_addr learned my addr 172.21.15.26:0/2873493448 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.521+0000 7fa7a5324700 1 -- 172.21.15.26:0/2873493448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a007a880 con 0x7fa7a007bd10 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.521+0000 7fa7a5324700 1 --2- 172.21.15.26:0/2873493448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098a0 tx=0x7fa79c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ef53bbed0a333b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.521+0000 7fa797fff700 1 -- 172.21.15.26:0/2873493448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c004160 con 0x7fa7a007bd10 2023-10-06T21:01:21.172 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.521+0000 7fa797fff700 1 -- 172.21.15.26:0/2873493448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa79c0042c0 con 0x7fa7a007bd10 2023-10-06T21:01:21.173 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.521+0000 7fa797fff700 1 -- 172.21.15.26:0/2873493448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c011a00 con 0x7fa7a007bd10 2023-10-06T21:01:21.173 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 -- 172.21.15.26:0/2873493448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 msgr2=0x7fa7a007a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:21.173 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2873493448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098a0 tx=0x7fa79c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 -- 172.21.15.26:0/2873493448 shutdown_connections 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2873493448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a007a170 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 -- 172.21.15.26:0/2873493448 >> 172.21.15.26:0/2873493448 conn(0x7fa7a0101ee0 msgr2=0x7fa7a0104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 -- 172.21.15.26:0/2873493448 shutdown_connections 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.522+0000 7fa7a7588700 1 -- 172.21.15.26:0/2873493448 wait complete. 2023-10-06T21:01:21.176 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.523+0000 7fa7a7588700 1 Processor -- start 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a7588700 1 -- start start 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a7588700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a7588700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a019cca0 con 0x7fa7a007bd10 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a5324700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a5324700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56256/0 (socket says 172.21.15.26:56256) 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.524+0000 7fa7a5324700 1 -- 172.21.15.26:0/2865904962 learned_addr learned my addr 172.21.15.26:0/2865904962 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:21.177 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7a5324700 1 -- 172.21.15.26:0/2865904962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c009550 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7a5324700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009fb0 tx=0x7fa79c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c003a40 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa79c003ba0 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a019cea0 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c01a750 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.525+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a019d340 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.526+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fa79c024b60 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.527+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a010cd10 con 0x7fa7a007bd10 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.527+0000 7fa7967fc700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 0x7fa78c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.527+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa79c04c8b0 con 0x7fa7a007bd10 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.527+0000 7fa7a4b23700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 0x7fa78c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.528+0000 7fa7a4b23700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 0x7fa78c03ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa790006fd0 tx=0x7fa790006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.532+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa79c018530 con 0x7fa7a007bd10 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.740+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a01033e0 con 0x7fa78c0387d0 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.741+0000 7fa7967fc700 1 -- 172.21.15.26:0/2865904962 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa7a01033e0 con 0x7fa78c0387d0 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.743+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 msgr2=0x7fa78c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.743+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 0x7fa78c03ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa790006fd0 tx=0x7fa790006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.743+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 msgr2=0x7fa7a019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009fb0 tx=0x7fa79c011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:21.179 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 shutdown_connections 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7fa78c0387d0 0x7fa78c03ac90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 --2- 172.21.15.26:0/2865904962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7a007bd10 0x7fa7a019c760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 >> 172.21.15.26:0/2865904962 conn(0x7fa7a0101ee0 msgr2=0x7fa7a0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 shutdown_connections 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:20.744+0000 7fa7a7588700 1 -- 172.21.15.26:0/2865904962 wait complete. 2023-10-06T21:01:21.180 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2023-10-06T21:01:21.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:21 smithi026 ceph-mon[98490]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:22.562 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-mon[98490]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:22.562 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: Generating public/private rsa key pair. 2023-10-06T21:01:22.562 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: Your identification has been saved in /tmp/tmpk_o93abn/key. 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: Your public key has been saved in /tmp/tmpk_o93abn/key.pub. 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: The key fingerprint is: 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: SHA256:SWQSzZTRd7LFxTD05v1JUsxNrCK3YJqp5RzuM74hxhA ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: The key's randomart image is: 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: +---[RSA 3072]----+ 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | o==+ o+=o| 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | ++ . o *++| 2023-10-06T21:01:22.563 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | E . . = =+| 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | . . .+ + oo.| 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | . S= + + .o| 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | o * . o o| 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | +*.. ..| 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | ...*. | 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: | o++ | 2023-10-06T21:01:22.564 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: +----[SHA256]-----+ 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.726+0000 7f8195283700 1 Processor -- start 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.726+0000 7f8195283700 1 -- start start 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f8195283700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f8195283700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190109560 con 0x7f8190108b70 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f818effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f818effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50820/0 (socket says 172.21.15.26:50820) 2023-10-06T21:01:22.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f818effd700 1 -- 172.21.15.26:0/228387231 learned_addr learned my addr 172.21.15.26:0/228387231 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.727+0000 7f818effd700 1 -- 172.21.15.26:0/228387231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190109d20 con 0x7f8190108b70 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f818effd700 1 --2- 172.21.15.26:0/228387231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81800098a0 tx=0x7f8180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc9781e48b5bd2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f818dffb700 1 -- 172.21.15.26:0/228387231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8180004160 con 0x7f8190108b70 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f818dffb700 1 -- 172.21.15.26:0/228387231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81800042c0 con 0x7f8190108b70 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f818dffb700 1 -- 172.21.15.26:0/228387231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8180011a00 con 0x7f8190108b70 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f8195283700 1 -- 172.21.15.26:0/228387231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 msgr2=0x7f8190108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.728+0000 7f8195283700 1 --2- 172.21.15.26:0/228387231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81800098a0 tx=0x7f8180009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.729+0000 7f8195283700 1 -- 172.21.15.26:0/228387231 shutdown_connections 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.729+0000 7f8195283700 1 --2- 172.21.15.26:0/228387231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f8190108f90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.729+0000 7f8195283700 1 -- 172.21.15.26:0/228387231 >> 172.21.15.26:0/228387231 conn(0x7f8190104350 msgr2=0x7f8190106740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.729+0000 7f8195283700 1 -- 172.21.15.26:0/228387231 shutdown_connections 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.729+0000 7f8195283700 1 -- 172.21.15.26:0/228387231 wait complete. 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f8195283700 1 Processor -- start 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f8195283700 1 -- start start 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f8195283700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f8195283700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819019cce0 con 0x7f8190108b70 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f818effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f818effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50822/0 (socket says 172.21.15.26:50822) 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.730+0000 7f818effd700 1 -- 172.21.15.26:0/2845801954 learned_addr learned my addr 172.21.15.26:0/2845801954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f818effd700 1 -- 172.21.15.26:0/2845801954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009550 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f818effd700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8180009130 tx=0x7f8180011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8180003ab0 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8180003c10 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819019cee0 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.731+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81800247d0 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.732+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819019d380 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.732+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f818001abb0 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.733+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81900624e0 con 0x7f8190108b70 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.733+0000 7f8177fff700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 0x7f817803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:22.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.733+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f818004c9c0 con 0x7f8190108b70 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.733+0000 7f818e7fc700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 0x7f817803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.734+0000 7f818e7fc700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 0x7f817803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8184006fd0 tx=0x7f8184006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.738+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8180018530 con 0x7f8190108b70 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:21.950+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8190106830 con 0x7f8178038870 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.381+0000 7f8177fff700 1 -- 172.21.15.26:0/2845801954 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8190106830 con 0x7f8178038870 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 msgr2=0x7f817803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 0x7f817803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8184006fd0 tx=0x7f8184006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 msgr2=0x7f819019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8180009130 tx=0x7f8180011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:22.861 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 shutdown_connections 2023-10-06T21:01:22.862 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f8178038870 0x7f817803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:22.862 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 --2- 172.21.15.26:0/2845801954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8190108b70 0x7f819019c7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:22.862 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 >> 172.21.15.26:0/2845801954 conn(0x7f8190104350 msgr2=0x7f8190106120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:22.862 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 shutdown_connections 2023-10-06T21:01:22.862 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:22.384+0000 7f8195283700 1 -- 172.21.15.26:0/2845801954 wait complete. 2023-10-06T21:01:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:23 smithi026 ceph-mon[98490]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:23 smithi026 ceph-mon[98490]: Generating ssh key... 2023-10-06T21:01:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:23 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:23 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:24.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:24.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.410+0000 7f90dd9dc700 1 Processor -- start 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.410+0000 7f90dd9dc700 1 -- start start 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.410+0000 7f90dd9dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.410+0000 7f90dd9dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8107310 con 0x7f90d8106920 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.411+0000 7f90d6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.411+0000 7f90d6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50838/0 (socket says 172.21.15.26:50838) 2023-10-06T21:01:24.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.411+0000 7f90d6ffd700 1 -- 172.21.15.26:0/2992974514 learned_addr learned my addr 172.21.15.26:0/2992974514 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.411+0000 7f90d6ffd700 1 -- 172.21.15.26:0/2992974514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8107b20 con 0x7f90d8106920 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.411+0000 7f90d6ffd700 1 --2- 172.21.15.26:0/2992974514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90c80098a0 tx=0x7f90c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780ede7b50274598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90d5ffb700 1 -- 172.21.15.26:0/2992974514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8004160 con 0x7f90d8106920 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90d5ffb700 1 -- 172.21.15.26:0/2992974514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90c80042c0 con 0x7f90d8106920 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90d5ffb700 1 -- 172.21.15.26:0/2992974514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8011a00 con 0x7f90d8106920 2023-10-06T21:01:24.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 -- 172.21.15.26:0/2992974514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 msgr2=0x7f90d8106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/2992974514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f90c80098a0 tx=0x7f90c8009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 -- 172.21.15.26:0/2992974514 shutdown_connections 2023-10-06T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/2992974514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d8106d40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 -- 172.21.15.26:0/2992974514 >> 172.21.15.26:0/2992974514 conn(0x7f90d8101ee0 msgr2=0x7f90d8104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:24.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 -- 172.21.15.26:0/2992974514 shutdown_connections 2023-10-06T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.412+0000 7f90dd9dc700 1 -- 172.21.15.26:0/2992974514 wait complete. 2023-10-06T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.413+0000 7f90dd9dc700 1 Processor -- start 2023-10-06T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90dd9dc700 1 -- start start 2023-10-06T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90dd9dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:24.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90dd9dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d819cca0 con 0x7f90d8106920 2023-10-06T21:01:24.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90d6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:24.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90d6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50848/0 (socket says 172.21.15.26:50848) 2023-10-06T21:01:24.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90d6ffd700 1 -- 172.21.15.26:0/3566614808 learned_addr learned my addr 172.21.15.26:0/3566614808 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:24.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.414+0000 7f90d6ffd700 1 -- 172.21.15.26:0/3566614808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8009550 con 0x7f90d8106920 2023-10-06T21:01:24.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90d6ffd700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009130 tx=0x7f90c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c8003ab0 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90c8003c10 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90c80247d0 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d819cea0 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.415+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d819d340 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.416+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f90c801a430 con 0x7f90d8106920 2023-10-06T21:01:24.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.416+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80624e0 con 0x7f90d8106920 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.417+0000 7f90dc9da700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 0x7f90c003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.417+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f90c804c9b0 con 0x7f90d8106920 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.417+0000 7f90d67fc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 0x7f90c003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.417+0000 7f90d67fc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 0x7f90c003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006fd0 tx=0x7f90cc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.422+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f90c8018b40 con 0x7f90d8106920 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.636+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f90d810ad20 con 0x7f90c0038870 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.637+0000 7f90dc9da700 1 -- 172.21.15.26:0/3566614808 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f90d810ad20 con 0x7f90c0038870 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 msgr2=0x7f90c003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 0x7f90c003ad30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90cc006fd0 tx=0x7f90cc006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:24.047 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 msgr2=0x7f90d819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009130 tx=0x7f90c8011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 shutdown_connections 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f90c0038870 0x7f90c003ad30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 --2- 172.21.15.26:0/3566614808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d8106920 0x7f90d819c760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 >> 172.21.15.26:0/3566614808 conn(0x7f90d8101ee0 msgr2=0x7f90d8102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 shutdown_connections 2023-10-06T21:01:24.048 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:23.640+0000 7f90dd9dc700 1 -- 172.21.15.26:0/3566614808 wait complete. 2023-10-06T21:01:24.049 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:01:24.049 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2023-10-06T21:01:24.049 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2023-10-06T21:01:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:24 smithi026 ceph-mon[98490]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:25.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:25 smithi026 ceph-mon[98490]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:26.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:26 smithi026 ceph-mon[98490]: Deploying cephadm binary to smithi026 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.581+0000 7f14348fc700 1 Processor -- start 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.581+0000 7f14348fc700 1 -- start start 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.581+0000 7f14348fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.581+0000 7f14348fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143007a760 con 0x7f143007bd30 2023-10-06T21:01:28.631 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.582+0000 7f142e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.582+0000 7f142e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50854/0 (socket says 172.21.15.26:50854) 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.582+0000 7f142e59c700 1 -- 172.21.15.26:0/205366199 learned_addr learned my addr 172.21.15.26:0/205366199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.582+0000 7f142e59c700 1 -- 172.21.15.26:0/205366199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143007a8a0 con 0x7f143007bd30 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.582+0000 7f142e59c700 1 --2- 172.21.15.26:0/205366199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14200098a0 tx=0x7f1420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a5b3c064e3d816 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f142d59a700 1 -- 172.21.15.26:0/205366199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1420004160 con 0x7f143007bd30 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f142d59a700 1 -- 172.21.15.26:0/205366199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14200042c0 con 0x7f143007bd30 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f142d59a700 1 -- 172.21.15.26:0/205366199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1420011a00 con 0x7f143007bd30 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f14348fc700 1 -- 172.21.15.26:0/205366199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 msgr2=0x7f143007a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:28.632 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f14348fc700 1 --2- 172.21.15.26:0/205366199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14200098a0 tx=0x7f1420009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f14348fc700 1 -- 172.21.15.26:0/205366199 shutdown_connections 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f14348fc700 1 --2- 172.21.15.26:0/205366199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143007a190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.583+0000 7f14348fc700 1 -- 172.21.15.26:0/205366199 >> 172.21.15.26:0/205366199 conn(0x7f1430101ec0 msgr2=0x7f14301042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.584+0000 7f14348fc700 1 -- 172.21.15.26:0/205366199 shutdown_connections 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.584+0000 7f14348fc700 1 -- 172.21.15.26:0/205366199 wait complete. 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f14348fc700 1 Processor -- start 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f14348fc700 1 -- start start 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f14348fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f14348fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143019ccc0 con 0x7f143007bd30 2023-10-06T21:01:28.633 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f142e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f142e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50870/0 (socket says 172.21.15.26:50870) 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.585+0000 7f142e59c700 1 -- 172.21.15.26:0/349860330 learned_addr learned my addr 172.21.15.26:0/349860330 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f142e59c700 1 -- 172.21.15.26:0/349860330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420009550 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f142e59c700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1420009130 tx=0x7f1420011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1420003ab0 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1420003c10 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143019cec0 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14200247d0 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.586+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143019d360 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.587+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f142002d3f0 con 0x7f143007bd30 2023-10-06T21:01:28.634 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.587+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143010ccd0 con 0x7f143007bd30 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.588+0000 7f141f7fe700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 0x7f141803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.588+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f142004cbb0 con 0x7f143007bd30 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.588+0000 7f142dd9b700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 0x7f141803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.589+0000 7f142dd9b700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 0x7f141803ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.593+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f142001a8d0 con 0x7f143007bd30 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:24.808+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) v1 -- 0x7f14301033c0 con 0x7f1418038870 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.169+0000 7f141f7fe700 1 -- 172.21.15.26:0/349860330 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f14301033c0 con 0x7f1418038870 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 msgr2=0x7f141803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 0x7f141803ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 msgr2=0x7f143019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:28.635 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1420009130 tx=0x7f1420011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 shutdown_connections 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1418038870 0x7f141803ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 --2- 172.21.15.26:0/349860330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f143007bd30 0x7f143019c780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.172+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 >> 172.21.15.26:0/349860330 conn(0x7f1430101ec0 msgr2=0x7f1430102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.173+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 shutdown_connections 2023-10-06T21:01:28.636 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:28.173+0000 7f14348fc700 1 -- 172.21.15.26:0/349860330 wait complete. 2023-10-06T21:01:28.637 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2023-10-06T21:01:29.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:29 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:29.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:29 smithi026 ceph-mon[98490]: Added host smithi026 2023-10-06T21:01:29.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:29 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:29.911 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.198+0000 7ff1c1701700 1 Processor -- start 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.198+0000 7ff1c1701700 1 -- start start 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.199+0000 7ff1c1701700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.199+0000 7ff1c1701700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc07a740 con 0x7ff1bc07bd10 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.199+0000 7ff1baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.199+0000 7ff1baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50884/0 (socket says 172.21.15.26:50884) 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.199+0000 7ff1baffd700 1 -- 172.21.15.26:0/1599056759 learned_addr learned my addr 172.21.15.26:0/1599056759 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:29.912 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.200+0000 7ff1baffd700 1 -- 172.21.15.26:0/1599056759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc07a880 con 0x7ff1bc07bd10 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.200+0000 7ff1baffd700 1 --2- 172.21.15.26:0/1599056759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098a0 tx=0x7ff1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881f60d567a19563 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.200+0000 7ff1b9ffb700 1 -- 172.21.15.26:0/1599056759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac004160 con 0x7ff1bc07bd10 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.200+0000 7ff1b9ffb700 1 -- 172.21.15.26:0/1599056759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1ac0042c0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.200+0000 7ff1b9ffb700 1 -- 172.21.15.26:0/1599056759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac011a00 con 0x7ff1bc07bd10 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 -- 172.21.15.26:0/1599056759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 msgr2=0x7ff1bc07a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 --2- 172.21.15.26:0/1599056759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098a0 tx=0x7ff1ac009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 -- 172.21.15.26:0/1599056759 shutdown_connections 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 --2- 172.21.15.26:0/1599056759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc07a170 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:29.913 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 -- 172.21.15.26:0/1599056759 >> 172.21.15.26:0/1599056759 conn(0x7ff1bc101ec0 msgr2=0x7ff1bc1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 -- 172.21.15.26:0/1599056759 shutdown_connections 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.201+0000 7ff1c1701700 1 -- 172.21.15.26:0/1599056759 wait complete. 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.202+0000 7ff1c1701700 1 Processor -- start 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.202+0000 7ff1c1701700 1 -- start start 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.202+0000 7ff1c1701700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.202+0000 7ff1c1701700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc19cce0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.203+0000 7ff1baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.203+0000 7ff1baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50896/0 (socket says 172.21.15.26:50896) 2023-10-06T21:01:29.914 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.203+0000 7ff1baffd700 1 -- 172.21.15.26:0/2273559552 learned_addr learned my addr 172.21.15.26:0/2273559552 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.203+0000 7ff1baffd700 1 -- 172.21.15.26:0/2273559552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac009550 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.203+0000 7ff1baffd700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009130 tx=0x7ff1ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.204+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac003b70 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.204+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1ac003cd0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.204+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc19cee0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.204+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1ac0197d0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.204+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc19d380 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.205+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7ff1ac01a430 con 0x7ff1bc07bd10 2023-10-06T21:01:29.915 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.205+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc10ccf0 con 0x7ff1bc07bd10 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.205+0000 7ff1a3fff700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 0x7ff1a403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.205+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff1ac04cb30 con 0x7ff1bc07bd10 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.205+0000 7ff1ba7fc700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 0x7ff1a403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.206+0000 7ff1ba7fc700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 0x7ff1a403ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006fd0 tx=0x7ff1b0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.211+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff1ac02d960 con 0x7ff1bc07bd10 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.439+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff1bc104180 con 0x7ff1a4038820 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.444+0000 7ff1a3fff700 1 -- 172.21.15.26:0/2273559552 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff1bc104180 con 0x7ff1a4038820 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 msgr2=0x7ff1a403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:29.916 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 0x7ff1a403ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006fd0 tx=0x7ff1b0006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 msgr2=0x7ff1bc19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009130 tx=0x7ff1ac011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 shutdown_connections 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7ff1a4038820 0x7ff1a403ace0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 --2- 172.21.15.26:0/2273559552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc07bd10 0x7ff1bc19c7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 >> 172.21.15.26:0/2273559552 conn(0x7ff1bc101ec0 msgr2=0x7ff1bc103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 shutdown_connections 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:29.446+0000 7ff1c1701700 1 -- 172.21.15.26:0/2273559552 wait complete. 2023-10-06T21:01:29.917 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2023-10-06T21:01:30.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:30 smithi026 ceph-mon[98490]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:30.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:30 smithi026 ceph-mon[98490]: Saving service mon spec with placement count:5 2023-10-06T21:01:30.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:30 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:31.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-06T21:01:31.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.605+0000 7f68d4e69700 1 Processor -- start 2023-10-06T21:01:31.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.605+0000 7f68d4e69700 1 -- start start 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68d4e69700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68d4e69700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d007a740 con 0x7f68d007bd10 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50906/0 (socket says 172.21.15.26:50906) 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68ce59c700 1 -- 172.21.15.26:0/96819344 learned_addr learned my addr 172.21.15.26:0/96819344 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68ce59c700 1 -- 172.21.15.26:0/96819344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d007a880 con 0x7f68d007bd10 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.606+0000 7f68ce59c700 1 --2- 172.21.15.26:0/96819344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098a0 tx=0x7f68c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dac5e5fc822a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68cd59a700 1 -- 172.21.15.26:0/96819344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68c0004160 con 0x7f68d007bd10 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68cd59a700 1 -- 172.21.15.26:0/96819344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68c00042c0 con 0x7f68d007bd10 2023-10-06T21:01:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68cd59a700 1 -- 172.21.15.26:0/96819344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68c0011a00 con 0x7f68d007bd10 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 -- 172.21.15.26:0/96819344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 msgr2=0x7f68d007a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 --2- 172.21.15.26:0/96819344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68c00098a0 tx=0x7f68c0009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 -- 172.21.15.26:0/96819344 shutdown_connections 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 --2- 172.21.15.26:0/96819344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d007a170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 -- 172.21.15.26:0/96819344 >> 172.21.15.26:0/96819344 conn(0x7f68d0101ec0 msgr2=0x7f68d01042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.607+0000 7f68d4e69700 1 -- 172.21.15.26:0/96819344 shutdown_connections 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.608+0000 7f68d4e69700 1 -- 172.21.15.26:0/96819344 wait complete. 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.608+0000 7f68d4e69700 1 Processor -- start 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68d4e69700 1 -- start start 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68d4e69700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68d4e69700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d019cce0 con 0x7f68d007bd10 2023-10-06T21:01:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50914/0 (socket says 172.21.15.26:50914) 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68ce59c700 1 -- 172.21.15.26:0/1755318469 learned_addr learned my addr 172.21.15.26:0/1755318469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.609+0000 7f68ce59c700 1 -- 172.21.15.26:0/1755318469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009550 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68ce59c700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009130 tx=0x7f68c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68c0003b70 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68c0003cd0 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68c00197d0 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d019cee0 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.610+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d019d380 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.611+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f68c001abb0 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.611+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d010ccf0 con 0x7f68d007bd10 2023-10-06T21:01:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.612+0000 7f68bf7fe700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 0x7f68b803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.612+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f68c004cb10 con 0x7f68d007bd10 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.612+0000 7f68cdd9b700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 0x7f68b803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.613+0000 7f68cdd9b700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 0x7f68b803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68c4006fd0 tx=0x7f68c4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.617+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f68c001a430 con 0x7f68d007bd10 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.840+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mon.0 v2:172.21.15.26:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68c001a430 con 0x7f68d007bd10 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.857+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f68d0104180 con 0x7f68b8038870 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.862+0000 7f68bf7fe700 1 -- 172.21.15.26:0/1755318469 <== mgr.14116 v2:172.21.15.26:6800/118366491 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f68d0104180 con 0x7f68b8038870 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 msgr2=0x7f68b803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 0x7f68b803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68c4006fd0 tx=0x7f68c4006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 msgr2=0x7f68d019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009130 tx=0x7f68c0011ad0 comp rx=0 tx=0).stop 2023-10-06T21:01:31.345 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 shutdown_connections 2023-10-06T21:01:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68b8038870 0x7f68b803ad30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 --2- 172.21.15.26:0/1755318469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d007bd10 0x7f68d019c7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 >> 172.21.15.26:0/1755318469 conn(0x7f68d0101ec0 msgr2=0x7f68d0103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 shutdown_connections 2023-10-06T21:01:31.346 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:30.865+0000 7f68d4e69700 1 -- 172.21.15.26:0/1755318469 wait complete. 2023-10-06T21:01:32.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:31 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:32.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:31 smithi026 ceph-mon[98490]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:32.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:31 smithi026 ceph-mon[98490]: Saving service mgr spec with placement count:2 2023-10-06T21:01:32.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:31 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:32.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:31 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.884+0000 7f68e8fbb700 1 Processor -- start 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.884+0000 7f68e8fbb700 1 -- start start 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.885+0000 7f68e8fbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.885+0000 7f68e8fbb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41072c0 con 0x7f68e41068d0 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.885+0000 7f68e259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.885+0000 7f68e259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33122/0 (socket says 172.21.15.26:33122) 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.885+0000 7f68e259c700 1 -- 172.21.15.26:0/919714304 learned_addr learned my addr 172.21.15.26:0/919714304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:32.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.886+0000 7f68e259c700 1 -- 172.21.15.26:0/919714304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4107ad0 con 0x7f68e41068d0 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.886+0000 7f68e259c700 1 --2- 172.21.15.26:0/919714304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098a0 tx=0x7f68d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df1744296737dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.886+0000 7f68e159a700 1 -- 172.21.15.26:0/919714304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68d4004160 con 0x7f68e41068d0 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.886+0000 7f68e159a700 1 -- 172.21.15.26:0/919714304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d40042c0 con 0x7f68e41068d0 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.886+0000 7f68e159a700 1 -- 172.21.15.26:0/919714304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68d40119b0 con 0x7f68e41068d0 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 -- 172.21.15.26:0/919714304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 msgr2=0x7f68e4106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/919714304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098a0 tx=0x7f68d4009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 -- 172.21.15.26:0/919714304 shutdown_connections 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/919714304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e4106cf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 -- 172.21.15.26:0/919714304 >> 172.21.15.26:0/919714304 conn(0x7f68e4101e70 msgr2=0x7f68e41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 -- 172.21.15.26:0/919714304 shutdown_connections 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.887+0000 7f68e8fbb700 1 -- 172.21.15.26:0/919714304 wait complete. 2023-10-06T21:01:32.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.888+0000 7f68e8fbb700 1 Processor -- start 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e8fbb700 1 -- start start 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e8fbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e8fbb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e419cd60 con 0x7f68e41068d0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33138/0 (socket says 172.21.15.26:33138) 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.889+0000 7f68e259c700 1 -- 172.21.15.26:0/3199944113 learned_addr learned my addr 172.21.15.26:0/3199944113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68e259c700 1 -- 172.21.15.26:0/3199944113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009550 con 0x7f68e41068d0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68e259c700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68d4003e60 tx=0x7f68d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68d4003a60 con 0x7f68e41068d0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d4003bc0 con 0x7f68e41068d0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68d40196c0 con 0x7f68e41068d0 2023-10-06T21:01:32.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e419cf60 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.890+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e419d400 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.892+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f68d401a430 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.892+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e410ad00 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.892+0000 7f68d37fe700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 0x7f68cc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.892+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f68d404c9e0 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.893+0000 7f68e1d9b700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 0x7f68cc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.893+0000 7f68e1d9b700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 0x7f68cc03ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006fd0 tx=0x7f68d8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:31.898+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f68d4010380 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.109+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f68e419fe20 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.112+0000 7f68d37fe700 1 -- 172.21.15.26:0/3199944113 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f68d40281c0 con 0x7f68e41068d0 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 msgr2=0x7f68cc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:32.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 0x7f68cc03ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f68d8006fd0 tx=0x7f68d8006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 msgr2=0x7f68e419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68d4003e60 tx=0x7f68d4011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 shutdown_connections 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f68cc0388b0 0x7f68cc03ad70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 --2- 172.21.15.26:0/3199944113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e41068d0 0x7f68e419c820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.114+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 >> 172.21.15.26:0/3199944113 conn(0x7f68e4101e70 msgr2=0x7f68e4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.115+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 shutdown_connections 2023-10-06T21:01:32.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:32.115+0000 7f68e8fbb700 1 -- 172.21.15.26:0/3199944113 wait complete. 2023-10-06T21:01:33.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:33 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3199944113' entity='client.admin' 2023-10-06T21:01:33.904 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.045+0000 7f98e4c16700 1 Processor -- start 2023-10-06T21:01:33.904 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.045+0000 7f98e4c16700 1 -- start start 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.046+0000 7f98e4c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.046+0000 7f98e4c16700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0107310 con 0x7f98e0106920 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.046+0000 7f98de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.046+0000 7f98de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33146/0 (socket says 172.21.15.26:33146) 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.046+0000 7f98de59c700 1 -- 172.21.15.26:0/1696220206 learned_addr learned my addr 172.21.15.26:0/1696220206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:33.905 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.047+0000 7f98de59c700 1 -- 172.21.15.26:0/1696220206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0107b20 con 0x7f98e0106920 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.047+0000 7f98de59c700 1 --2- 172.21.15.26:0/1696220206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d00098a0 tx=0x7f98d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115998e15d21e922 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.047+0000 7f98dd59a700 1 -- 172.21.15.26:0/1696220206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98d0004160 con 0x7f98e0106920 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.047+0000 7f98dd59a700 1 -- 172.21.15.26:0/1696220206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98d00042c0 con 0x7f98e0106920 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.047+0000 7f98dd59a700 1 -- 172.21.15.26:0/1696220206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98d00119b0 con 0x7f98e0106920 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 -- 172.21.15.26:0/1696220206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 msgr2=0x7f98e0106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 --2- 172.21.15.26:0/1696220206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d00098a0 tx=0x7f98d0009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 -- 172.21.15.26:0/1696220206 shutdown_connections 2023-10-06T21:01:33.906 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 --2- 172.21.15.26:0/1696220206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e0106d40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 -- 172.21.15.26:0/1696220206 >> 172.21.15.26:0/1696220206 conn(0x7f98e0101ec0 msgr2=0x7f98e0104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 -- 172.21.15.26:0/1696220206 shutdown_connections 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.048+0000 7f98e4c16700 1 -- 172.21.15.26:0/1696220206 wait complete. 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.049+0000 7f98e4c16700 1 Processor -- start 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.049+0000 7f98e4c16700 1 -- start start 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.049+0000 7f98e4c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.049+0000 7f98e4c16700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e007c020 con 0x7f98e0106920 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33160/0 (socket says 172.21.15.26:33160) 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98de59c700 1 -- 172.21.15.26:0/2140433812 learned_addr learned my addr 172.21.15.26:0/2140433812 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98de59c700 1 -- 172.21.15.26:0/2140433812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0009550 con 0x7f98e0106920 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98de59c700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98d0006af0 tx=0x7f98d0011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:33.907 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98d0003a40 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98d0003ba0 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98d0019690 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.050+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e007a130 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.051+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e007a5d0 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.052+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e00624e0 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.052+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f98d001aa60 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.052+0000 7f98cf7fe700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 0x7f98c803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.052+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f98d004ca90 con 0x7f98e0106920 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.053+0000 7f98ddd9b700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 0x7f98c803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.053+0000 7f98ddd9b700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 0x7f98c803ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006fd0 tx=0x7f98d4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:33.908 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.058+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f98d001a430 con 0x7f98e0106920 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.284+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f98e007a980 con 0x7f98e0106920 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.286+0000 7f98cf7fe700 1 -- 172.21.15.26:0/2140433812 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f98d0018350 con 0x7f98e0106920 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.289+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 msgr2=0x7f98c803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.289+0000 7f98e4c16700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 0x7f98c803ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006fd0 tx=0x7f98d4006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.289+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 msgr2=0x7f98e007bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.289+0000 7f98e4c16700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98d0006af0 tx=0x7f98d0011a10 comp rx=0 tx=0).stop 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 shutdown_connections 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f98c8038820 0x7f98c803ace0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 --2- 172.21.15.26:0/2140433812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e0106920 0x7f98e007bae0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 >> 172.21.15.26:0/2140433812 conn(0x7f98e0101ec0 msgr2=0x7f98e01035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 shutdown_connections 2023-10-06T21:01:33.909 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:33.290+0000 7f98e4c16700 1 -- 172.21.15.26:0/2140433812 wait complete. 2023-10-06T21:01:33.910 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2023-10-06T21:01:34.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:34 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2140433812' entity='client.admin' 2023-10-06T21:01:34.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:34 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:35.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2794329046' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-06T21:01:35.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-mon[98490]: from='mgr.14116 172.21.15.26:0/2550269524' entity='mgr.x' 2023-10-06T21:01:35.574 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setuser ceph since I am not root 2023-10-06T21:01:35.574 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setgroup ceph since I am not root 2023-10-06T21:01:35.787 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.465+0000 7f47d93d3700 1 Processor -- start 2023-10-06T21:01:35.787 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.465+0000 7f47d93d3700 1 -- start start 2023-10-06T21:01:35.787 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.465+0000 7f47d93d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:35.787 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.465+0000 7f47d93d3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d407a740 con 0x7f47d407bd10 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.466+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.466+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33168/0 (socket says 172.21.15.26:33168) 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.466+0000 7f47d2ffd700 1 -- 172.21.15.26:0/463505310 learned_addr learned my addr 172.21.15.26:0/463505310 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.466+0000 7f47d2ffd700 1 -- 172.21.15.26:0/463505310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d407a880 con 0x7f47d407bd10 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d2ffd700 1 --2- 172.21.15.26:0/463505310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47c40098a0 tx=0x7f47c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5359253cb600f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d1ffb700 1 -- 172.21.15.26:0/463505310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47c4004160 con 0x7f47d407bd10 2023-10-06T21:01:35.788 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d1ffb700 1 -- 172.21.15.26:0/463505310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c40042c0 con 0x7f47d407bd10 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d1ffb700 1 -- 172.21.15.26:0/463505310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47c40119b0 con 0x7f47d407bd10 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d93d3700 1 -- 172.21.15.26:0/463505310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 msgr2=0x7f47d407a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.467+0000 7f47d93d3700 1 --2- 172.21.15.26:0/463505310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47c40098a0 tx=0x7f47c4009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.468+0000 7f47d93d3700 1 -- 172.21.15.26:0/463505310 shutdown_connections 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.468+0000 7f47d93d3700 1 --2- 172.21.15.26:0/463505310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d407a170 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.468+0000 7f47d93d3700 1 -- 172.21.15.26:0/463505310 >> 172.21.15.26:0/463505310 conn(0x7f47d4101ec0 msgr2=0x7f47d41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.468+0000 7f47d93d3700 1 -- 172.21.15.26:0/463505310 shutdown_connections 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.468+0000 7f47d93d3700 1 -- 172.21.15.26:0/463505310 wait complete. 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.469+0000 7f47d93d3700 1 Processor -- start 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d93d3700 1 -- start start 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d93d3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d93d3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419ccf0 con 0x7f47d407bd10 2023-10-06T21:01:35.790 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33172/0 (socket says 172.21.15.26:33172) 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.470+0000 7f47d2ffd700 1 -- 172.21.15.26:0/2794329046 learned_addr learned my addr 172.21.15.26:0/2794329046 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47d2ffd700 1 -- 172.21.15.26:0/2794329046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4009550 con 0x7f47d407bd10 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47d2ffd700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003f00 tx=0x7f47c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47c4003b20 con 0x7f47d407bd10 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c4003c80 con 0x7f47d407bd10 2023-10-06T21:01:35.791 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47c4019770 con 0x7f47d407bd10 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d419cef0 con 0x7f47d407bd10 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.471+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d419d390 con 0x7f47d407bd10 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.473+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f47c401ab60 con 0x7f47d407bd10 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.473+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d410cd40 con 0x7f47d407bd10 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.473+0000 7f47bbfff700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 0x7f47bc03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.473+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f47c404cb50 con 0x7f47d407bd10 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.474+0000 7f47d27fc700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 0x7f47bc03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.474+0000 7f47d27fc700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 0x7f47bc03ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47c8006fd0 tx=0x7f47c8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.478+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f47c401a430 con 0x7f47d407bd10 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:34.756+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f47d410cfb0 con 0x7f47d407bd10 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.289+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f47c402db40 con 0x7f47d407bd10 2023-10-06T21:01:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.291+0000 7f47bbfff700 1 -- 172.21.15.26:0/2794329046 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f47c402d350 con 0x7f47d407bd10 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 msgr2=0x7f47bc03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 0x7f47bc03ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47c8006fd0 tx=0x7f47c8006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 msgr2=0x7f47d419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47c4003f00 tx=0x7f47c4011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 shutdown_connections 2023-10-06T21:01:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f47bc038870 0x7f47bc03ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 --2- 172.21.15.26:0/2794329046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47d407bd10 0x7f47d419c7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 >> 172.21.15.26:0/2794329046 conn(0x7f47d4101ec0 msgr2=0x7f47d4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 shutdown_connections 2023-10-06T21:01:35.795 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:35.294+0000 7f47d93d3700 1 -- 172.21.15.26:0/2794329046 wait complete. 2023-10-06T21:01:35.971 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:35.603+0000 7fc6232d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:01:35.972 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:35.711+0000 7fc6232d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:01:35.972 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:35.840+0000 7fc6232d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:01:36.222 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:35.970+0000 7fc6232d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:01:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:36 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2794329046' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-06T21:01:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:36 smithi026 ceph-mon[98490]: mgrmap e9: x(active, since 18s) 2023-10-06T21:01:36.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:36.220+0000 7fc6232d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:01:36.939 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:36.754+0000 7fc6232d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:01:37.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:36.937+0000 7fc6232d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:01:37.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:37.072+0000 7fc6232d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:01:37.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:37.202+0000 7fc6232d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:01:37.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:01:37.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.390+0000 7f32f0db0700 1 Processor -- start 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32f0db0700 1 -- start start 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32f0db0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32f0db0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec07a740 con 0x7f32ec07bd10 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33208/0 (socket says 172.21.15.26:33208) 2023-10-06T21:01:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.391+0000 7f32ea59c700 1 -- 172.21.15.26:0/2552117024 learned_addr learned my addr 172.21.15.26:0/2552117024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.392+0000 7f32ea59c700 1 -- 172.21.15.26:0/2552117024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec07a880 con 0x7f32ec07bd10 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.392+0000 7f32ea59c700 1 --2- 172.21.15.26:0/2552117024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098a0 tx=0x7f32dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d39a19cd05e6aca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.392+0000 7f32e959a700 1 -- 172.21.15.26:0/2552117024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc004160 con 0x7f32ec07bd10 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.392+0000 7f32e959a700 1 -- 172.21.15.26:0/2552117024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32dc0042c0 con 0x7f32ec07bd10 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.392+0000 7f32e959a700 1 -- 172.21.15.26:0/2552117024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc0119b0 con 0x7f32ec07bd10 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 -- 172.21.15.26:0/2552117024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 msgr2=0x7f32ec07a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2552117024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098a0 tx=0x7f32dc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 -- 172.21.15.26:0/2552117024 shutdown_connections 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2552117024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec07a170 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 -- 172.21.15.26:0/2552117024 >> 172.21.15.26:0/2552117024 conn(0x7f32ec101ee0 msgr2=0x7f32ec104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 -- 172.21.15.26:0/2552117024 shutdown_connections 2023-10-06T21:01:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.393+0000 7f32f0db0700 1 -- 172.21.15.26:0/2552117024 wait complete. 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.394+0000 7f32f0db0700 1 Processor -- start 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.394+0000 7f32f0db0700 1 -- start start 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.394+0000 7f32f0db0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32f0db0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec19ccd0 con 0x7f32ec07bd10 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33220/0 (socket says 172.21.15.26:33220) 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32ea59c700 1 -- 172.21.15.26:0/2526574768 learned_addr learned my addr 172.21.15.26:0/2526574768 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32ea59c700 1 -- 172.21.15.26:0/2526574768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc009550 con 0x7f32ec07bd10 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.395+0000 7f32ea59c700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32dc003f00 tx=0x7f32dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.396+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc003b20 con 0x7f32ec07bd10 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.396+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec19ced0 con 0x7f32ec07bd10 2023-10-06T21:01:37.453 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.396+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32dc003c80 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.396+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32dc019770 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.396+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec19d370 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.397+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec10cd60 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.397+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f32dc01ab60 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.398+0000 7f32db7fe700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 0x7f32d403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.398+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f32dc04cb80 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.398+0000 7f32e9d9b700 1 -- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 msgr2=0x7f32d403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.398+0000 7f32e9d9b700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 0x7f32d403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.402+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f32dc04ee60 con 0x7f32ec07bd10 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.599+0000 7f32e9d9b700 1 -- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 msgr2=0x7f32d403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:37.454 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.599+0000 7f32e9d9b700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 0x7f32d403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.690+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f32ec10cfd0 con 0x7f32ec07bd10 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.691+0000 7f32db7fe700 1 -- 172.21.15.26:0/2526574768 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f32dc01a430 con 0x7f32ec07bd10 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 msgr2=0x7f32d403ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 0x7f32d403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 msgr2=0x7f32ec19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32dc003f00 tx=0x7f32dc011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 shutdown_connections 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f32d40388c0 0x7f32d403ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 --2- 172.21.15.26:0/2526574768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32ec07bd10 0x7f32ec19c790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 >> 172.21.15.26:0/2526574768 conn(0x7f32ec101ee0 msgr2=0x7f32ec102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.693+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 shutdown_connections 2023-10-06T21:01:37.455 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:36.694+0000 7f32f0db0700 1 -- 172.21.15.26:0/2526574768 wait complete. 2023-10-06T21:01:37.456 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2023-10-06T21:01:37.456 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2023-10-06T21:01:37.501 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:37 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2526574768' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-06T21:01:38.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:37.787+0000 7fc6232d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:01:39.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:38.668+0000 7fc6232d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:01:39.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:38.777+0000 7fc6232d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:01:40.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:39.765+0000 7fc6232d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:01:40.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:39.889+0000 7fc6232d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:01:40.497 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:40 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:40.121+0000 7fc6232d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:01:40.780 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:40 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:40.495+0000 7fc6232d1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:01:41.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:40 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:40.777+0000 7fc6232d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:01:41.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:40 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:40.977+0000 7fc6232d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:01:41.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:41 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:41.332+0000 7fc6232d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:01:42.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:42 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:42.295+0000 7fc6232d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:01:42.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:42 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:42.475+0000 7fc6232d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:01:43.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:42 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:42.731+0000 7fc6232d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:01:43.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:42 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:42.882+0000 7fc6232d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:01:43.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:43 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:43.169+0000 7fc6232d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:01:44.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:01:44.281+0000 7fc6232d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:01:44.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: Active manager daemon x restarted 2023-10-06T21:01:44.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: Activating manager daemon x 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: osdmap e3: 0 total, 0 up, 0 in 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: mgrmap e10: x(active, starting, since 0.00381499s) 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: Manager daemon x is now available 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:01:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:01:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:45 smithi026 ceph-mon[98490]: mgrmap e11: x(active, since 1.00812s) 2023-10-06T21:01:45.841 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.972+0000 7f1be58c8700 1 Processor -- start 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.972+0000 7f1be58c8700 1 -- start start 2023-10-06T21:01:45.848 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1be58c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1be58c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0107310 con 0x7f1be0106920 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33236/0 (socket says 172.21.15.26:33236) 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1bdeffd700 1 -- 172.21.15.26:0/2272264456 learned_addr learned my addr 172.21.15.26:0/2272264456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.973+0000 7f1bdeffd700 1 -- 172.21.15.26:0/2272264456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be0107b20 con 0x7f1be0106920 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.974+0000 7f1bdeffd700 1 --2- 172.21.15.26:0/2272264456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00098a0 tx=0x7f1bd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616684b39d8ce93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.974+0000 7f1bddffb700 1 -- 172.21.15.26:0/2272264456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd0004160 con 0x7f1be0106920 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.974+0000 7f1bddffb700 1 -- 172.21.15.26:0/2272264456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd00042c0 con 0x7f1be0106920 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.974+0000 7f1bddffb700 1 -- 172.21.15.26:0/2272264456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd00119b0 con 0x7f1be0106920 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 -- 172.21.15.26:0/2272264456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 msgr2=0x7f1be0106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:45.849 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 --2- 172.21.15.26:0/2272264456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00098a0 tx=0x7f1bd0009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 -- 172.21.15.26:0/2272264456 shutdown_connections 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 --2- 172.21.15.26:0/2272264456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be0106d40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 -- 172.21.15.26:0/2272264456 >> 172.21.15.26:0/2272264456 conn(0x7f1be0101ee0 msgr2=0x7f1be0104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 -- 172.21.15.26:0/2272264456 shutdown_connections 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.975+0000 7f1be58c8700 1 -- 172.21.15.26:0/2272264456 wait complete. 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.976+0000 7f1be58c8700 1 Processor -- start 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.976+0000 7f1be58c8700 1 -- start start 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.976+0000 7f1be58c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.976+0000 7f1be58c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be007c020 con 0x7f1be0106920 2023-10-06T21:01:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33240/0 (socket says 172.21.15.26:33240) 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1bdeffd700 1 -- 172.21.15.26:0/4030808042 learned_addr learned my addr 172.21.15.26:0/4030808042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1bdeffd700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0009550 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1bdeffd700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0003e60 tx=0x7f1bd0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.977+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd0003a60 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.978+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd0003bc0 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.978+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd00196c0 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.978+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be007a130 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.978+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be007a5d0 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.979+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f1bd001ab60 con 0x7f1be0106920 2023-10-06T21:01:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.979+0000 7f1be48c6700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.979+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bc803b470 con 0x7f1bc80388a0 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.980+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1bd004cba0 con 0x7f1be0106920 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.980+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:37.980+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:38.181+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:38.181+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:38.582+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:38.582+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:39.383+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:39.383+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:40.983+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:40.984+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:44.187+0000 7f1bde7fc700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/118366491 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:44.187+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:44.286+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) v1 ==== 45359+0+0 (secure 0 0 0) 0x7f1bd0019b60 con 0x7f1be0106920 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:44.286+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 msgr2=0x7f1bc803ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:44.286+0000 7f1be48c6700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.289+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) v1 ==== 45486+0+0 (secure 0 0 0) 0x7f1bd004e6b0 con 0x7f1be0106920 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.289+0000 7f1be48c6700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 0x7f1bc803e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.289+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bc803b470 con 0x7f1bc803c310 2023-10-06T21:01:45.853 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.290+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 0x7f1bc803e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.291+0000 7f1bde7fc700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 0x7f1bc803e700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4003a10 tx=0x7f1bd4009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.292+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f1bc803b470 con 0x7f1bc803c310 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.305+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1be007b680 con 0x7f1bc803c310 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be48c6700 1 -- 172.21.15.26:0/4030808042 <== mgr.14148 v2:172.21.15.26:6800/2830651730 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f1be007b680 con 0x7f1bc803c310 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 msgr2=0x7f1bc803e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 0x7f1bc803e700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4003a10 tx=0x7f1bd4009270 comp rx=0 tx=0).stop 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 msgr2=0x7f1be007bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0003e60 tx=0x7f1bd0011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 shutdown_connections 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1bc803c310 0x7f1bc803e700 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.854 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be0106920 0x7f1be007bae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 --2- 172.21.15.26:0/4030808042 >> [v2:172.21.15.26:6800/118366491,v1:172.21.15.26:6801/118366491] conn(0x7f1bc80388a0 0x7f1bc803ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 >> 172.21.15.26:0/4030808042 conn(0x7f1be0101ee0 msgr2=0x7f1be0102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 shutdown_connections 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:45.306+0000 7f1be58c8700 1 -- 172.21.15.26:0/4030808042 wait complete. 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2023-10-06T21:01:45.855 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2023-10-06T21:01:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:44] ENGINE Bus STARTING 2023-10-06T21:01:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:45] ENGINE Serving on https://172.21.15.26:7150 2023-10-06T21:01:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:45] ENGINE Serving on http://172.21.15.26:8765 2023-10-06T21:01:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: [06/Oct/2023:21:01:45] ENGINE Bus STARTED 2023-10-06T21:01:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-06T21:01:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:46 smithi026 ceph-mon[98490]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-06T21:01:47.427 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.590+0000 7fec6750b700 1 Processor -- start 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.590+0000 7fec6750b700 1 -- start start 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.590+0000 7fec6750b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.590+0000 7fec6750b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60107310 con 0x7fec60106920 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec652a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec652a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54592/0 (socket says 172.21.15.26:54592) 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec652a7700 1 -- 172.21.15.26:0/3263804657 learned_addr learned my addr 172.21.15.26:0/3263804657 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec652a7700 1 -- 172.21.15.26:0/3263804657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60107b20 con 0x7fec60106920 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec652a7700 1 --2- 172.21.15.26:0/3263804657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0098a0 tx=0x7fec5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc0b849c406a5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:47.428 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.591+0000 7fec57fff700 1 -- 172.21.15.26:0/3263804657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec5c004160 con 0x7fec60106920 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec57fff700 1 -- 172.21.15.26:0/3263804657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5c0042c0 con 0x7fec60106920 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec57fff700 1 -- 172.21.15.26:0/3263804657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec5c0119b0 con 0x7fec60106920 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 -- 172.21.15.26:0/3263804657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 msgr2=0x7fec60106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 --2- 172.21.15.26:0/3263804657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0098a0 tx=0x7fec5c009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 -- 172.21.15.26:0/3263804657 shutdown_connections 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 --2- 172.21.15.26:0/3263804657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec60106d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 -- 172.21.15.26:0/3263804657 >> 172.21.15.26:0/3263804657 conn(0x7fec60101ee0 msgr2=0x7fec60104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 -- 172.21.15.26:0/3263804657 shutdown_connections 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.592+0000 7fec6750b700 1 -- 172.21.15.26:0/3263804657 wait complete. 2023-10-06T21:01:47.429 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.593+0000 7fec6750b700 1 Processor -- start 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec6750b700 1 -- start start 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec6750b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec6750b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6019ccd0 con 0x7fec60106920 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec652a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec652a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54594/0 (socket says 172.21.15.26:54594) 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec652a7700 1 -- 172.21.15.26:0/739700229 learned_addr learned my addr 172.21.15.26:0/739700229 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.594+0000 7fec652a7700 1 -- 172.21.15.26:0/739700229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009550 con 0x7fec60106920 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec652a7700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006af0 tx=0x7fec5c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec5c003a40 con 0x7fec60106920 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5c003ba0 con 0x7fec60106920 2023-10-06T21:01:47.430 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec5c019690 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6019ced0 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.595+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6019d370 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.596+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6010ad50 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.596+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fec5c01aa60 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.597+0000 7fec567fc700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 0x7fec4c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.597+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec5c04cb30 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.597+0000 7fec64aa6700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 0x7fec4c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.597+0000 7fec64aa6700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 0x7fec4c03ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec50006fd0 tx=0x7fec50006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.602+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec5c01a430 con 0x7fec60106920 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.819+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fec6019fd90 con 0x7fec4c038870 2023-10-06T21:01:47.431 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.946+0000 7fec567fc700 1 -- 172.21.15.26:0/739700229 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fec6019fd90 con 0x7fec4c038870 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 msgr2=0x7fec4c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 0x7fec4c03ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec50006fd0 tx=0x7fec50006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 msgr2=0x7fec6019c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006af0 tx=0x7fec5c011a10 comp rx=0 tx=0).stop 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 shutdown_connections 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c038870 0x7fec4c03ad30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 --2- 172.21.15.26:0/739700229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec60106920 0x7fec6019c790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:47.432 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 >> 172.21.15.26:0/739700229 conn(0x7fec60101ee0 msgr2=0x7fec60102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:47.433 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 shutdown_connections 2023-10-06T21:01:47.433 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:46.949+0000 7fec6750b700 1 -- 172.21.15.26:0/739700229 wait complete. 2023-10-06T21:01:47.433 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2023-10-06T21:01:47.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:47 smithi026 ceph-mon[98490]: mgrmap e12: x(active, since 2s) 2023-10-06T21:01:47.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:47.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:47.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:48.596 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:48 smithi026 ceph-mon[98490]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:48.596 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:48.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pHaCF5I9CDNaE..J.sVdqO2bF7Z.7q.ixZn3NjZflHNR47BhXaOZ.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696626108, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-06T21:01:48.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.968+0000 7f19f0957700 1 Processor -- start 2023-10-06T21:01:48.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.968+0000 7f19f0957700 1 -- start start 2023-10-06T21:01:48.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.968+0000 7f19f0957700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:48.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.968+0000 7f19f0957700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec109560 con 0x7f19ec108b70 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.969+0000 7f19ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.969+0000 7f19ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54608/0 (socket says 172.21.15.26:54608) 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.969+0000 7f19ea59c700 1 -- 172.21.15.26:0/2522000372 learned_addr learned my addr 172.21.15.26:0/2522000372 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.969+0000 7f19ea59c700 1 -- 172.21.15.26:0/2522000372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec109d20 con 0x7f19ec108b70 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.969+0000 7f19ea59c700 1 --2- 172.21.15.26:0/2522000372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0098a0 tx=0x7f19dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4807a989bdf9aeba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19e959a700 1 -- 172.21.15.26:0/2522000372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc004160 con 0x7f19ec108b70 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19e959a700 1 -- 172.21.15.26:0/2522000372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19dc0042c0 con 0x7f19ec108b70 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19e959a700 1 -- 172.21.15.26:0/2522000372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0119b0 con 0x7f19ec108b70 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 -- 172.21.15.26:0/2522000372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 msgr2=0x7f19ec108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:48.821 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 --2- 172.21.15.26:0/2522000372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0098a0 tx=0x7f19dc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 -- 172.21.15.26:0/2522000372 shutdown_connections 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 --2- 172.21.15.26:0/2522000372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec108f90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 -- 172.21.15.26:0/2522000372 >> 172.21.15.26:0/2522000372 conn(0x7f19ec104350 msgr2=0x7f19ec106740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.970+0000 7f19f0957700 1 -- 172.21.15.26:0/2522000372 shutdown_connections 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.971+0000 7f19f0957700 1 -- 172.21.15.26:0/2522000372 wait complete. 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.971+0000 7f19f0957700 1 Processor -- start 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19f0957700 1 -- start start 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19f0957700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19f0957700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19cd30 con 0x7f19ec108b70 2023-10-06T21:01:48.822 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54618/0 (socket says 172.21.15.26:54618) 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.972+0000 7f19ea59c700 1 -- 172.21.15.26:0/225486728 learned_addr learned my addr 172.21.15.26:0/225486728 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19ea59c700 1 -- 172.21.15.26:0/225486728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009550 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19ea59c700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19dc003e60 tx=0x7f19dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc003a60 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19dc003bc0 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0196c0 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec19cf30 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.973+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec19d3d0 con 0x7f19ec108b70 2023-10-06T21:01:48.823 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.974+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f19dc01ab60 con 0x7f19ec108b70 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.974+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec10cf50 con 0x7f19ec108b70 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.975+0000 7f19db7fe700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 0x7f19d403ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.975+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19dc04cc20 con 0x7f19ec108b70 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.975+0000 7f19e9d9b700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 0x7f19d403ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.976+0000 7f19e9d9b700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 0x7f19d403ad60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:47.980+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f19dc019930 con 0x7f19ec108b70 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.210+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f19ec062300 con 0x7f19d40388a0 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.458+0000 7f19db7fe700 1 -- 172.21.15.26:0/225486728 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f19ec062300 con 0x7f19d40388a0 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 msgr2=0x7f19d403ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:48.824 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 0x7f19d403ad60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 msgr2=0x7f19ec19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19dc003e60 tx=0x7f19dc011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 shutdown_connections 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f19d40388a0 0x7f19d403ad60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 --2- 172.21.15.26:0/225486728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ec108b70 0x7f19ec19c7f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 >> 172.21.15.26:0/225486728 conn(0x7f19ec104350 msgr2=0x7f19ec1069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.460+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 shutdown_connections 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:48.461+0000 7f19f0957700 1 -- 172.21.15.26:0/225486728 wait complete. 2023-10-06T21:01:48.825 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2023-10-06T21:01:49.603 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:49 smithi026 ceph-mon[98490]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:01:49.604 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.387+0000 7f5911f29700 1 Processor -- start 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.388+0000 7f5911f29700 1 -- start start 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.388+0000 7f5911f29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.388+0000 7f5911f29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c07a740 con 0x7f590c07bd10 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54628/0 (socket says 172.21.15.26:54628) 2023-10-06T21:01:50.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590b7fe700 1 -- 172.21.15.26:0/1245895428 learned_addr learned my addr 172.21.15.26:0/1245895428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590b7fe700 1 -- 172.21.15.26:0/1245895428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c07a880 con 0x7f590c07bd10 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590b7fe700 1 --2- 172.21.15.26:0/1245895428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098a0 tx=0x7f58fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b96398398e2d92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590a7fc700 1 -- 172.21.15.26:0/1245895428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58fc004160 con 0x7f590c07bd10 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.389+0000 7f590a7fc700 1 -- 172.21.15.26:0/1245895428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc0042c0 con 0x7f590c07bd10 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f590a7fc700 1 -- 172.21.15.26:0/1245895428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58fc0119b0 con 0x7f590c07bd10 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 -- 172.21.15.26:0/1245895428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 msgr2=0x7f590c07a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 --2- 172.21.15.26:0/1245895428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0098a0 tx=0x7f58fc009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 -- 172.21.15.26:0/1245895428 shutdown_connections 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 --2- 172.21.15.26:0/1245895428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c07a170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 -- 172.21.15.26:0/1245895428 >> 172.21.15.26:0/1245895428 conn(0x7f590c101ec0 msgr2=0x7f590c1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 -- 172.21.15.26:0/1245895428 shutdown_connections 2023-10-06T21:01:50.302 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.390+0000 7f5911f29700 1 -- 172.21.15.26:0/1245895428 wait complete. 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.391+0000 7f5911f29700 1 Processor -- start 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f5911f29700 1 -- start start 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f5911f29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f5911f29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c19ccf0 con 0x7f590c07bd10 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54630/0 (socket says 172.21.15.26:54630) 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.392+0000 7f590b7fe700 1 -- 172.21.15.26:0/3831214291 learned_addr learned my addr 172.21.15.26:0/3831214291 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f590b7fe700 1 -- 172.21.15.26:0/3831214291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc009550 con 0x7f590c07bd10 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f590b7fe700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58fc003f00 tx=0x7f58fc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58fc003b20 con 0x7f590c07bd10 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc003c80 con 0x7f590c07bd10 2023-10-06T21:01:50.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c19cef0 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58fc019770 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.393+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c19d390 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.395+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f58fc01ab60 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.395+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c10cd40 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.395+0000 7f5908ff9700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 0x7f58f403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.395+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58fc04cc80 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.396+0000 7f590affd700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 0x7f58f403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.397+0000 7f590affd700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 0x7f58f403ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.403+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58fc01a430 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.635+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f590c10cfb0 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.635+0000 7f5908ff9700 1 -- 172.21.15.26:0/3831214291 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f58fc02db40 con 0x7f590c07bd10 2023-10-06T21:01:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 msgr2=0x7f58f403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 0x7f58f403ad80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 msgr2=0x7f590c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58fc003f00 tx=0x7f58fc011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 shutdown_connections 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58f40388c0 0x7f58f403ad80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 --2- 172.21.15.26:0/3831214291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c07bd10 0x7f590c19c7b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 >> 172.21.15.26:0/3831214291 conn(0x7f590c101ec0 msgr2=0x7f590c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 shutdown_connections 2023-10-06T21:01:50.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:49.638+0000 7f5911f29700 1 -- 172.21.15.26:0/3831214291 wait complete. 2023-10-06T21:01:50.318 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-06T21:01:50.318 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-10-06T21:01:50.335 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-06T21:01:50.336 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-10-06T21:01:50.336 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-10-06T21:01:50.336 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-06T21:01:50.338 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2023-10-06T21:01:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:50.338 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026.front.sepia.ceph.com:8443/ 2023-10-06T21:01:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2023-10-06T21:01:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: Password: 5mp3rj8tpx 2023-10-06T21:01:50.339 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:50.339 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config directory 2023-10-06T21:01:50.340 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling autotune for osd_memory_target 2023-10-06T21:01:50.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3831214291' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-06T21:01:51.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.834+0000 7f47847fb700 1 Processor -- start 2023-10-06T21:01:51.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.834+0000 7f47847fb700 1 -- start start 2023-10-06T21:01:51.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.834+0000 7f47847fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:51.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.834+0000 7f47847fb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c07a760 con 0x7f477c07bd30 2023-10-06T21:01:51.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.835+0000 7f4782597700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.835+0000 7f4782597700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54634/0 (socket says 172.21.15.26:54634) 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.835+0000 7f4782597700 1 -- 172.21.15.26:0/910818246 learned_addr learned my addr 172.21.15.26:0/910818246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.835+0000 7f4782597700 1 -- 172.21.15.26:0/910818246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c07a8a0 con 0x7f477c07bd30 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.835+0000 7f4782597700 1 --2- 172.21.15.26:0/910818246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47780098a0 tx=0x7f4778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c404b0470f92a314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f4781595700 1 -- 172.21.15.26:0/910818246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4778004160 con 0x7f477c07bd30 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f4781595700 1 -- 172.21.15.26:0/910818246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47780042c0 con 0x7f477c07bd30 2023-10-06T21:01:51.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f4781595700 1 -- 172.21.15.26:0/910818246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47780119b0 con 0x7f477c07bd30 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 -- 172.21.15.26:0/910818246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 msgr2=0x7f477c07a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 --2- 172.21.15.26:0/910818246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47780098a0 tx=0x7f4778009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 -- 172.21.15.26:0/910818246 shutdown_connections 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 --2- 172.21.15.26:0/910818246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c07a190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 -- 172.21.15.26:0/910818246 >> 172.21.15.26:0/910818246 conn(0x7f477c101ec0 msgr2=0x7f477c1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.836+0000 7f47847fb700 1 -- 172.21.15.26:0/910818246 shutdown_connections 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.837+0000 7f47847fb700 1 -- 172.21.15.26:0/910818246 wait complete. 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.837+0000 7f47847fb700 1 Processor -- start 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f47847fb700 1 -- start start 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f47847fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:51.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f47847fb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c19cce0 con 0x7f477c07bd30 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f4782597700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f4782597700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54640/0 (socket says 172.21.15.26:54640) 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f4782597700 1 -- 172.21.15.26:0/1543470362 learned_addr learned my addr 172.21.15.26:0/1543470362 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.838+0000 7f4782597700 1 -- 172.21.15.26:0/1543470362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009550 con 0x7f477c07bd30 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f4782597700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4778003e60 tx=0x7f4778011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4778003a60 con 0x7f477c07bd30 2023-10-06T21:01:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4778003bc0 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c19cee0 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47780196c0 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.839+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c19d380 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.840+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f477801a430 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.840+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c10cd20 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.841+0000 7f47737fe700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 0x7f476803ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.841+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f477804ccf0 con 0x7f477c07bd30 2023-10-06T21:01:51.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.841+0000 7f4781d96700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 0x7f476803ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.841+0000 7f4781d96700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 0x7f476803ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:50.846+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4778010380 con 0x7f477c07bd30 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.051+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f477c10cf60 con 0x7f477c07bd30 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.051+0000 7f47737fe700 1 -- 172.21.15.26:0/1543470362 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4778010380 con 0x7f477c07bd30 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.053+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 msgr2=0x7f476803ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.053+0000 7f47847fb700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 0x7f476803ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 msgr2=0x7f477c19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4778003e60 tx=0x7f4778011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:51.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 shutdown_connections 2023-10-06T21:01:51.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4768038940 0x7f476803ae00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:51.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 --2- 172.21.15.26:0/1543470362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f477c07bd30 0x7f477c19c7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:51.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 >> 172.21.15.26:0/1543470362 conn(0x7f477c101ec0 msgr2=0x7f477c102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:51.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 shutdown_connections 2023-10-06T21:01:51.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:51.054+0000 7f47847fb700 1 -- 172.21.15.26:0/1543470362 wait complete. 2023-10-06T21:01:51.787 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:51 smithi026 ceph-mon[98490]: mgrmap e13: x(active, since 6s) 2023-10-06T21:01:52.810 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.049+0000 7f81a8a9d700 1 Processor -- start 2023-10-06T21:01:52.811 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a8a9d700 1 -- start start 2023-10-06T21:01:52.811 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a8a9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:52.811 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a8a9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a407a760 con 0x7f81a407bd30 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53454/0 (socket says 172.21.15.26:53454) 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.050+0000 7f81a259c700 1 -- 172.21.15.26:0/2689370108 learned_addr learned my addr 172.21.15.26:0/2689370108 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.051+0000 7f81a259c700 1 -- 172.21.15.26:0/2689370108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a407a8a0 con 0x7f81a407bd30 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.051+0000 7f81a259c700 1 --2- 172.21.15.26:0/2689370108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81940098a0 tx=0x7f8194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2b2ce9f079797c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:52.812 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.051+0000 7f81a159a700 1 -- 172.21.15.26:0/2689370108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8194004160 con 0x7f81a407bd30 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.051+0000 7f81a159a700 1 -- 172.21.15.26:0/2689370108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81940042c0 con 0x7f81a407bd30 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.051+0000 7f81a159a700 1 -- 172.21.15.26:0/2689370108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81940119b0 con 0x7f81a407bd30 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 -- 172.21.15.26:0/2689370108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 msgr2=0x7f81a407a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/2689370108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81940098a0 tx=0x7f8194009bb0 comp rx=0 tx=0).stop 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 -- 172.21.15.26:0/2689370108 shutdown_connections 2023-10-06T21:01:52.813 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/2689370108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a407a190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:52.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 -- 172.21.15.26:0/2689370108 >> 172.21.15.26:0/2689370108 conn(0x7f81a4101ec0 msgr2=0x7f81a41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:52.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 -- 172.21.15.26:0/2689370108 shutdown_connections 2023-10-06T21:01:52.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.052+0000 7f81a8a9d700 1 -- 172.21.15.26:0/2689370108 wait complete. 2023-10-06T21:01:52.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.053+0000 7f81a8a9d700 1 Processor -- start 2023-10-06T21:01:52.814 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.053+0000 7f81a8a9d700 1 -- start start 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.053+0000 7f81a8a9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a8a9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a419cce0 con 0x7f81a407bd30 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53458/0 (socket says 172.21.15.26:53458) 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a259c700 1 -- 172.21.15.26:0/812944536 learned_addr learned my addr 172.21.15.26:0/812944536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:52.815 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a259c700 1 -- 172.21.15.26:0/812944536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194009550 con 0x7f81a407bd30 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81a259c700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8194003e60 tx=0x7f8194011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.054+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8194003a60 con 0x7f81a407bd30 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.055+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8194003bc0 con 0x7f81a407bd30 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.055+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a419cee0 con 0x7f81a407bd30 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.055+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81940196c0 con 0x7f81a407bd30 2023-10-06T21:01:52.816 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.055+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a419d380 con 0x7f81a407bd30 2023-10-06T21:01:52.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.056+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f819401ab60 con 0x7f81a407bd30 2023-10-06T21:01:52.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.056+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a410cd20 con 0x7f81a407bd30 2023-10-06T21:01:52.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.056+0000 7f81937fe700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 0x7f818c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:52.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.056+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f819404cce0 con 0x7f81a407bd30 2023-10-06T21:01:52.817 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.056+0000 7f81a1d9b700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 0x7f818c03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.057+0000 7f81a1d9b700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 0x7f818c03ae00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8198006fd0 tx=0x7f8198006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.061+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8194010790 con 0x7f81a407bd30 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.383+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f81a410cf60 con 0x7f81a407bd30 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.386+0000 7f81937fe700 1 -- 172.21.15.26:0/812944536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8194018350 con 0x7f81a407bd30 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 msgr2=0x7f818c03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:52.818 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 0x7f818c03ae00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8198006fd0 tx=0x7f8198006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 msgr2=0x7f81a419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8194003e60 tx=0x7f8194011a30 comp rx=0 tx=0).stop 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 shutdown_connections 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f818c038940 0x7f818c03ae00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 --2- 172.21.15.26:0/812944536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81a407bd30 0x7f81a419c7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:52.819 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.388+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 >> 172.21.15.26:0/812944536 conn(0x7f81a4101ec0 msgr2=0x7f81a4102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.389+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 shutdown_connections 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-10-06T21:01:52.389+0000 7f81a8a9d700 1 -- 172.21.15.26:0/812944536 wait complete. 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.821 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:01:52.822 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2023-10-06T21:01:52.867 INFO:tasks.cephadm:Fetching config... 2023-10-06T21:01:52.867 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:01:52.867 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-06T21:01:52.917 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-06T21:01:52.917 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:01:52.917 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-06T21:01:53.003 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-06T21:01:53.003 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:01:53.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-06T21:01:53.115 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-06T21:01:53.116 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:01:53.116 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-06T21:01:53.202 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-06T21:01:53.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:01:53.343 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:53.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:01:53.423 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:53.436 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-06T21:01:53.483 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHy120CrUjzQOvMXh2cG1VI2axoXSC3elGk0eIVWU+IMRcotm1tfNwh8VcCi9Qcw25OSaBU0tuiSkJkxBmzWTtAqOxoooGh8hdIQCVuyfB8kwKWW+gcCDFCST5vPRykhdayJ7E3HAJRJjZHtc14XjrpOZjLETAshHEKUuIeuULrbggpy8eu0vP/6NCDuJSXIX0rJQXhOkUsyJ1qsMC6apDKjzmm0PBcjJP4SFPBTaVYCWLyMeXDf1TMCcKdBVGXlBsZSbtLtTyqwtCJOkMR2jJGHvGzLCGmIVkzW/hNblkNMgq2w+MbH+/J0bnJzzcF46CD+uEmwVoBpLdkJlYvhkCTvVg9TIML7PXprUxzZViior8HoQVq9FySsYWhm4/ytv2aCjIcYIHXP5W5ZfhN7J0o2UdR/XrNBw3xc2EgdnaI696JbukUvajPTfRw13CDD0m8erFbVxIUasuOsJDeWvIvWRvzdXml8OWBMtn+Q149CSTo601rDUEBxK26Rrrx0U= ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:01:53.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-06T21:01:53.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:53 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/812944536' entity='client.admin' 2023-10-06T21:01:54.243 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:01:56.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 -- 172.21.15.26:0/477006950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 msgr2=0x7facd4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:56.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 --2- 172.21.15.26:0/477006950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd4073340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facd0009910 tx=0x7facd0009c20 comp rx=0 tx=0).stop 2023-10-06T21:01:56.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 -- 172.21.15.26:0/477006950 shutdown_connections 2023-10-06T21:01:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 --2- 172.21.15.26:0/477006950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd4073340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 -- 172.21.15.26:0/477006950 >> 172.21.15.26:0/477006950 conn(0x7facd40fbf80 msgr2=0x7facd40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 -- 172.21.15.26:0/477006950 shutdown_connections 2023-10-06T21:01:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.228+0000 7facdc51a700 1 -- 172.21.15.26:0/477006950 wait complete. 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.229+0000 7facdc51a700 1 Processor -- start 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facdc51a700 1 -- start start 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facdc51a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facdc51a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd419c630 con 0x7facd4074ee0 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facda2b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facda2b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53468/0 (socket says 172.21.15.26:53468) 2023-10-06T21:01:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.230+0000 7facda2b6700 1 -- 172.21.15.26:0/3410132928 learned_addr learned my addr 172.21.15.26:0/3410132928 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7facda2b6700 1 -- 172.21.15.26:0/3410132928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00095f0 con 0x7facd4074ee0 2023-10-06T21:01:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7facda2b6700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facd0006010 tx=0x7facd00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facd001d070 con 0x7facd4074ee0 2023-10-06T21:01:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd00044e0 con 0x7facd4074ee0 2023-10-06T21:01:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facd00039b0 con 0x7facd4074ee0 2023-10-06T21:01:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd419c830 con 0x7facd4074ee0 2023-10-06T21:01:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.231+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd419cc70 con 0x7facd4074ee0 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.232+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7facd0003c20 con 0x7facd4074ee0 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.232+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd4195b50 con 0x7facd4074ee0 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.233+0000 7faccb7fe700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 0x7facc003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.233+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facd004c980 con 0x7facd4074ee0 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.233+0000 7facd9ab5700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 0x7facc003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.234+0000 7facd9ab5700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 0x7facc003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7facc4006fd0 tx=0x7facc4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.238+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7facd002a540 con 0x7facd4074ee0 2023-10-06T21:01:56.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.453+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7facd402d160 con 0x7facd4074ee0 2023-10-06T21:01:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.456+0000 7faccb7fe700 1 -- 172.21.15.26:0/3410132928 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7facd002a360 con 0x7facd4074ee0 2023-10-06T21:01:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 msgr2=0x7facc003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 0x7facc003add0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7facc4006fd0 tx=0x7facc4006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 msgr2=0x7facd419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facd0006010 tx=0x7facd00057c0 comp rx=0 tx=0).stop 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 shutdown_connections 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7facc0038910 0x7facc003add0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 --2- 172.21.15.26:0/3410132928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facd4074ee0 0x7facd419c0f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 >> 172.21.15.26:0/3410132928 conn(0x7facd40fbf80 msgr2=0x7facd40fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.459+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 shutdown_connections 2023-10-06T21:01:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:56.460+0000 7facdc51a700 1 -- 172.21.15.26:0/3410132928 wait complete. 2023-10-06T21:01:56.983 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-06T21:01:56.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-06T21:01:57.493 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:57 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3410132928' entity='client.admin' 2023-10-06T21:01:57.911 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:01:58.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:01:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:01:59.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 -- 172.21.15.26:0/1732865049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 msgr2=0x7f253c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:59.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 --2- 172.21.15.26:0/1732865049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c100e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f252c009910 tx=0x7f252c009c20 comp rx=0 tx=0).stop 2023-10-06T21:01:59.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 -- 172.21.15.26:0/1732865049 shutdown_connections 2023-10-06T21:01:59.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 --2- 172.21.15.26:0/1732865049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c100e20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:59.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 -- 172.21.15.26:0/1732865049 >> 172.21.15.26:0/1732865049 conn(0x7f253c0fbf80 msgr2=0x7f253c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:59.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 -- 172.21.15.26:0/1732865049 shutdown_connections 2023-10-06T21:01:59.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.297+0000 7f2540c74700 1 -- 172.21.15.26:0/1732865049 wait complete. 2023-10-06T21:01:59.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.298+0000 7f2540c74700 1 Processor -- start 2023-10-06T21:01:59.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.298+0000 7f2540c74700 1 -- start start 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f2540c74700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f2540c74700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c196050 con 0x7f253c100a00 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f253a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f253a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53480/0 (socket says 172.21.15.26:53480) 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f253a59c700 1 -- 172.21.15.26:0/3013652011 learned_addr learned my addr 172.21.15.26:0/3013652011 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.299+0000 7f253a59c700 1 -- 172.21.15.26:0/3013652011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0095f0 con 0x7f253c100a00 2023-10-06T21:01:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f253a59c700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f252c004f40 tx=0x7f252c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f252c01d070 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252c00bc10 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f252c003a70 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c196250 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.300+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c1966f0 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.301+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f252c003ce0 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.301+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c18f510 con 0x7f253c100a00 2023-10-06T21:01:59.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.301+0000 7f252b7fe700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 0x7f252403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:01:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.302+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f252c04c6b0 con 0x7f253c100a00 2023-10-06T21:01:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.302+0000 7f2539d9b700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 0x7f252403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:01:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.302+0000 7f2539d9b700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 0x7f252403add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:01:59.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.307+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f252c02abf0 con 0x7f253c100a00 2023-10-06T21:01:59.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.524+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f253c02d120 con 0x7f2524038910 2023-10-06T21:01:59.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.528+0000 7f252b7fe700 1 -- 172.21.15.26:0/3013652011 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f253c02d120 con 0x7f2524038910 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 msgr2=0x7f252403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 0x7f252403add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).stop 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 msgr2=0x7f253c195b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f252c004f40 tx=0x7f252c0057c0 comp rx=0 tx=0).stop 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 shutdown_connections 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2524038910 0x7f252403add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 --2- 172.21.15.26:0/3013652011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c100a00 0x7f253c195b10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:01:59.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 >> 172.21.15.26:0/3013652011 conn(0x7f253c0fbf80 msgr2=0x7f253c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:01:59.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 shutdown_connections 2023-10-06T21:01:59.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:01:59.530+0000 7f2540c74700 1 -- 172.21.15.26:0/3013652011 wait complete. 2023-10-06T21:01:59.995 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2023-10-06T21:01:59.995 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T21:01:59.995 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2023-10-06T21:02:00.015 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T21:02:00.016 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:00.075 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2023-10-06T21:02:00.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi078 2023-10-06T21:02:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:00.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: Updating smithi026:/etc/ceph/ceph.conf 2023-10-06T21:02:00.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:00 smithi026 ceph-mon[98490]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:01.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:01 smithi026 ceph-mon[98490]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:01.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:01 smithi026 ceph-mon[98490]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:02:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.651+0000 7f097bf35700 1 -- 172.21.15.26:0/3197377132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 msgr2=0x7f09740736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.651+0000 7f097bf35700 1 --2- 172.21.15.26:0/3197377132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f09740736c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0970009910 tx=0x7f0970009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.652+0000 7f097bf35700 1 -- 172.21.15.26:0/3197377132 shutdown_connections 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.652+0000 7f097bf35700 1 --2- 172.21.15.26:0/3197377132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f09740736c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.652+0000 7f097bf35700 1 -- 172.21.15.26:0/3197377132 >> 172.21.15.26:0/3197377132 conn(0x7f09740fc4a0 msgr2=0x7f09740fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.652+0000 7f097bf35700 1 -- 172.21.15.26:0/3197377132 shutdown_connections 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.652+0000 7f097bf35700 1 -- 172.21.15.26:0/3197377132 wait complete. 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.653+0000 7f097bf35700 1 Processor -- start 2023-10-06T21:02:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.653+0000 7f097bf35700 1 -- start start 2023-10-06T21:02:02.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.653+0000 7f097bf35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:02.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.653+0000 7f097bf35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741985d0 con 0x7f09740732e0 2023-10-06T21:02:02.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f0979cd1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:02.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f0979cd1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38826/0 (socket says 172.21.15.26:38826) 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f0979cd1700 1 -- 172.21.15.26:0/4158875722 learned_addr learned my addr 172.21.15.26:0/4158875722 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f0979cd1700 1 -- 172.21.15.26:0/4158875722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700095f0 con 0x7f09740732e0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f0979cd1700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.654+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f097001d070 con 0x7f09740732e0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.655+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09700044a0 con 0x7f09740732e0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.655+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741987d0 con 0x7f09740732e0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.655+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0970003a70 con 0x7f09740732e0 2023-10-06T21:02:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.655+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0974198bf0 con 0x7f09740732e0 2023-10-06T21:02:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.656+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f0970003cb0 con 0x7f09740732e0 2023-10-06T21:02:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.656+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09740624e0 con 0x7f09740732e0 2023-10-06T21:02:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.656+0000 7f096affd700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 0x7f096003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.656+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f097004c800 con 0x7f09740732e0 2023-10-06T21:02:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.656+0000 7f09794d0700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 0x7f096003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.657+0000 7f09794d0700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 0x7f096003ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.661+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0970029540 con 0x7f09740732e0 2023-10-06T21:02:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:02.877+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}) v1 -- 0x7f097410d050 con 0x7f09600388c0 2023-10-06T21:02:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:03.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:03 smithi026 ceph-mon[98490]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:03 smithi026 ceph-mon[98490]: Deploying cephadm binary to smithi078 2023-10-06T21:02:05.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.336+0000 7f096affd700 1 -- 172.21.15.26:0/4158875722 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f097410d050 con 0x7f09600388c0 2023-10-06T21:02:05.339 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi078' with addr '172.21.15.78' 2023-10-06T21:02:05.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 msgr2=0x7f096003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 0x7f096003ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0964006fd0 tx=0x7f0964006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 msgr2=0x7f0974198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0970006010 tx=0x7f0970004ff0 comp rx=0 tx=0).stop 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 shutdown_connections 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f09600388c0 0x7f096003ad80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 --2- 172.21.15.26:0/4158875722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09740732e0 0x7f0974198090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 >> 172.21.15.26:0/4158875722 conn(0x7f09740fc4a0 msgr2=0x7f09741074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 shutdown_connections 2023-10-06T21:02:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:05.339+0000 7f097bf35700 1 -- 172.21.15.26:0/4158875722 wait complete. 2023-10-06T21:02:05.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:05 smithi026 ceph-mon[98490]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:05.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T21:02:06.559 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:06.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:06.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:06 smithi026 ceph-mon[98490]: Added host smithi078 2023-10-06T21:02:06.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:06.810 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:07.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:07 smithi026 ceph-mon[98490]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:07.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 -- 172.21.15.26:0/2993326187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 msgr2=0x7f5d78106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 --2- 172.21.15.26:0/2993326187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78106d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009910 tx=0x7f5d74009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 -- 172.21.15.26:0/2993326187 shutdown_connections 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 --2- 172.21.15.26:0/2993326187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78106d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 -- 172.21.15.26:0/2993326187 >> 172.21.15.26:0/2993326187 conn(0x7f5d780fc4a0 msgr2=0x7f5d780fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 -- 172.21.15.26:0/2993326187 shutdown_connections 2023-10-06T21:02:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.965+0000 7f5d800d6700 1 -- 172.21.15.26:0/2993326187 wait complete. 2023-10-06T21:02:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.966+0000 7f5d800d6700 1 Processor -- start 2023-10-06T21:02:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.966+0000 7f5d800d6700 1 -- start start 2023-10-06T21:02:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d800d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d800d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78198570 con 0x7f5d78106970 2023-10-06T21:02:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d7de72700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d7de72700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38850/0 (socket says 172.21.15.26:38850) 2023-10-06T21:02:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d7de72700 1 -- 172.21.15.26:0/921986355 learned_addr learned my addr 172.21.15.26:0/921986355 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d7de72700 1 -- 172.21.15.26:0/921986355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740095f0 con 0x7f5d78106970 2023-10-06T21:02:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.967+0000 7f5d7de72700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d74006010 tx=0x7f5d74004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.968+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7401d070 con 0x7f5d78106970 2023-10-06T21:02:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.968+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7400bbb0 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.968+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d74003930 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.968+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d781987d0 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.968+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d78198c50 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.969+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f5d7400bd20 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.969+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d780624e0 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.969+0000 7f5d6effd700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 0x7f5d6403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.969+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d7404c690 con 0x7f5d78106970 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.970+0000 7f5d7d671700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 0x7f5d6403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.970+0000 7f5d7d671700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 0x7f5d6403add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:07.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:07.975+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d74029770 con 0x7f5d78106970 2023-10-06T21:02:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.194+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d7810adf0 con 0x7f5d64038910 2023-10-06T21:02:08.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.195+0000 7f5d6effd700 1 -- 172.21.15.26:0/921986355 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f5d7810adf0 con 0x7f5d64038910 2023-10-06T21:02:08.198 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:02:08.198 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2023-10-06T21:02:08.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 msgr2=0x7f5d6403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 0x7f5d6403add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 msgr2=0x7f5d78198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d74006010 tx=0x7f5d74004ff0 comp rx=0 tx=0).stop 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 shutdown_connections 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5d64038910 0x7f5d6403add0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 --2- 172.21.15.26:0/921986355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d78106970 0x7f5d78198030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 >> 172.21.15.26:0/921986355 conn(0x7f5d780fc4a0 msgr2=0x7f5d78108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 shutdown_connections 2023-10-06T21:02:08.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:08.198+0000 7f5d800d6700 1 -- 172.21.15.26:0/921986355 wait complete. 2023-10-06T21:02:08.739 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2023-10-06T21:02:08.739 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T21:02:08.739 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2023-10-06T21:02:08.760 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T21:02:08.760 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:08.819 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2023-10-06T21:02:08.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi167 2023-10-06T21:02:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:08 smithi026 ceph-mon[98490]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:02:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:08 smithi026 ceph-mon[98490]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.058+0000 7fdb45eea700 1 -- 172.21.15.26:0/1984116019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 msgr2=0x7fdb40106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.058+0000 7fdb45eea700 1 --2- 172.21.15.26:0/1984116019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40106d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009910 tx=0x7fdb30009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.059+0000 7fdb45eea700 1 -- 172.21.15.26:0/1984116019 shutdown_connections 2023-10-06T21:02:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.059+0000 7fdb45eea700 1 --2- 172.21.15.26:0/1984116019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40106d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.059+0000 7fdb45eea700 1 -- 172.21.15.26:0/1984116019 >> 172.21.15.26:0/1984116019 conn(0x7fdb400fc4a0 msgr2=0x7fdb400fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.059+0000 7fdb45eea700 1 -- 172.21.15.26:0/1984116019 shutdown_connections 2023-10-06T21:02:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.059+0000 7fdb45eea700 1 -- 172.21.15.26:0/1984116019 wait complete. 2023-10-06T21:02:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.060+0000 7fdb45eea700 1 Processor -- start 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.060+0000 7fdb45eea700 1 -- start start 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.060+0000 7fdb45eea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.060+0000 7fdb45eea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401985a0 con 0x7fdb40106970 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.061+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.061+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39252/0 (socket says 172.21.15.26:39252) 2023-10-06T21:02:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.061+0000 7fdb3f7fe700 1 -- 172.21.15.26:0/2690299760 learned_addr learned my addr 172.21.15.26:0/2690299760 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.061+0000 7fdb3f7fe700 1 -- 172.21.15.26:0/2690299760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300095f0 con 0x7fdb40106970 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.061+0000 7fdb3f7fe700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb30006010 tx=0x7fdb30004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.062+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb3001d070 con 0x7fdb40106970 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.062+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb40198800 con 0x7fdb40106970 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.062+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3000bbb0 con 0x7fdb40106970 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.062+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb30003930 con 0x7fdb40106970 2023-10-06T21:02:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.062+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb40198c80 con 0x7fdb40106970 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.063+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb400624e0 con 0x7fdb40106970 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.063+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fdb3000bd20 con 0x7fdb40106970 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.063+0000 7fdb3cff9700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 0x7fdb2803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.063+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb3004c740 con 0x7fdb40106970 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.064+0000 7fdb3effd700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 0x7fdb2803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.064+0000 7fdb3effd700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 0x7fdb2803ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.068+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb30029770 con 0x7fdb40106970 2023-10-06T21:02:11.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:11.286+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb4010adf0 con 0x7fdb280388c0 2023-10-06T21:02:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:11.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:11.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:02:11.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:11.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: Updating smithi078:/etc/ceph/ceph.conf 2023-10-06T21:02:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:12.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:12.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:12.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:12.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:13.801 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi167' with addr '172.21.15.167' 2023-10-06T21:02:13.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.797+0000 7fdb3cff9700 1 -- 172.21.15.26:0/2690299760 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdb4010adf0 con 0x7fdb280388c0 2023-10-06T21:02:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 msgr2=0x7fdb2803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 0x7fdb2803ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 msgr2=0x7fdb40198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb30006010 tx=0x7fdb30004ff0 comp rx=0 tx=0).stop 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 shutdown_connections 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdb280388c0 0x7fdb2803ad80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 --2- 172.21.15.26:0/2690299760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb40106970 0x7fdb40198060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 >> 172.21.15.26:0/2690299760 conn(0x7fdb400fc4a0 msgr2=0x7fdb40108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 shutdown_connections 2023-10-06T21:02:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:13.800+0000 7fdb45eea700 1 -- 172.21.15.26:0/2690299760 wait complete. 2023-10-06T21:02:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:13 smithi026 ceph-mon[98490]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:02:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:13 smithi026 ceph-mon[98490]: Deploying cephadm binary to smithi167 2023-10-06T21:02:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:13 smithi026 ceph-mon[98490]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:14.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-06T21:02:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:14 smithi026 ceph-mon[98490]: Added host smithi167 2023-10-06T21:02:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:14 smithi026 ceph-mon[98490]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:16.250 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 -- 172.21.15.26:0/1868846608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 msgr2=0x7faa4c106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 --2- 172.21.15.26:0/1868846608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c106d50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009910 tx=0x7faa3c009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 -- 172.21.15.26:0/1868846608 shutdown_connections 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 --2- 172.21.15.26:0/1868846608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c106d50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 -- 172.21.15.26:0/1868846608 >> 172.21.15.26:0/1868846608 conn(0x7faa4c0fc4a0 msgr2=0x7faa4c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 -- 172.21.15.26:0/1868846608 shutdown_connections 2023-10-06T21:02:16.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.564+0000 7faa5140e700 1 -- 172.21.15.26:0/1868846608 wait complete. 2023-10-06T21:02:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.565+0000 7faa5140e700 1 Processor -- start 2023-10-06T21:02:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.565+0000 7faa5140e700 1 -- start start 2023-10-06T21:02:16.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa5140e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa5140e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c198570 con 0x7faa4c106970 2023-10-06T21:02:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39262/0 (socket says 172.21.15.26:39262) 2023-10-06T21:02:16.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa4affd700 1 -- 172.21.15.26:0/1701795861 learned_addr learned my addr 172.21.15.26:0/1701795861 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.566+0000 7faa4affd700 1 -- 172.21.15.26:0/1701795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0095f0 con 0x7faa4c106970 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa4affd700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006010 tx=0x7faa3c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa3c01d070 con 0x7faa4c106970 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa3c00bbb0 con 0x7faa4c106970 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c1987d0 con 0x7faa4c106970 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa3c003930 con 0x7faa4c106970 2023-10-06T21:02:16.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.567+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c198c50 con 0x7faa4c106970 2023-10-06T21:02:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.568+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7faa3c00bd20 con 0x7faa4c106970 2023-10-06T21:02:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.568+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c0624e0 con 0x7faa4c106970 2023-10-06T21:02:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.569+0000 7faa33fff700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 0x7faa3403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.569+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa3c04c760 con 0x7faa4c106970 2023-10-06T21:02:16.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.569+0000 7faa4a7fc700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 0x7faa3403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:16.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.569+0000 7faa4a7fc700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 0x7faa3403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa40006fd0 tx=0x7faa40006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.574+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faa3c029360 con 0x7faa4c106970 2023-10-06T21:02:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.790+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa4c0008d0 con 0x7faa34038910 2023-10-06T21:02:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.791+0000 7faa33fff700 1 -- 172.21.15.26:0/1701795861 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7faa4c0008d0 con 0x7faa34038910 2023-10-06T21:02:16.794 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:02:16.794 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 msgr2=0x7faa3403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 0x7faa3403add0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faa40006fd0 tx=0x7faa40006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 msgr2=0x7faa4c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006010 tx=0x7faa3c004ff0 comp rx=0 tx=0).stop 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 shutdown_connections 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faa34038910 0x7faa3403add0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 --2- 172.21.15.26:0/1701795861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa4c106970 0x7faa4c198030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 >> 172.21.15.26:0/1701795861 conn(0x7faa4c0fc4a0 msgr2=0x7faa4c108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:16.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 shutdown_connections 2023-10-06T21:02:16.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:16.794+0000 7faa5140e700 1 -- 172.21.15.26:0/1701795861 wait complete. 2023-10-06T21:02:17.255 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-06T21:02:17.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-06T21:02:17.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:17.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:17 smithi026 ceph-mon[98490]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:18.545 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:18 smithi026 ceph-mon[98490]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:02:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 -- 172.21.15.26:0/835082164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 msgr2=0x7f794c106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 --2- 172.21.15.26:0/835082164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c106d50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7948009910 tx=0x7f7948009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 -- 172.21.15.26:0/835082164 shutdown_connections 2023-10-06T21:02:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 --2- 172.21.15.26:0/835082164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c106d50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 -- 172.21.15.26:0/835082164 >> 172.21.15.26:0/835082164 conn(0x7f794c0fc4a0 msgr2=0x7f794c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 -- 172.21.15.26:0/835082164 shutdown_connections 2023-10-06T21:02:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.515+0000 7f7953571700 1 -- 172.21.15.26:0/835082164 wait complete. 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.516+0000 7f7953571700 1 Processor -- start 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.516+0000 7f7953571700 1 -- start start 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f7953571700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f7953571700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1985a0 con 0x7f794c106970 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f795130d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f795130d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39282/0 (socket says 172.21.15.26:39282) 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f795130d700 1 -- 172.21.15.26:0/3839910533 learned_addr learned my addr 172.21.15.26:0/3839910533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f795130d700 1 -- 172.21.15.26:0/3839910533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480095f0 con 0x7f794c106970 2023-10-06T21:02:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.517+0000 7f795130d700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7948006010 tx=0x7f7948004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.518+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f794801d070 con 0x7f794c106970 2023-10-06T21:02:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.518+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794800bbb0 con 0x7f794c106970 2023-10-06T21:02:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.518+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c198800 con 0x7f794c106970 2023-10-06T21:02:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.518+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7948003930 con 0x7f794c106970 2023-10-06T21:02:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.518+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c198c80 con 0x7f794c106970 2023-10-06T21:02:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.519+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f794800bd20 con 0x7f794c106970 2023-10-06T21:02:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.519+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c0624e0 con 0x7f794c106970 2023-10-06T21:02:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.519+0000 7f79427fc700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 0x7f793803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.519+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f794804c6a0 con 0x7f794c106970 2023-10-06T21:02:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.520+0000 7f7950b0c700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 0x7f793803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.520+0000 7f7950b0c700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 0x7f793803ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f793c006fd0 tx=0x7f793c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.525+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7948029360 con 0x7f794c106970 2023-10-06T21:02:19.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:02:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:19.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:19.740+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f794c0008d0 con 0x7f794c106970 2023-10-06T21:02:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.232+0000 7f79427fc700 1 -- 172.21.15.26:0/3839910533 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7948026020 con 0x7f794c106970 2023-10-06T21:02:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 msgr2=0x7f793803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 0x7f793803ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f793c006fd0 tx=0x7f793c006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 msgr2=0x7f794c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7948006010 tx=0x7f7948004ff0 comp rx=0 tx=0).stop 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 shutdown_connections 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f79380388c0 0x7f793803ad80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 --2- 172.21.15.26:0/3839910533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f794c106970 0x7f794c198060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.234+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 >> 172.21.15.26:0/3839910533 conn(0x7f794c0fc4a0 msgr2=0x7f794c108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.235+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 shutdown_connections 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:20.235+0000 7f7953571700 1 -- 172.21.15.26:0/3839910533 wait complete. 2023-10-06T21:02:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2023-10-06T21:02:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: Updating smithi167:/etc/ceph/ceph.conf 2023-10-06T21:02:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3839910533' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T21:02:20.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:20.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:20.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:20.679 INFO:tasks.cephadm:Adding mon.a on smithi026 2023-10-06T21:02:20.679 INFO:tasks.cephadm:Adding mon.b on smithi078 2023-10-06T21:02:20.679 INFO:tasks.cephadm:Adding mon.c on smithi167 2023-10-06T21:02:20.680 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c' 2023-10-06T21:02:21.255 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:21.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:21 smithi026 ceph-mon[98490]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:21.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:21 smithi026 ceph-mon[98490]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:02:21.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:21 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3839910533' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T21:02:21.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:21 smithi026 ceph-mon[98490]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:21.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:21 smithi026 ceph-mon[98490]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 -- 172.21.15.167:0/4006004001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 msgr2=0x7f3218105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 --2- 172.21.15.167:0/4006004001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218105c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3208009910 tx=0x7f3208009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 -- 172.21.15.167:0/4006004001 shutdown_connections 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 --2- 172.21.15.167:0/4006004001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218105c30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 -- 172.21.15.167:0/4006004001 >> 172.21.15.167:0/4006004001 conn(0x7f32180fb360 msgr2=0x7f32180fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 -- 172.21.15.167:0/4006004001 shutdown_connections 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.364+0000 7f321fee6700 1 -- 172.21.15.167:0/4006004001 wait complete. 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.365+0000 7f321fee6700 1 Processor -- start 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.365+0000 7f321fee6700 1 -- start start 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.365+0000 7f321fee6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.365+0000 7f321fee6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218195260 con 0x7f3218105850 2023-10-06T21:02:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321dc82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321dc82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.167:44894/0 (socket says 172.21.15.167:44894) 2023-10-06T21:02:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321dc82700 1 -- 172.21.15.167:0/323391684 learned_addr learned my addr 172.21.15.167:0/323391684 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:02:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321dc82700 1 -- 172.21.15.167:0/323391684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080095f0 con 0x7f3218105850 2023-10-06T21:02:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321dc82700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3208004f40 tx=0x7f3208005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f320801d070 con 0x7f3218105850 2023-10-06T21:02:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32181954c0 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3208004370 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.366+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3208003970 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.367+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3218195940 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.367+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32180624e0 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.367+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f3208003c30 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.368+0000 7f320effd700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 0x7f320403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.368+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f320804c670 con 0x7f3218105850 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.368+0000 7f321d481700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 0x7f320403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:22.372 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.369+0000 7f321d481700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 0x7f320403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:22.376 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.373+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f320802a590 con 0x7f3218105850 2023-10-06T21:02:22.574 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.571+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f3218109da0 con 0x7f32040388c0 2023-10-06T21:02:22.581 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.579+0000 7f320effd700 1 -- 172.21.15.167:0/323391684 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3218109da0 con 0x7f32040388c0 2023-10-06T21:02:22.582 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 msgr2=0x7f320403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 0x7f320403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 msgr2=0x7f3218194d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3208004f40 tx=0x7f3208005020 comp rx=0 tx=0).stop 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 shutdown_connections 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f32040388c0 0x7f320403ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 --2- 172.21.15.167:0/323391684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3218105850 0x7f3218194d20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 >> 172.21.15.167:0/323391684 conn(0x7f32180fb360 msgr2=0x7f3218107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 shutdown_connections 2023-10-06T21:02:22.584 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:22.581+0000 7f321fee6700 1 -- 172.21.15.167:0/323391684 wait complete. 2023-10-06T21:02:23.030 DEBUG:teuthology.orchestra.run.smithi078:mon.b> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-06T21:02:23.033 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-06T21:02:23.035 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:02:23.035 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:02:23.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:02:23.068 INFO:journalctl@ceph.mon.b.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:02:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:23.879 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:24 smithi026 ceph-mon[98490]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:24 smithi026 ceph-mon[98490]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c;count:3 2023-10-06T21:02:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:24 smithi026 ceph-mon[98490]: Deploying daemon mon.c on smithi167 2023-10-06T21:02:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:25 smithi026 ceph-mon[98490]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 -- 172.21.15.167:0/1692981167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 msgr2=0x7ff9bc106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1692981167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc106d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009910 tx=0x7ff9ac009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 -- 172.21.15.167:0/1692981167 shutdown_connections 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1692981167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc106d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 -- 172.21.15.167:0/1692981167 >> 172.21.15.167:0/1692981167 conn(0x7ff9bc075370 msgr2=0x7ff9bc075780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 -- 172.21.15.167:0/1692981167 shutdown_connections 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.771+0000 7ff9c299a700 1 -- 172.21.15.167:0/1692981167 wait complete. 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9c299a700 1 Processor -- start 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9c299a700 1 -- start start 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9c299a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9c299a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc196400 con 0x7ff9bc106970 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9bbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9bbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.167:44900/0 (socket says 172.21.15.167:44900) 2023-10-06T21:02:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.772+0000 7ff9bbfff700 1 -- 172.21.15.167:0/1920585656 learned_addr learned my addr 172.21.15.167:0/1920585656 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:02:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9bbfff700 1 -- 172.21.15.167:0/1920585656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0095f0 con 0x7ff9bc106970 2023-10-06T21:02:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9bbfff700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac004f40 tx=0x7ff9ac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac01d070 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ac004370 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc196660 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ac003970 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.773+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc196ae0 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.774+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7ff9ac003c30 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.774+0000 7ff9b97fa700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 0x7ff9a403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.774+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9ac04d610 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.774+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc0624e0 con 0x7ff9bc106970 2023-10-06T21:02:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.775+0000 7ff9bb7fe700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 0x7ff9a403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:25.779 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.775+0000 7ff9bb7fe700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 0x7ff9a403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0006fd0 tx=0x7ff9b0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:25.782 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:25.780+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff9ac026070 con 0x7ff9bc106970 2023-10-06T21:02:26.063 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.059+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9bc197220 con 0x7ff9bc106970 2023-10-06T21:02:26.063 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.060+0000 7ff9b97fa700 1 -- 172.21.15.167:0/1920585656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7ff9ac02b440 con 0x7ff9bc106970 2023-10-06T21:02:26.064 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T21:02:26.064 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:00:42.223924Z","created":"2023-10-06T21:00:42.223924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-06T21:02:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 msgr2=0x7ff9a403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 0x7ff9a403ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0006fd0 tx=0x7ff9b0006e40 comp rx=0 tx=0).stop 2023-10-06T21:02:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 msgr2=0x7ff9bc195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac004f40 tx=0x7ff9ac005020 comp rx=0 tx=0).stop 2023-10-06T21:02:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 shutdown_connections 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9a40388c0 0x7ff9a403ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 --2- 172.21.15.167:0/1920585656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9bc106970 0x7ff9bc195ec0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 >> 172.21.15.167:0/1920585656 conn(0x7ff9bc075370 msgr2=0x7ff9bc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 shutdown_connections 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:26.062+0000 7ff9c299a700 1 -- 172.21.15.167:0/1920585656 wait complete. 2023-10-06T21:02:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2023-10-06T21:02:26.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:26 smithi167 ceph-mon[134494]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-10-06T21:02:26.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:26 smithi167 systemd[1]: Started Ceph mon.c for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:02:27.619 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:02:27.620 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:02:28.192 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:02:31.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: mon.a calling monitor election 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: mon.c calling monitor election 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: fsmap 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: mgrmap e13: x(active, since 46s) 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: overall HEALTH_OK 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:31.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:31.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:31.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: mon.a calling monitor election 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: mon.c calling monitor election 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:31.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: fsmap 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: mgrmap e13: x(active, since 46s) 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: overall HEALTH_OK 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 -- 172.21.15.167:0/2363080523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 msgr2=0x7efeec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 --2- 172.21.15.167:0/2363080523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7efeec005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff040ff8d0 tx=0x7efef800feb0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 -- 172.21.15.167:0/2363080523 shutdown_connections 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 --2- 172.21.15.167:0/2363080523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7efeec005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 -- 172.21.15.167:0/2363080523 >> 172.21.15.167:0/2363080523 conn(0x7eff040fb4d0 msgr2=0x7eff040fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 -- 172.21.15.167:0/2363080523 shutdown_connections 2023-10-06T21:02:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.396+0000 7eff0a97b700 1 -- 172.21.15.167:0/2363080523 wait complete. 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 Processor -- start 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 -- start start 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7eff041981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04198ca0 con 0x7efeec0035c0 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff0a97b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04198e10 con 0x7eff04198710 2023-10-06T21:02:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.397+0000 7eff037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58970/0 (socket says 172.21.15.167:58970) 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 -- 172.21.15.167:0/1379895872 learned_addr learned my addr 172.21.15.167:0/1379895872 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff03fff700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7eff041981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 -- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 msgr2=0x7eff0419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 -- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 msgr2=0x7eff0419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff037fe700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff03fff700 1 -- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 msgr2=0x7eff0419cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff03fff700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff03fff700 1 -- 172.21.15.167:0/1379895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef800f900 con 0x7efeec0035c0 2023-10-06T21:02:32.403 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.398+0000 7eff03fff700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7eff041981d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efef8010010 tx=0x7efef80047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.399+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7efef8013960 con 0x7efeec0035c0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.399+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efef8004030 con 0x7efeec0035c0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.399+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0419d120 con 0x7efeec0035c0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.399+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7efef8003a80 con 0x7efeec0035c0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.399+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0419d670 con 0x7efeec0035c0 2023-10-06T21:02:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.400+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7efef8003d40 con 0x7efeec0035c0 2023-10-06T21:02:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.400+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff040624e0 con 0x7efeec0035c0 2023-10-06T21:02:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.400+0000 7eff017fa700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 0x7efef003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.400+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efef8000e40 con 0x7efeec0035c0 2023-10-06T21:02:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.400+0000 7eff037fe700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 0x7efef003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.401+0000 7eff037fe700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 0x7efef003abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efef4009180 tx=0x7efef4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:32.409 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.406+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efef801a750 con 0x7efeec0035c0 2023-10-06T21:02:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:32 smithi167 ceph-mon[134494]: Deploying daemon mon.b on smithi078 2023-10-06T21:02:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:32 smithi026 ceph-mon[98490]: Deploying daemon mon.b on smithi078 2023-10-06T21:02:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:32.676 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.673+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff04107710 con 0x7efeec0035c0 2023-10-06T21:02:32.677 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.674+0000 7eff017fa700 1 -- 172.21.15.167:0/1379895872 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7efef801a390 con 0x7efeec0035c0 2023-10-06T21:02:32.677 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T21:02:32.677 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:02:26.162337Z","created":"2023-10-06T21:00:42.223924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T21:02:32.679 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.676+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 msgr2=0x7efef003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:32.679 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.676+0000 7eff0a97b700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 0x7efef003abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efef4009180 tx=0x7efef4008040 comp rx=0 tx=0).stop 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.676+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 msgr2=0x7eff041981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.676+0000 7eff0a97b700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7eff041981d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efef8010010 tx=0x7efef80047a0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 shutdown_connections 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efef0038700 0x7efef003abc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeec0035c0 0x7eff041981d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 --2- 172.21.15.167:0/1379895872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff04198710 0x7eff0419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 >> 172.21.15.167:0/1379895872 conn(0x7eff040fb4d0 msgr2=0x7eff041034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 shutdown_connections 2023-10-06T21:02:32.680 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:32.677+0000 7eff0a97b700 1 -- 172.21.15.167:0/1379895872 wait complete. 2023-10-06T21:02:32.681 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2023-10-06T21:02:33.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:33 smithi167 ceph-mon[134494]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:33.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:33 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/1379895872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:33.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:33 smithi026 ceph-mon[98490]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:33.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:33 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/1379895872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:33.574 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:02:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:02:33.161+0000 7fc5dc958700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T21:02:34.045 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-06T21:02:34.045 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-06T21:02:34.306 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 systemd[1]: Starting Ceph mon.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:02:34.594 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 podman[134857]: 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: set uid:gid to 167:167 (ceph:ceph) 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev), process ceph-mon, pid 7 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 bash[134857]: 67126393c925aa6c7b6132a0f7591ca7a5a82d0fd19cfcf4f56f7a7f45755ecd 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pidfile_write: ignore empty --pid-file 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: load: jerasure load: lrc 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: RocksDB version: 7.9.2 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Git sha 0 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Compile date 2023-10-06 09:28:03 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: DB SUMMARY 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: DB Session ID: W8XHTQBS7B1N7V20SHCX 2023-10-06T21:02:34.595 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: CURRENT file: CURRENT 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: IDENTITY file: IDENTITY 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.error_if_exists: 0 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.create_if_missing: 0 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.paranoid_checks: 1 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-06T21:02:34.596 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.env: 0x56161aba8da0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.fs: PosixFileSystem 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.info_log: 0x56161be095c0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_file_opening_threads: 16 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.statistics: (nil) 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.use_fsync: 0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_log_file_size: 0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.keep_log_file_num: 1000 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.recycle_log_file_num: 0 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_fallocate: 1 2023-10-06T21:02:34.597 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_mmap_reads: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_mmap_writes: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.use_direct_reads: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.create_missing_column_families: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.db_log_dir: 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.wal_dir: 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-06T21:02:34.598 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.advise_random_on_open: 1 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.db_write_buffer_size: 0 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.write_buffer_manager: 0x56161bdc1360 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.rate_limiter: (nil) 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.wal_recovery_mode: 2 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enable_thread_tracking: 0 2023-10-06T21:02:34.599 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enable_pipelined_write: 0 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.unordered_write: 0 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.row_cache: None 2023-10-06T21:02:34.600 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.wal_filter: None 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_ingest_behind: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.two_write_queues: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.manual_wal_flush: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.wal_compression: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.atomic_flush: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-06T21:02:34.601 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.log_readahead_size: 0 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.best_efforts_recovery: 0 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.allow_data_in_errors: 0 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.db_host_id: __hostname__ 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_background_jobs: 2 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_background_compactions: -1 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_subcompactions: 1 2023-10-06T21:02:34.602 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_total_wal_size: 0 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_open_files: -1 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bytes_per_sync: 0 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-06T21:02:34.603 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_readahead_size: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_background_flushes: -1 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Compression algorithms supported: 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kZSTD supported: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kXpressCompression supported: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kLZ4HCCompression supported: 1 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kZlibCompression supported: 1 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kSnappyCompression supported: 1 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kLZ4Compression supported: 1 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: kBZip2Compression supported: 0 2023-10-06T21:02:34.604 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.merge_operator: 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_filter: None 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_filter_factory: None 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.sst_partitioner_factory: None 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-06T21:02:34.605 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56161be096c0) 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: cache_index_and_filter_blocks: 1 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: pin_top_level_index_and_filter: 1 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_type: 0 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: data_block_index_type: 0 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_shortening: 1 2023-10-06T21:02:34.607 INFO:journalctl@ceph.mon.b.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: checksum: 4 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: no_block_cache: 0 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache: 0x56161be5f1f0 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_name: BinnedLRUCache 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_options: 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: capacity : 536870912 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: num_shard_bits : 4 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: strict_capacity_limit : 0 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: high_pri_pool_ratio: 0.000 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_compressed: (nil) 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: persistent_cache: (nil) 2023-10-06T21:02:34.608 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_size: 4096 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_size_deviation: 10 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_restart_interval: 16 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_block_restart_interval: 1 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: metadata_block_size: 4096 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: partition_filters: 0 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: use_delta_encoding: 1 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: filter_policy: bloomfilter 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: whole_key_filtering: 1 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: verify_compression: 0 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: read_amp_bytes_per_bit: 0 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: format_version: 5 2023-10-06T21:02:34.609 INFO:journalctl@ceph.mon.b.smithi078.stdout: enable_index_compression: 1 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_align: 0 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout: max_auto_readahead_size: 262144 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout: prepopulate_block_cache: 0 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout: initial_auto_readahead_size: 8192 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.write_buffer_size: 33554432 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_write_buffer_number: 2 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression: NoCompression 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression: Disabled 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.prefix_extractor: nullptr 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-06T21:02:34.610 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.num_levels: 7 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-06T21:02:34.611 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.level: 32767 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.strategy: 0 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.enabled: false 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-06T21:02:34.612 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.target_file_size_base: 67108864 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-06T21:02:34.613 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.arena_block_size: 1048576 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.disable_auto_compactions: 0 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-06T21:02:34.614 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.table_properties_collectors: 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.inplace_update_support: 0 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-06T21:02:34.615 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.bloom_locality: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.max_successive_merges: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.paranoid_file_checks: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.force_consistency_checks: 1 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.report_bg_io_stats: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.ttl: 2592000 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-06T21:02:34.616 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enable_blob_files: false 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.min_blob_size: 0 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_file_size: 268435456 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.blob_file_starting_level: 0 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-06T21:02:34.617 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: af364901-0ef5-4f69-816d-02131b5178ab 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626154509383, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626154509783, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696626154, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "af364901-0ef5-4f69-816d-02131b5178ab", "db_session_id": "W8XHTQBS7B1N7V20SHCX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696626154509849, "job": 1, "event": "recovery_finished"} 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56161befc000 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: DB pointer 0x56161bee8000 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-06T21:02:34.618 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** DB Stats ** 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** Compaction Stats [default] ** 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-06T21:02:34.619 INFO:journalctl@ceph.mon.b.smithi078.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** Compaction Stats [default] ** 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-06T21:02:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: Block cache BinnedLRUCache@0x56161be5f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-06T21:02:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: using public_addr v2:172.21.15.78:0/0 -> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: starting mon.b rank -1 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(???) e0 preinit fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).mds e1 new map 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: e1 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: legacy client fscid: -1 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2023-10-06T21:02:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout: No filesystems configured 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi078:/etc/ceph/ceph.conf 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Deploying cephadm binary to smithi167 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Added host smithi167 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:34.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi167:/etc/ceph/ceph.conf 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3839910533' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3839910533' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c;count:3 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Deploying daemon mon.c on smithi167 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.a calling monitor election 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.c calling monitor election 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.629 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: fsmap 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mgrmap e13: x(active, since 46s) 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: overall HEALTH_OK 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:34.630 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: Deploying daemon mon.b on smithi078 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/1379895872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expand map: {default=false} 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta from 'false' to 'false' 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expanded map: {default=false} 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expand map: {default=info} 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta from 'info' to 'info' 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expanded map: {default=info} 2023-10-06T21:02:34.631 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expand map: {default=daemon} 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expanded map: {default=daemon} 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expand map: {default=debug} 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta from 'debug' to 'debug' 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 ceph-mon[134896]: expand_channel_meta expanded map: {default=debug} 2023-10-06T21:02:34.632 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:34 smithi078 systemd[1]: Started Ceph mon.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:02:34.632 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: mon.c calling monitor election 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: mon.a calling monitor election 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: fsmap 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: mgrmap e13: x(active, since 55s) 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T21:02:40.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2023-10-06T21:02:40.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:40.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: mon.c calling monitor election 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: mon.a calling monitor election 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: fsmap 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: mgrmap e13: x(active, since 55s) 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:40.199 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- 172.21.15.167:0/996929238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 msgr2=0x7fd1d4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:40.200 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/996929238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1d4005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8005fa0 tx=0x7fd1d800ff70 comp rx=0 tx=0).stop 2023-10-06T21:02:40.200 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- 172.21.15.167:0/996929238 shutdown_connections 2023-10-06T21:02:40.200 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/996929238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1d4005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.200 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- 172.21.15.167:0/996929238 >> 172.21.15.167:0/996929238 conn(0x7fd1e80fb4a0 msgr2=0x7fd1e80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:40.200 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- 172.21.15.167:0/996929238 shutdown_connections 2023-10-06T21:02:40.201 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- 172.21.15.167:0/996929238 wait complete. 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 Processor -- start 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.198+0000 7fd1efb4f700 1 -- start start 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1e8194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81a4c70 con 0x7fd1e8105990 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81a4de0 con 0x7fd1e8195210 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1efb4f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81a4fa0 con 0x7fd1e81a2340 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58996/0 (socket says 172.21.15.167:58996) 2023-10-06T21:02:40.203 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 -- 172.21.15.167:0/3330762240 learned_addr learned my addr 172.21.15.167:0/3330762240 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed0ea700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1e8194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 msgr2=0x7fd1e81a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 msgr2=0x7fd1e81a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ee0ec700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed0ea700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 msgr2=0x7fd1e8195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed0ea700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 msgr2=0x7fd1e8195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed0ea700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed0ea700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:02:40.204 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 msgr2=0x7fd1e81a4730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 msgr2=0x7fd1e8195690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.199+0000 7fd1ed8eb700 1 -- 172.21.15.167:0/3330762240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d800f970 con 0x7fd1e8105990 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1ed8eb700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1e8194cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003cd0 con 0x7fd1e8105990 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d80042f0 con 0x7fd1e8105990 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e81a5220 con 0x7fd1e8105990 2023-10-06T21:02:40.205 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8014070 con 0x7fd1e8105990 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.200+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e81a5720 con 0x7fd1e8105990 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.201+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fd1d8024050 con 0x7fd1e8105990 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.201+0000 7fd1deffd700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 0x7fd1cc03b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.201+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80624e0 con 0x7fd1e8105990 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.201+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1d804e720 con 0x7fd1e8105990 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.202+0000 7fd1ed0ea700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 0x7fd1cc03b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:40.206 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.202+0000 7fd1ed0ea700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 0x7fd1cc03b200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009850 tx=0x7fd1e4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:40.209 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.207+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd1d80107e0 con 0x7fd1e8105990 2023-10-06T21:02:40.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.494+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd1e81076a0 con 0x7fd1e8105990 2023-10-06T21:02:40.496 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.495+0000 7fd1deffd700 1 -- 172.21.15.167:0/3330762240 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7fd1d802b070 con 0x7fd1e8105990 2023-10-06T21:02:40.497 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T21:02:40.497 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","modified":"2023-10-06T21:02:34.537823Z","created":"2023-10-06T21:00:42.223924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-06T21:02:40.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 msgr2=0x7fd1cc03b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:40.499 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 0x7fd1cc03b200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009850 tx=0x7fd1e4008040 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 msgr2=0x7fd1e8194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1e8194cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000c00 tx=0x7fd1d8005e70 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 shutdown_connections 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1e81a2340 0x7fd1e81a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd1cc038d40 0x7fd1cc03b200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1e8105990 0x7fd1e8194cd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 --2- 172.21.15.167:0/3330762240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1e8195210 0x7fd1e8195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 >> 172.21.15.167:0/3330762240 conn(0x7fd1e80fb4a0 msgr2=0x7fd1e81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 shutdown_connections 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:40.497+0000 7fd1efb4f700 1 -- 172.21.15.167:0/3330762240 wait complete. 2023-10-06T21:02:40.500 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2023-10-06T21:02:40.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:40 smithi167 ceph-mon[134494]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:40 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/3330762240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:40.863 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-06T21:02:40.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-06T21:02:40.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:40.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:40 smithi026 ceph-mon[98490]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:40.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:40 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/3330762240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:40.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.625 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:41.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: mon.c calling monitor election 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: mon.a calling monitor election 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: fsmap 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: mgrmap e13: x(active, since 55s) 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2023-10-06T21:02:41.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/3330762240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-06T21:02:41.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:42.563 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mon.b calling monitor election 2023-10-06T21:02:42.563 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mon.b calling monitor election 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mon.c calling monitor election 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mon.a calling monitor election 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: fsmap 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: mgrmap e13: x(active, since 57s) 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: Cluster is now healthy 2023-10-06T21:02:42.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: overall HEALTH_OK 2023-10-06T21:02:42.565 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.565 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.565 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.565 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mon.b calling monitor election 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mon.b calling monitor election 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mon.c calling monitor election 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mon.a calling monitor election 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: fsmap 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: mgrmap e13: x(active, since 57s) 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T21:02:42.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: Cluster is now healthy 2023-10-06T21:02:42.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: overall HEALTH_OK 2023-10-06T21:02:42.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mon.b calling monitor election 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mon.b calling monitor election 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mon.c calling monitor election 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mon.a calling monitor election 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-06T21:02:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} removed_ranks: {} 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: fsmap 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: osdmap e4: 0 total, 0 up, 0 in 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: mgrmap e13: x(active, since 57s) 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: Cluster is now healthy 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: overall HEALTH_OK 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:42.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 -- 172.21.15.26:0/3396336869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 msgr2=0x7f44e80736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 --2- 172.21.15.26:0/3396336869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e80736c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009910 tx=0x7f44e4009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 -- 172.21.15.26:0/3396336869 shutdown_connections 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 --2- 172.21.15.26:0/3396336869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e80736c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 -- 172.21.15.26:0/3396336869 >> 172.21.15.26:0/3396336869 conn(0x7f44e80fc4a0 msgr2=0x7f44e80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 -- 172.21.15.26:0/3396336869 shutdown_connections 2023-10-06T21:02:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.162+0000 7f44f0645700 1 -- 172.21.15.26:0/3396336869 wait complete. 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.163+0000 7f44f0645700 1 Processor -- start 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.163+0000 7f44f0645700 1 -- start start 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.163+0000 7f44f0645700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.163+0000 7f44f0645700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 0x7f44e819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44f0645700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e819d160 0x7f44e819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44f0645700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8198ea0 con 0x7f44e80732e0 2023-10-06T21:02:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44f0645700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a4ed0 con 0x7f44e819d160 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44f0645700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a5010 con 0x7f44e81987b0 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57546/0 (socket says 172.21.15.26:57546) 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 -- 172.21.15.26:0/4116295364 learned_addr learned my addr 172.21.15.26:0/4116295364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44edbe0700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 0x7f44e819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44eebe2700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e819d160 0x7f44e819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 -- 172.21.15.26:0/4116295364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 msgr2=0x7f44e819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 0x7f44e819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 -- 172.21.15.26:0/4116295364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e819d160 msgr2=0x7f44e819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e819d160 0x7f44e819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 -- 172.21.15.26:0/4116295364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40095f0 con 0x7f44e80732e0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44edbe0700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 0x7f44e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.164+0000 7f44ee3e1700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44e4004cd0 tx=0x7f44e4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.165+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e401d070 con 0x7f44e80732e0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.165+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e400be20 con 0x7f44e80732e0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.165+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81a5290 con 0x7f44e80732e0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.165+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4022e60 con 0x7f44e80732e0 2023-10-06T21:02:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.165+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81a5730 con 0x7f44e80732e0 2023-10-06T21:02:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.166+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f44e4022470 con 0x7f44e80732e0 2023-10-06T21:02:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.166+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e810a610 con 0x7f44e80732e0 2023-10-06T21:02:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.166+0000 7f44df7fe700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 0x7f44d403b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.167+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f44e404c9c0 con 0x7f44e80732e0 2023-10-06T21:02:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.167+0000 7f44edbe0700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 0x7f44d403b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.167+0000 7f44edbe0700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 0x7f44d403b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009910 tx=0x7f44d8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.172+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f44e4020550 con 0x7f44e80732e0 2023-10-06T21:02:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.408+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f44e81996a0 con 0x7f44e80732e0 2023-10-06T21:02:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.408+0000 7f44df7fe700 1 -- 172.21.15.26:0/4116295364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f44e4027070 con 0x7f44e80732e0 2023-10-06T21:02:43.411 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:02:43.411 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2023-10-06T21:02:43.411 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:02:43.411 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2023-10-06T21:02:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 msgr2=0x7f44d403b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 0x7f44d403b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009910 tx=0x7f44d8008040 comp rx=0 tx=0).stop 2023-10-06T21:02:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 msgr2=0x7f44e8198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f44e4004cd0 tx=0x7f44e4004db0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 shutdown_connections 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e819d160 0x7f44e819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44d4038d90 0x7f44d403b250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44e80732e0 0x7f44e8198270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 --2- 172.21.15.26:0/4116295364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44e81987b0 0x7f44e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 >> 172.21.15.26:0/4116295364 conn(0x7f44e80fc4a0 msgr2=0x7f44e8107750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 shutdown_connections 2023-10-06T21:02:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:43.411+0000 7f44f0645700 1 -- 172.21.15.26:0/4116295364 wait complete. 2023-10-06T21:02:43.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:43.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:43.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:43.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.690 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:43.691 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:02:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:44.213 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-06T21:02:44.213 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:02:44.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:02:44.286 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:02:44.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:44.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:44 smithi026 ceph-mon[98490]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:02:44.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:44 smithi026 ceph-mon[98490]: Reconfiguring daemon mon.a on smithi026 2023-10-06T21:02:44.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:44 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4116295364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:44.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:44.399 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T21:02:44.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:02:44.436 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T21:02:44.436 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:44.509 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T21:02:44.509 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2023-10-06T21:02:44.545 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T21:02:44.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:02:44.616 INFO:tasks.cephadm:Adding mgr.x on smithi026 2023-10-06T21:02:44.616 INFO:tasks.cephadm:Adding mgr.y on smithi078 2023-10-06T21:02:44.617 INFO:tasks.cephadm:Adding mgr.z on smithi167 2023-10-06T21:02:44.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi026=x;smithi078=y;smithi167=z' 2023-10-06T21:02:44.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:44 smithi167 ceph-mon[134494]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:02:44.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:44 smithi167 ceph-mon[134494]: Reconfiguring daemon mon.a on smithi026 2023-10-06T21:02:44.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:44 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4116295364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:44.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:44 smithi078 ceph-mon[134896]: Reconfiguring mon.a (unknown last config time)... 2023-10-06T21:02:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:44 smithi078 ceph-mon[134896]: Reconfiguring daemon mon.a on smithi026 2023-10-06T21:02:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:44 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4116295364' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.216 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:02:45.496 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:45.497 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.497 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.497 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:45.497 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:45.497 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:45.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:46.425 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 -- 172.21.15.167:0/1407391315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa818100af0 msgr2=0x7fa818100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1407391315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa818100af0 0x7fa818100f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009910 tx=0x7fa80c009c20 comp rx=0 tx=0).stop 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 -- 172.21.15.167:0/1407391315 shutdown_connections 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1407391315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa818100af0 0x7fa818100f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 -- 172.21.15.167:0/1407391315 >> 172.21.15.167:0/1407391315 conn(0x7fa8180fc0b0 msgr2=0x7fa8180fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 -- 172.21.15.167:0/1407391315 shutdown_connections 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.423+0000 7fa81eef2700 1 -- 172.21.15.167:0/1407391315 wait complete. 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 Processor -- start 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 -- start start 2023-10-06T21:02:46.426 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa818100af0 0x7fa818074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8180732a0 0x7fa818073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181a0290 con 0x7fa8180732a0 2023-10-06T21:02:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181a03d0 con 0x7fa818100af0 2023-10-06T21:02:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81eef2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181a0510 con 0x7fa818073c60 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59006/0 (socket says 172.21.15.167:59006) 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 -- 172.21.15.167:0/1007198354 learned_addr learned my addr 172.21.15.167:0/1007198354 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81cc8e700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa818100af0 0x7fa818074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa817fff700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8180732a0 0x7fa818073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 -- 172.21.15.167:0/1007198354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa818100af0 msgr2=0x7fa818074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:46.429 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa818100af0 0x7fa818074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 -- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8180732a0 msgr2=0x7fa818073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8180732a0 0x7fa818073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.424+0000 7fa81d48f700 1 -- 172.21.15.167:0/1007198354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0095f0 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa81d48f700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa81000ba50 tx=0x7fa81000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81000c930 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8181a07f0 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa810014440 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8100075b0 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.425+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8181a0d20 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.426+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7fa810007710 con 0x7fa818073c60 2023-10-06T21:02:46.430 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.426+0000 7fa815ffb700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 0x7fa80803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.426+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa81004aad0 con 0x7fa818073c60 2023-10-06T21:02:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.426+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81818f600 con 0x7fa818073c60 2023-10-06T21:02:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.426+0000 7fa81cc8e700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 0x7fa80803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.427+0000 7fa81cc8e700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 0x7fa80803b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:46.433 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.431+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa81001b8b0 con 0x7fa818073c60 2023-10-06T21:02:46.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:02:46.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: Reconfiguring daemon mon.b on smithi078 2023-10-06T21:02:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:46.646 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.644+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi078=y;smithi167=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa818106920 con 0x7fa808038d90 2023-10-06T21:02:46.653 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.651+0000 7fa815ffb700 1 -- 172.21.15.167:0/1007198354 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa818106920 con 0x7fa808038d90 2023-10-06T21:02:46.653 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2023-10-06T21:02:46.655 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.653+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 msgr2=0x7fa80803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 0x7fa80803b250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 msgr2=0x7fa81819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa81000ba50 tx=0x7fa81000bd60 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 shutdown_connections 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa818073c60 0x7fa81819fd50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa818100af0 0x7fa818074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa808038d90 0x7fa80803b250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 --2- 172.21.15.167:0/1007198354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8180732a0 0x7fa818073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 >> 172.21.15.167:0/1007198354 conn(0x7fa8180fc0b0 msgr2=0x7fa8180fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 shutdown_connections 2023-10-06T21:02:46.656 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:02:46.654+0000 7fa81eef2700 1 -- 172.21.15.167:0/1007198354 wait complete. 2023-10-06T21:02:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:02:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: Reconfiguring daemon mon.b on smithi078 2023-10-06T21:02:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:46.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: Reconfiguring mon.b (monmap changed)... 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: Reconfiguring daemon mon.b on smithi078 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-06T21:02:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.362 DEBUG:teuthology.orchestra.run.smithi078:mgr.y> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-06T21:02:47.364 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-06T21:02:47.399 INFO:journalctl@ceph.mgr.y.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:02:47.404 INFO:tasks.cephadm:Deploying OSDs... 2023-10-06T21:02:47.404 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:02:47.404 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:02:47.432 INFO:journalctl@ceph.mgr.z.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:02:47.452 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:02:47.453 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2023-10-06T21:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-10-06 21:01:56.464773149 +0000 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-10-06 20:56:17.137026351 +0000 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-10-06 20:56:17.137026351 +0000 2023-10-06T21:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-10-06 20:56:17.114026679 +0000 2023-10-06T21:02:47.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:02:47.609 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-10-06T21:02:47.609 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-10-06T21:02:47.609 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000224377 s, 2.3 MB/s 2023-10-06T21:02:47.612 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:02:47.657 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2023-10-06T21:02:47.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:02:47.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: Reconfiguring daemon mon.c on smithi167 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:47.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:47.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:47.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:47.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.741 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:02:47.741 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:47.741 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-10-06 21:01:56.557771818 +0000 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-10-06 20:56:17.957014645 +0000 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-10-06 20:56:17.957014645 +0000 2023-10-06T21:02:47.742 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-10-06 20:56:17.936014945 +0000 2023-10-06T21:02:47.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:02:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:02:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: Reconfiguring daemon mon.c on smithi167 2023-10-06T21:02:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-10-06T21:02:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-10-06T21:02:47.847 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000191848 s, 2.7 MB/s 2023-10-06T21:02:47.849 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:02:47.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: Reconfiguring mon.c (monmap changed)... 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: Reconfiguring daemon mon.c on smithi167 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:47.935 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2023-10-06T21:02:48.019 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:02:48.019 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T21:02:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-10-06 21:01:56.648770515 +0000 2023-10-06T21:02:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-10-06 20:56:18.778002925 +0000 2023-10-06T21:02:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-10-06 20:56:18.778002925 +0000 2023-10-06T21:02:48.020 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-10-06 20:56:18.757003225 +0000 2023-10-06T21:02:48.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:02:48.124 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-10-06T21:02:48.124 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-10-06T21:02:48.125 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000172071 s, 3.0 MB/s 2023-10-06T21:02:48.127 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:02:48.172 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2023-10-06T21:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-10-06 21:01:56.725769413 +0000 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-10-06 20:56:19.697989792 +0000 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-10-06 20:56:19.697989792 +0000 2023-10-06T21:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-10-06 20:56:19.674990120 +0000 2023-10-06T21:02:48.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:02:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-10-06T21:02:48.362 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-10-06T21:02:48.363 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000175067 s, 2.9 MB/s 2023-10-06T21:02:48.365 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:02:48.411 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-10-06T21:02:48.411 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:02:48.432 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:02:48.433 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2023-10-06T21:02:48.493 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:02:48.493 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:48.493 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 377708 Links: 1 2023-10-06T21:02:48.493 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:48.494 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:48.494 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-06 21:02:33.959991935 +0000 2023-10-06T21:02:48.494 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-06 20:52:36.415689167 +0000 2023-10-06T21:02:48.494 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-06 20:52:36.415689167 +0000 2023-10-06T21:02:48.494 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-06T21:02:48.494 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:02:48.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:48 smithi167 ceph-mon[134494]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi078=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:48.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:48 smithi167 ceph-mon[134494]: Saving service mgr spec with placement smithi026=x;smithi078=y;smithi167=z;count:3 2023-10-06T21:02:48.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:48 smithi167 ceph-mon[134494]: Deploying daemon mgr.z on smithi167 2023-10-06T21:02:48.565 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-06T21:02:48.565 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-06T21:02:48.565 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000245134 s, 2.1 MB/s 2023-10-06T21:02:48.566 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:02:48.628 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 377808 Links: 1 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-06 21:02:33.959991935 +0000 2023-10-06T21:02:48.689 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-06 20:52:36.933680786 +0000 2023-10-06T21:02:48.690 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-06 20:52:36.933680786 +0000 2023-10-06T21:02:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-06T21:02:48.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:02:48.757 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:48 smithi078 ceph-mon[134896]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi078=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:48.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:48 smithi078 ceph-mon[134896]: Saving service mgr spec with placement smithi026=x;smithi078=y;smithi167=z;count:3 2023-10-06T21:02:48.758 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:48 smithi078 ceph-mon[134896]: Deploying daemon mgr.z on smithi167 2023-10-06T21:02:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-06T21:02:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-06T21:02:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000237846 s, 2.2 MB/s 2023-10-06T21:02:48.762 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:02:48.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:48 smithi026 ceph-mon[98490]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi078=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:48.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:48 smithi026 ceph-mon[98490]: Saving service mgr spec with placement smithi026=x;smithi078=y;smithi167=z;count:3 2023-10-06T21:02:48.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:48 smithi026 ceph-mon[98490]: Deploying daemon mgr.z on smithi167 2023-10-06T21:02:48.824 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2023-10-06T21:02:48.872 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:48 smithi167 systemd[1]: Starting Ceph mgr.z for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:02:48.885 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:02:48.885 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:48.885 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 390612 Links: 1 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-06 21:02:33.959991935 +0000 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-06 20:52:37.452672390 +0000 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-06 20:52:37.452672390 +0000 2023-10-06T21:02:48.886 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-06T21:02:48.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:02:48.956 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-06T21:02:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-06T21:02:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.0002634 s, 1.9 MB/s 2023-10-06T21:02:48.958 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:02:49.020 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2023-10-06T21:02:49.081 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:02:49.081 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:49.081 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 391617 Links: 1 2023-10-06T21:02:49.081 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:49.082 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:49.082 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-10-06 21:02:33.959991935 +0000 2023-10-06T21:02:49.082 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-10-06 20:52:38.006663426 +0000 2023-10-06T21:02:49.082 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-10-06 20:52:38.006663426 +0000 2023-10-06T21:02:49.082 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-10-06T21:02:49.082 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:02:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-10-06T21:02:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-10-06T21:02:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000289005 s, 1.8 MB/s 2023-10-06T21:02:49.155 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:02:49.166 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:48 smithi167 podman[136359]: 2023-10-06T21:02:49.166 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 bash[136359]: 0efcc8cf6f9b24e55493cd7ab8b3d8bdc35b06ccc8c13fadb147bd0f5a1a42f2 2023-10-06T21:02:49.166 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mgr[136399]: -- 172.21.15.167:0/89052857 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560c78af1e0 con 0x5560c7885000 2023-10-06T21:02:49.217 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-06T21:02:49.217 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T21:02:49.237 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T21:02:49.237 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-10-06T21:02:49.294 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 392691 Links: 1 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 21:02:48.535126852 +0000 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 20:52:45.364094661 +0000 2023-10-06T21:02:49.295 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 20:52:45.364094661 +0000 2023-10-06T21:02:49.296 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T21:02:49.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T21:02:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T21:02:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T21:02:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000243169 s, 2.1 MB/s 2023-10-06T21:02:49.364 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T21:02:49.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 systemd[1]: Started Ceph mgr.z for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:02:49.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:49.239+0000 7fa4e7e84200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:02:49.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:49.322+0000 7fa4e7e84200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:02:49.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:49.418+0000 7fa4e7e84200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:49.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:49.422 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-10-06T21:02:49.479 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 387549 Links: 1 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 21:02:48.535126852 +0000 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 20:52:45.879085175 +0000 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 20:52:45.879085175 +0000 2023-10-06T21:02:49.480 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T21:02:49.480 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T21:02:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T21:02:49.547 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T21:02:49.547 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000220752 s, 2.3 MB/s 2023-10-06T21:02:49.548 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T21:02:49.606 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-10-06T21:02:49.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:49.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:49.663 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T21:02:49.663 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 390086 Links: 1 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 21:02:48.535126852 +0000 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 20:52:46.404075506 +0000 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 20:52:46.404075506 +0000 2023-10-06T21:02:49.664 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T21:02:49.664 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T21:02:49.696 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:49.515+0000 7fa4e7e84200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:02:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T21:02:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T21:02:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000230976 s, 2.2 MB/s 2023-10-06T21:02:49.723 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T21:02:49.780 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-10-06T21:02:49.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:49.837 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T21:02:49.837 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 391695 Links: 1 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-06 21:02:48.535126852 +0000 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-06 20:52:46.919066020 +0000 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-06 20:52:46.919066020 +0000 2023-10-06T21:02:49.838 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-06T21:02:49.839 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T21:02:49.903 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-06T21:02:49.903 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-06T21:02:49.903 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000225159 s, 2.3 MB/s 2023-10-06T21:02:49.904 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T21:02:49.962 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2023-10-06T21:02:49.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:02:50.033 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:49.694+0000 7fa4e7e84200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:02:50.400 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:50.078+0000 7fa4e7e84200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:02:50.400 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:50.214+0000 7fa4e7e84200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:02:50.400 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:50.303+0000 7fa4e7e84200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:02:50.543 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:50 smithi078 ceph-mon[134896]: Deploying daemon mgr.y on smithi078 2023-10-06T21:02:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:50.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-mon[134494]: Deploying daemon mgr.y on smithi078 2023-10-06T21:02:50.783 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:50.399+0000 7fa4e7e84200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:02:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:50 smithi026 ceph-mon[98490]: Deploying daemon mgr.y on smithi078 2023-10-06T21:02:50.856 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:50 smithi078 systemd[1]: Starting Ceph mgr.y for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:02:51.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:50.825+0000 7fa4e7e84200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:02:51.399 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 podman[136339]: 2023-10-06T21:02:51.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:51 smithi026 ceph-mon[98490]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-mon[134896]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 bash[136339]: 1196f0b7f519d668c24a4a5e8f514eb1cc81cf2c2086eabf982c08f1182e60fc 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-mgr[136380]: -- 172.21.15.78:0/355107605 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f910a9d1e0 con 0x55f910a73000 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 systemd[1]: Started Ceph mgr.y for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:51.536+0000 7f4cc6f23200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:51.615+0000 7f4cc6f23200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:02:51.712 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:51.709+0000 7f4cc6f23200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:02:51.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:51 smithi167 ceph-mon[134494]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:51.783 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:51.495+0000 7fa4e7e84200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:02:51.783 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:51.576+0000 7fa4e7e84200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:02:51.991 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:51.807+0000 7f4cc6f23200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:02:51.991 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:51 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:51.987+0000 7f4cc6f23200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:02:52.665 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:52.381+0000 7fa4e7e84200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:02:52.665 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:52.482+0000 7fa4e7e84200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:02:52.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:52.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-mon[134494]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:52.702 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:52.371+0000 7f4cc6f23200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:02:52.702 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:52.509+0000 7f4cc6f23200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:52.603+0000 7f4cc6f23200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:52.703 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-mon[134896]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:52.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:52.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:52 smithi026 ceph-mon[98490]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:52.842 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:02:52.842 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:02:52.842 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-10-06T21:02:52.842 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-10-06T21:02:52.842 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0297957 s, 352 MB/s 2023-10-06T21:02:52.843 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-10-06T21:02:52.965 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:52.663+0000 7fa4e7e84200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:02:53.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:52.963+0000 7fa4e7e84200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:02:53.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:53.204+0000 7fa4e7e84200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:02:53.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2023-10-06T21:02:53.620 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:52.699+0000 7f4cc6f23200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:02:53.620 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:53 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:53.140+0000 7f4cc6f23200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:02:53.622 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:53.353+0000 7fa4e7e84200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:02:54.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:53.620+0000 7fa4e7e84200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:02:54.053 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:02:54.120 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:53 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:53.813+0000 7f4cc6f23200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:02:54.121 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:53 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:53.894+0000 7f4cc6f23200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:02:54.459 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:54.326+0000 7fa4e7e84200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:54.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:54.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:54.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-06T21:02:54.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-06T21:02:54.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:54.768 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:54.457+0000 7fa4e7e84200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:02:54.768 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:54.652+0000 7fa4e7e84200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:02:54.918 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:54.656+0000 7f4cc6f23200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:02:54.918 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:54.746+0000 7f4cc6f23200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:02:55.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:54.766+0000 7fa4e7e84200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:02:55.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:54.985+0000 7fa4e7e84200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:02:55.192 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:54 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:54.915+0000 7f4cc6f23200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-mon[134896]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-mon[134896]: Reconfiguring daemon mgr.x on smithi026 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-mon[134896]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:55.189+0000 7f4cc6f23200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:02:55.562 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:55.413+0000 7f4cc6f23200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:02:55.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:55 smithi026 ceph-mon[98490]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:02:55.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:55 smithi026 ceph-mon[98490]: Reconfiguring daemon mgr.x on smithi026 2023-10-06T21:02:55.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:55 smithi026 ceph-mon[98490]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:55.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:55.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:55 smithi167 ceph-mon[134494]: Reconfiguring mgr.x (unknown last config time)... 2023-10-06T21:02:55.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:55 smithi167 ceph-mon[134494]: Reconfiguring daemon mgr.x on smithi026 2023-10-06T21:02:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:55 smithi167 ceph-mon[134494]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:55.829 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:55.559+0000 7f4cc6f23200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 -- 172.21.15.26:0/2660480924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac102180 msgr2=0x7f4eac1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 --2- 172.21.15.26:0/2660480924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac102180 0x7f4eac1025e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009990 tx=0x7f4e9c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 -- 172.21.15.26:0/2660480924 shutdown_connections 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 --2- 172.21.15.26:0/2660480924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 0x7f4eac1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 --2- 172.21.15.26:0/2660480924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac100f80 0x7f4eac1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 --2- 172.21.15.26:0/2660480924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac102180 0x7f4eac1025e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 -- 172.21.15.26:0/2660480924 >> 172.21.15.26:0/2660480924 conn(0x7f4eac0fc520 msgr2=0x7f4eac0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:02:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 -- 172.21.15.26:0/2660480924 shutdown_connections 2023-10-06T21:02:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.921+0000 7f4eb385b700 1 -- 172.21.15.26:0/2660480924 wait complete. 2023-10-06T21:02:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.922+0000 7f4eb385b700 1 Processor -- start 2023-10-06T21:02:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 -- start start 2023-10-06T21:02:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 0x7f4eac198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 0x7f4eac1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1a0a10 con 0x7f4eac102b20 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1a0b50 con 0x7f4eac102180 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb385b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1a0ce0 con 0x7f4eac100f80 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb0df6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb15f7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 0x7f4eac198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb15f7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 0x7f4eac198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60726/0 (socket says 172.21.15.26:60726) 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb0df6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36914/0 (socket says 172.21.15.26:36914) 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb15f7700 1 -- 172.21.15.26:0/3089541513 learned_addr learned my addr 172.21.15.26:0/3089541513 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:02:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.923+0000 7f4eb1df8700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 0x7f4eac1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 -- 172.21.15.26:0/3089541513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 msgr2=0x7f4eac198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 0x7f4eac198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 -- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 msgr2=0x7f4eac1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 0x7f4eac1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 -- 172.21.15.26:0/3089541513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4009c40 con 0x7f4eac102180 2023-10-06T21:02:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb0df6700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0098e0 tx=0x7f4e9c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c01d070 con 0x7f4eac102180 2023-10-06T21:02:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e9c021410 con 0x7f4eac102180 2023-10-06T21:02:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c017650 con 0x7f4eac102180 2023-10-06T21:02:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.924+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0095f0 con 0x7f4eac102180 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.925+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac1a1190 con 0x7f4eac102180 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.926+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) v1 ==== 45635+0+0 (secure 0 0 0) 0x7f4e9c021580 con 0x7f4eac102180 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.926+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac1935c0 con 0x7f4eac102180 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.926+0000 7f4ea27fc700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 0x7f4e9803afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.926+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4e9c04c790 con 0x7f4eac102180 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.926+0000 7f4eb15f7700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 0x7f4e9803afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:02:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.927+0000 7f4eb15f7700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 0x7f4e9803afb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005950 tx=0x7f4ea8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:02:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:55.931+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e9c029950 con 0x7f4eac102180 2023-10-06T21:02:56.120 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:55 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:55.827+0000 7f4cc6f23200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:02:56.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:56.155+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eac10c7d0 con 0x7f4e98038af0 2023-10-06T21:02:56.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:02:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:02:55.853+0000 7fa4e7e84200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:02:56.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:56.614+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 14) v1 ==== 90791+0+0 (secure 0 0 0) 0x7f4e9c02b440 con 0x7f4eac102180 2023-10-06T21:02:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: Standby manager daemon z started 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:02:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:56.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:56 smithi026 ceph-mon[98490]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:56.845 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:56.518+0000 7f4cc6f23200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:02:56.845 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:56.651+0000 7f4cc6f23200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: Standby manager daemon z started 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:56.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:56.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-mon[134896]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:57.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: Standby manager daemon z started 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:02:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/1865709119' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:02:57.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:56 smithi167 ceph-mon[134494]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:57.120 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:56.842+0000 7f4cc6f23200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:02:57.121 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:56 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:56.956+0000 7f4cc6f23200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:02:57.616 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:57.173+0000 7f4cc6f23200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:02:57.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:57 smithi026 ceph-mon[98490]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T21:02:57.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:02:57.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:57 smithi078 ceph-mon[134896]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T21:02:57.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:02:58.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:57 smithi167 ceph-mon[134494]: mgrmap e14: x(active, since 72s), standbys: z 2023-10-06T21:02:58.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:02:58.370 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:02:58.046+0000 7f4cc6f23200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:02:58.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:02:58.626+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f4e9c045ba0 con 0x7f4eac102180 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: Standby manager daemon y started 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:58 smithi078 ceph-mon[134896]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: Standby manager daemon y started 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:58 smithi167 ceph-mon[134494]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: Standby manager daemon y started 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/2100825379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:02:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:58 smithi026 ceph-mon[98490]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:00.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/387290498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]': finished 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:03:00.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:02:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/387290498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]': finished 2023-10-06T21:03:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:03:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:02:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/387290498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]: dispatch 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e106834-15f2-43e2-86a4-4256e7169c93"}]': finished 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: osdmap e5: 1 total, 0 up, 1 in 2023-10-06T21:03:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:02:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:00 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2124293687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:00 smithi167 ceph-mon[134494]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:00 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2124293687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:00 smithi026 ceph-mon[98490]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:00 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2124293687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:00 smithi078 ceph-mon[134896]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:03.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:03 smithi167 ceph-mon[134494]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:03.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:03 smithi026 ceph-mon[98490]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:03.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:03 smithi078 ceph-mon[134896]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:05.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:05 smithi078 ceph-mon[134896]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:05.736 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:05 smithi026 ceph-mon[98490]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:05.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:05 smithi167 ceph-mon[134494]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:07.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:07 smithi078 ceph-mon[134896]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:07.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:07 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:03:07.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:07 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:07.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:07 smithi026 ceph-mon[98490]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:07.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:03:07.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:07 smithi167 ceph-mon[134494]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:07 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-06T21:03:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:07 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:08.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:08 smithi026 ceph-mon[98490]: Deploying daemon osd.0 on smithi026 2023-10-06T21:03:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:08 smithi078 ceph-mon[134896]: Deploying daemon osd.0 on smithi026 2023-10-06T21:03:08.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:08 smithi167 ceph-mon[134494]: Deploying daemon osd.0 on smithi026 2023-10-06T21:03:09.554 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:09 smithi026 ceph-mon[98490]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:09.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:09 smithi078 ceph-mon[134896]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:09.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:09 smithi167 ceph-mon[134494]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:11.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:11 smithi078 ceph-mon[134896]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:11.660 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:11 smithi026 ceph-mon[98490]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:11.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:11 smithi167 ceph-mon[134494]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:13.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:13 smithi167 ceph-mon[134494]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:13.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:13 smithi026 ceph-mon[98490]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:13.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:13.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:13 smithi078 ceph-mon[134896]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:15.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:15 smithi026 ceph-mon[98490]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:15.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:15 smithi078 ceph-mon[134896]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:15 smithi167 ceph-mon[134494]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:16.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.621+0000 7f4ea27fc700 1 -- 172.21.15.26:0/3089541513 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4eac10c7d0 con 0x7f4e98038af0 2023-10-06T21:03:16.625 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2023-10-06T21:03:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 msgr2=0x7f4e9803afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 0x7f4e9803afb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005950 tx=0x7f4ea8009260 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 msgr2=0x7f4eac1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0098e0 tx=0x7f4e9c009580 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 shutdown_connections 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4e98038af0 0x7f4e9803afb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac102b20 0x7f4eac1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac100f80 0x7f4eac198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 --2- 172.21.15.26:0/3089541513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eac102180 0x7f4eac1993c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.624+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 >> 172.21.15.26:0/3089541513 conn(0x7f4eac0fc520 msgr2=0x7f4eac107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.625+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 shutdown_connections 2023-10-06T21:03:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:03:16.625+0000 7f4eb385b700 1 -- 172.21.15.26:0/3089541513 wait complete. 2023-10-06T21:03:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:17.187 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-06T21:03:17.191 INFO:tasks.cephadm:Deploying osd.1 on smithi078 with /dev/vg_nvme/lv_4... 2023-10-06T21:03:17.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:03:17.271 INFO:journalctl@ceph.osd.0.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:03:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:03:18.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:17 smithi026 ceph-mon[98490]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:03:18.006 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:17 smithi078 ceph-mon[134896]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:03:18.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:17 smithi167 ceph-mon[134494]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-06T21:03:18.896 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:18 smithi078 ceph-mon[134896]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:03:18.897 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:18 smithi078 ceph-mon[134896]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:03:18.897 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:18 smithi078 ceph-mon[134896]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:03:18.897 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:18.897 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:18 smithi078 ceph-mon[134896]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:19.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:18 smithi167 ceph-mon[134494]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:03:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:18 smithi167 ceph-mon[134494]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:03:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:18 smithi167 ceph-mon[134494]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:03:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:18 smithi167 ceph-mon[134494]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-mon[98490]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-06T21:03:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-mon[98490]: osdmap e6: 1 total, 0 up, 1 in 2023-10-06T21:03:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-mon[98490]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:03:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-mon[98490]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:19.074 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0[116687]: 2023-10-06T21:03:18.640+0000 7f9b89bf2700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T21:03:19.074 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:03:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0[116687]: 2023-10-06T21:03:18.648+0000 7f9b841e4700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:03:19.385 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:03:19.385 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:03:19.385 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2023-10-06T21:03:19.385 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2023-10-06T21:03:19.385 INFO:teuthology.orchestra.run.smithi078.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0141278 s, 742 MB/s 2023-10-06T21:03:19.386 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2023-10-06T21:03:19.739 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:19 smithi078 ceph-mon[134896]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:03:19.739 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:19 smithi078 ceph-mon[134896]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:03:19.740 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:19.740 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:19.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2023-10-06T21:03:20.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:19 smithi167 ceph-mon[134494]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:03:20.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:19 smithi167 ceph-mon[134494]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:03:20.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:19 smithi026 ceph-mon[98490]: from='osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:03:20.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:19 smithi026 ceph-mon[98490]: osdmap e7: 1 total, 0 up, 1 in 2023-10-06T21:03:20.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.039 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:03:20.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] boot 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:20.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] boot 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:03:20.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:20.834 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: osd.0 [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] boot 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: osdmap e8: 1 total, 1 up, 1 in 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- 172.21.15.78:0/3742883457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 msgr2=0x7f3ca4106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/3742883457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca4106760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009990 tx=0x7f3c8c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- 172.21.15.78:0/3742883457 shutdown_connections 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/3742883457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4106ca0 0x7f3ca41090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/3742883457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ca4101a40 0x7f3ca4103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/3742883457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca4106760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- 172.21.15.78:0/3742883457 >> 172.21.15.78:0/3742883457 conn(0x7f3ca40fb3f0 msgr2=0x7f3ca40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:03:21.540 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- 172.21.15.78:0/3742883457 shutdown_connections 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- 172.21.15.78:0/3742883457 wait complete. 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 Processor -- start 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.535+0000 7f3caa9cf700 1 -- start start 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4101a40 0x7f3ca41013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ca4106ca0 0x7f3ca40fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca41006b0 con 0x7f3ca4106ca0 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca4100820 con 0x7f3ca4104370 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca41a0c60 con 0x7f3ca4101a40 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40286/0 (socket says 172.21.15.78:40286) 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 -- 172.21.15.78:0/2572962074 learned_addr learned my addr 172.21.15.78:0/2572962074 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca3fff700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4101a40 0x7f3ca41013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 -- 172.21.15.78:0/2572962074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4101a40 msgr2=0x7f3ca41013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4101a40 0x7f3ca41013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 -- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ca4106ca0 msgr2=0x7f3ca40fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ca4106ca0 0x7f3ca40fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 -- 172.21.15.78:0/2572962074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0095f0 con 0x7f3ca4104370 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca37fe700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c005910 tx=0x7f3c8c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c01d070 con 0x7f3ca4104370 2023-10-06T21:03:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c8c00fc60 con 0x7f3ca4104370 2023-10-06T21:03:21.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca41a0ee0 con 0x7f3ca4104370 2023-10-06T21:03:21.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c017650 con 0x7f3ca4104370 2023-10-06T21:03:21.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.536+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca41a1240 con 0x7f3ca4104370 2023-10-06T21:03:21.543 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.537+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca40fcf70 con 0x7f3ca4104370 2023-10-06T21:03:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.538+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3c8c022470 con 0x7f3ca4104370 2023-10-06T21:03:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.539+0000 7f3ca17fa700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 0x7f3c900a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.539+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f3c8c0cc640 con 0x7f3ca4104370 2023-10-06T21:03:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.539+0000 7f3ca3fff700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 0x7f3c900a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.539+0000 7f3ca3fff700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 0x7f3c900a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005950 tx=0x7f3c94005cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:03:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.543+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3c8c05d100 con 0x7f3ca4104370 2023-10-06T21:03:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:21.740+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ca402d340 con 0x7f3c900a13f0 2023-10-06T21:03:22.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:21 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi026 2023-10-06T21:03:22.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:21 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 17272M 2023-10-06T21:03:22.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:21 smithi167 ceph-mon[134494]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:03:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:21 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi026 2023-10-06T21:03:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:21 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 17272M 2023-10-06T21:03:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:21 smithi026 ceph-mon[98490]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:03:22.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:21 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi026 2023-10-06T21:03:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:21 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 17272M 2023-10-06T21:03:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:21 smithi078 ceph-mon[134896]: osdmap e9: 1 total, 1 up, 1 in 2023-10-06T21:03:22.858 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:22 smithi078 ceph-mon[134896]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:22.858 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:22.858 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:22.859 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:22.859 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:22 smithi078 ceph-mon[134896]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:22 smithi167 ceph-mon[134494]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:22 smithi167 ceph-mon[134494]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:22 smithi026 ceph-mon[98490]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:23.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:23.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:22 smithi026 ceph-mon[98490]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:24 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/3507554452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:24 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:24 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]': finished 2023-10-06T21:03:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:24 smithi167 ceph-mon[134494]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:03:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:24 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:24 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/3507554452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:24 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:24 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]': finished 2023-10-06T21:03:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:24 smithi026 ceph-mon[98490]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:03:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:24 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:24 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/3507554452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:24 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]: dispatch 2023-10-06T21:03:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:24 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc8d8aba-6316-4996-bb52-6f32af502c53"}]': finished 2023-10-06T21:03:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:24 smithi078 ceph-mon[134896]: osdmap e10: 2 total, 1 up, 2 in 2023-10-06T21:03:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:24 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:25.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:25 smithi167 ceph-mon[134494]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:25.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:25 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/4161421404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:25 smithi026 ceph-mon[98490]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:25.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:25 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/4161421404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:25.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:25 smithi078 ceph-mon[134896]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:25.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:25 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/4161421404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:27.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:27 smithi078 ceph-mon[134896]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:27.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:27 smithi167 ceph-mon[134494]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:27.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:27 smithi026 ceph-mon[98490]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:29.490 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:29 smithi078 ceph-mon[134896]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:29.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:29 smithi167 ceph-mon[134494]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:29.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:29 smithi026 ceph-mon[98490]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:31 smithi078 ceph-mon[134896]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:03:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:31.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:31 smithi167 ceph-mon[134494]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:03:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:31.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:31 smithi026 ceph-mon[98490]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-06T21:03:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:32.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:32 smithi078 ceph-mon[134896]: Deploying daemon osd.1 on smithi078 2023-10-06T21:03:32.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:32 smithi167 ceph-mon[134494]: Deploying daemon osd.1 on smithi078 2023-10-06T21:03:32.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:32 smithi026 ceph-mon[98490]: Deploying daemon osd.1 on smithi078 2023-10-06T21:03:33.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:33 smithi078 ceph-mon[134896]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:33 smithi167 ceph-mon[134494]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:33 smithi026 ceph-mon[98490]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:35.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:35 smithi167 ceph-mon[134494]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:35 smithi026 ceph-mon[98490]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:35 smithi078 ceph-mon[134896]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:37.750 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:37.750 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.750 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.750 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:37.751 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:37.751 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.810 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 1 on host 'smithi078' 2023-10-06T21:03:37.810 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.807+0000 7f3ca17fa700 1 -- 172.21.15.78:0/2572962074 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3ca402d340 con 0x7f3c900a13f0 2023-10-06T21:03:37.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.809+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 msgr2=0x7f3c900a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:37.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.809+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 0x7f3c900a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005950 tx=0x7f3c94005cf0 comp rx=0 tx=0).stop 2023-10-06T21:03:37.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.809+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 msgr2=0x7f3ca40ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:37.813 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.809+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c005910 tx=0x7f3c8c00f740 comp rx=0 tx=0).stop 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 shutdown_connections 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3c900a13f0 0x7f3c900a38b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ca4106ca0 0x7f3ca40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ca4101a40 0x7f3ca41013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 --2- 172.21.15.78:0/2572962074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ca4104370 0x7f3ca40ff9f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:37.814 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 >> 172.21.15.78:0/2572962074 conn(0x7f3ca40fb3f0 msgr2=0x7f3ca41026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:03:37.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 shutdown_connections 2023-10-06T21:03:37.815 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:03:37.810+0000 7f3caa9cf700 1 -- 172.21.15.78:0/2572962074 wait complete. 2023-10-06T21:03:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:38.228 DEBUG:teuthology.orchestra.run.smithi078:osd.1> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-06T21:03:38.230 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2023-10-06T21:03:38.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-06T21:03:38.261 INFO:journalctl@ceph.osd.1.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:03:38.797 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:03:38.972 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:38.972 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:38.973 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:38.973 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:38 smithi078 ceph-mon[134896]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:38 smithi167 ceph-mon[134494]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:39.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:38 smithi026 ceph-mon[98490]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:39.266 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:03:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:03:39.066+0000 7fd618644640 -1 osd.1 0 log_to_monitors true 2023-10-06T21:03:40.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:39 smithi026 ceph-mon[98490]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:39 smithi026 ceph-mon[98490]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:39 smithi078 ceph-mon[134896]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:39 smithi078 ceph-mon[134896]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:39 smithi167 ceph-mon[134494]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:39 smithi167 ceph-mon[134494]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-06T21:03:40.309 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-06T21:03:40.309 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-06T21:03:40.309 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-10-06T21:03:40.309 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-10-06T21:03:40.309 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127349 s, 823 MB/s 2023-10-06T21:03:40.310 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-10-06T21:03:40.753 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:40 smithi167 ceph-mon[134494]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:03:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:03:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:40 smithi026 ceph-mon[98490]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:03:41.121 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:03:40.791+0000 7fd60d4ba700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T21:03:41.121 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:03:40.794+0000 7fd607aac700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:03:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-06T21:03:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: osdmap e11: 2 total, 1 up, 2 in 2023-10-06T21:03:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:41.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:40 smithi078 ceph-mon[134896]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:03:41.311 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:03:42.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi078 2023-10-06T21:03:42.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 17168M 2023-10-06T21:03:42.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:03:42.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:41 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi078 2023-10-06T21:03:42.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 17168M 2023-10-06T21:03:42.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:03:42.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:41 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi078 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 17168M 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: osdmap e12: 2 total, 1 up, 2 in 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 -- 172.21.15.167:0/1574564553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50100fd0 msgr2=0x7f2a50101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/1574564553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50100fd0 0x7f2a50101450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009990 tx=0x7f2a4c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 -- 172.21.15.167:0/1574564553 shutdown_connections 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/1574564553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50101990 0x7f2a501082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/1574564553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/1574564553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50100fd0 0x7f2a50101450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 -- 172.21.15.167:0/1574564553 >> 172.21.15.167:0/1574564553 conn(0x7f2a500fb3f0 msgr2=0x7f2a500fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:03:42.506 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 -- 172.21.15.167:0/1574564553 shutdown_connections 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.501+0000 7f2a57dcd700 1 -- 172.21.15.167:0/1574564553 wait complete. 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 Processor -- start 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 -- start start 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 0x7f2a5019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5019e6b0 con 0x7f2a50100fd0 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5019e820 con 0x7f2a500ffe70 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a57dcd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5019e9e0 con 0x7f2a50101990 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a5636a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:42.507 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a55b69700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a5636a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59038/0 (socket says 172.21.15.167:59038) 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.502+0000 7f2a55b69700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.167:53952/0 (socket says 172.21.15.167:53952) 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 -- 172.21.15.167:0/3930733138 learned_addr learned my addr 172.21.15.167:0/3930733138 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a55368700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 0x7f2a5019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 -- 172.21.15.167:0/3930733138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 msgr2=0x7f2a50069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 -- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 msgr2=0x7f2a5019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 0x7f2a5019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 -- 172.21.15.167:0/3930733138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0095f0 con 0x7f2a50101990 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a55368700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 0x7f2a5019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a5636a700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800ada0 tx=0x7f2a4800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:03:42.508 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800f070 con 0x7f2a50101990 2023-10-06T21:03:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a48007b30 con 0x7f2a50101990 2023-10-06T21:03:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480053e0 con 0x7f2a50101990 2023-10-06T21:03:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5019ecc0 con 0x7f2a50101990 2023-10-06T21:03:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.503+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5019f170 con 0x7f2a50101990 2023-10-06T21:03:42.509 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.504+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a501074a0 con 0x7f2a50101990 2023-10-06T21:03:42.510 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.505+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2a4800c980 con 0x7f2a50101990 2023-10-06T21:03:42.510 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.506+0000 7f2a42ffd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 0x7f2a3c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:03:42.510 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.506+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f2a480caff0 con 0x7f2a50101990 2023-10-06T21:03:42.510 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.506+0000 7f2a55b69700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 0x7f2a3c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:03:42.510 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.507+0000 7f2a55b69700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 0x7f2a3c0a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005950 tx=0x7f2a4400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:03:42.511 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.509+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2a48095f30 con 0x7f2a50101990 2023-10-06T21:03:42.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:42.707+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a5002d270 con 0x7f2a3c0a13f0 2023-10-06T21:03:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:03:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:03:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] boot 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] boot 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: osd.1 [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] boot 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: osdmap e13: 2 total, 2 up, 2 in 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:03:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:44.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:43 smithi026 ceph-mon[98490]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:44.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:43 smithi026 ceph-mon[98490]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:03:44.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:43 smithi078 ceph-mon[134896]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:44.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:43 smithi078 ceph-mon[134896]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:03:44.126 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:43 smithi167 ceph-mon[134494]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:03:44.126 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:43 smithi167 ceph-mon[134494]: osdmap e14: 2 total, 2 up, 2 in 2023-10-06T21:03:45.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:44 smithi078 ceph-mon[134896]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:45.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:44 smithi167 ceph-mon[134494]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:45.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:44 smithi026 ceph-mon[98490]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:46.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/1180328983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]': finished 2023-10-06T21:03:46.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:03:46.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:03:46.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:45 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/1491501540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/1180328983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]': finished 2023-10-06T21:03:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:03:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:03:46.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:45 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/1491501540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:46.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/1180328983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]: dispatch 2023-10-06T21:03:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30a5233e-8442-48f1-af3e-3a0752ff274e"}]': finished 2023-10-06T21:03:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: osdmap e15: 3 total, 2 up, 3 in 2023-10-06T21:03:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:03:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:45 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/1491501540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:03:47.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:46 smithi078 ceph-mon[134896]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:47.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:46 smithi167 ceph-mon[134494]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:47.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:46 smithi026 ceph-mon[98490]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:49.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:49 smithi078 ceph-mon[134896]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:49.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:49 smithi167 ceph-mon[134494]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:49.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:49 smithi026 ceph-mon[98490]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:51.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:51 smithi078 ceph-mon[134896]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:51 smithi167 ceph-mon[134494]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:51 smithi026 ceph-mon[98490]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:03:52.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:52.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:03:52.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-06T21:03:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:53.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:53 smithi167 ceph-mon[134494]: Deploying daemon osd.2 on smithi167 2023-10-06T21:03:53.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:53 smithi167 ceph-mon[134494]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:53.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:53 smithi026 ceph-mon[98490]: Deploying daemon osd.2 on smithi167 2023-10-06T21:03:53.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:53 smithi026 ceph-mon[98490]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:53 smithi078 ceph-mon[134896]: Deploying daemon osd.2 on smithi167 2023-10-06T21:03:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:53 smithi078 ceph-mon[134896]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:55.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:55 smithi167 ceph-mon[134494]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:55 smithi026 ceph-mon[98490]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:55.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:55 smithi078 ceph-mon[134896]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:57.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:57 smithi167 ceph-mon[134494]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:57 smithi026 ceph-mon[98490]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:57.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:57.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:57.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:57 smithi078 ceph-mon[134896]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:58.823 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.820+0000 7f2a42ffd700 1 -- 172.21.15.167:0/3930733138 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2a5002d270 con 0x7f2a3c0a13f0 2023-10-06T21:03:58.823 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 msgr2=0x7f2a3c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 0x7f2a3c0a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005950 tx=0x7f2a4400b410 comp rx=0 tx=0).stop 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 msgr2=0x7f2a5019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800ada0 tx=0x7f2a4800c330 comp rx=0 tx=0).stop 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 shutdown_connections 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a50101990 0x7f2a5019df10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:58.826 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a500ffe70 0x7f2a50069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:58.827 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2a3c0a13f0 0x7f2a3c0a38b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:58.827 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 --2- 172.21.15.167:0/3930733138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a50100fd0 0x7f2a5019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:03:58.827 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 >> 172.21.15.167:0/3930733138 conn(0x7f2a500fb3f0 msgr2=0x7f2a501045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:03:58.827 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 shutdown_connections 2023-10-06T21:03:58.827 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:03:58.823+0000 7f2a57dcd700 1 -- 172.21.15.167:0/3930733138 wait complete. 2023-10-06T21:03:59.197 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-06T21:03:59.200 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2023-10-06T21:03:59.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:03:59.233 INFO:journalctl@ceph.osd.2.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:03:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:59.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:03:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:59.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:03:59.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:03:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:03:59.946 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:04:00.227 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:04:00.143+0000 7f14063ef640 -1 osd.2 0 log_to_monitors true 2023-10-06T21:04:00.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:00 smithi026 ceph-mon[98490]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:00.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:00 smithi026 ceph-mon[98490]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:00.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:00 smithi167 ceph-mon[134494]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:00.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:00 smithi167 ceph-mon[134494]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:00.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:00 smithi078 ceph-mon[134896]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:00.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:00 smithi078 ceph-mon[134896]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-06T21:04:01.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:04:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:04:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:01 smithi026 ceph-mon[98490]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:01.609 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:04:01.332+0000 7f13fb265700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T21:04:01.609 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:04:01.336+0000 7f13f5857700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:04:01.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:01.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:04:01.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:04:01.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:01.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:01.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:01 smithi167 ceph-mon[134494]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:01.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-06T21:04:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: osdmap e16: 3 total, 2 up, 3 in 2023-10-06T21:04:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: from='osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:01 smithi078 ceph-mon[134896]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:04:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:04:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:04:02.035 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-10-06T21:04:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-10-06T21:04:02.035 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0294744 s, 356 MB/s 2023-10-06T21:04:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-10-06T21:04:02.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2023-10-06T21:04:02.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:04:02.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:04:02.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:02.593 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:02.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:02.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: osdmap e17: 3 total, 2 up, 3 in 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:04:03.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:04:03.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:04:03.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi167 2023-10-06T21:04:03.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 17168M 2023-10-06T21:04:03.597 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:03.598 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] boot 2023-10-06T21:04:03.598 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:04:03.598 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:03.598 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:03.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:04:03.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:04:03.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi167 2023-10-06T21:04:03.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 17168M 2023-10-06T21:04:03.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:03.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] boot 2023-10-06T21:04:03.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:04:03.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:03.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:04:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:04:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi167 2023-10-06T21:04:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 17168M 2023-10-06T21:04:03.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-06T21:04:03.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: osd.2 [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] boot 2023-10-06T21:04:03.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: osdmap e18: 3 total, 3 up, 3 in 2023-10-06T21:04:03.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:04:03.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:04.613 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:04 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:04.614 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:04 smithi026 ceph-mon[98490]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:04:04.614 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:04 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:04.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:04 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:04 smithi078 ceph-mon[134896]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:04:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:04 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:04.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:04 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:04 smithi167 ceph-mon[134494]: osdmap e19: 3 total, 3 up, 3 in 2023-10-06T21:04:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:04 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:04:04.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 -- 172.21.15.26:0/3150847551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac073910 msgr2=0x7ff2ac073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/3150847551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac073910 0x7ff2ac073d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009990 tx=0x7ff2a0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 -- 172.21.15.26:0/3150847551 shutdown_connections 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/3150847551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac10d560 0x7ff2ac10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/3150847551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/3150847551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac073910 0x7ff2ac073d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 -- 172.21.15.26:0/3150847551 >> 172.21.15.26:0/3150847551 conn(0x7ff2ac0fc520 msgr2=0x7ff2ac0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 -- 172.21.15.26:0/3150847551 shutdown_connections 2023-10-06T21:04:04.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.793+0000 7ff2b0f43700 1 -- 172.21.15.26:0/3150847551 wait complete. 2023-10-06T21:04:04.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 Processor -- start 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 -- start start 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac073910 0x7ff2ac19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac10d560 0x7ff2ac1a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1a4ea0 con 0x7ff2ac10d560 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1a4fe0 con 0x7ff2ac074ee0 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.794+0000 7ff2b0f43700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1a5170 con 0x7ff2ac073910 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57732/0 (socket says 172.21.15.26:57732) 2023-10-06T21:04:04.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 -- 172.21.15.26:0/869157866 learned_addr learned my addr 172.21.15.26:0/869157866 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2aa59c700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac073910 0x7ff2ac19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2aad9d700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac10d560 0x7ff2ac1a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 -- 172.21.15.26:0/869157866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac073910 msgr2=0x7ff2ac19d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac073910 0x7ff2ac19d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 -- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac10d560 msgr2=0x7ff2ac1a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac10d560 0x7ff2ac1a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 -- 172.21.15.26:0/869157866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009520 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.795+0000 7ff2a9d9b700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0004710 tx=0x7ff2a0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.796+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a001d070 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.796+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a0021410 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.796+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0017650 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.796+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a00095f0 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.796+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac1a5620 con 0x7ff2ac074ee0 2023-10-06T21:04:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.797+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac1030b0 con 0x7ff2ac074ee0 2023-10-06T21:04:04.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.799+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff2a0021580 con 0x7ff2ac074ee0 2023-10-06T21:04:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.801+0000 7ff29b7fe700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 0x7ff2940a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.801+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7ff2a00cd2a0 con 0x7ff2ac074ee0 2023-10-06T21:04:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.801+0000 7ff2aa59c700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 0x7ff2940a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.802+0000 7ff2aa59c700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 0x7ff2940a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0752c0 tx=0x7ff29c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:04.803+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff2a0097da0 con 0x7ff2ac074ee0 2023-10-06T21:04:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:05.025+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2ac02d260 con 0x7ff2940a1470 2023-10-06T21:04:05.323 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122729]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:05.323 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:05.323 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122729]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:05.324 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122729]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:05.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:05.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:05.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:05.823 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122819]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:05.823 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:05.823 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122819]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:05.824 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:04:05 smithi026 sudo[122819]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: osdmap e20: 3 total, 3 up, 3 in 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:06.100 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:05 smithi078 sudo[144856]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:06.100 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:05 smithi078 sudo[144856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:06.100 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:05 smithi078 sudo[144856]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:06.100 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:05 smithi078 sudo[144856]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:06.365 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:06 smithi078 sudo[144859]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:06.365 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:06 smithi078 sudo[144859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:06.365 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:06 smithi078 sudo[144859]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:06.365 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:04:06 smithi078 sudo[144859]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:06.385 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:06 smithi026 ceph-mon[98490]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:06.385 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:06 smithi026 ceph-mon[98490]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:04:06.385 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:06 smithi078 ceph-mon[134896]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:06 smithi078 ceph-mon[134896]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:04:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:06 smithi167 ceph-mon[134494]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:06.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:06 smithi167 ceph-mon[134494]: osdmap e21: 3 total, 3 up, 3 in 2023-10-06T21:04:06.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:06.625 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145168]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:06.625 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145168]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:06.625 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145168]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:06.625 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145168]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:07.032 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145171]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:07.032 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145171]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:07.033 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145171]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:07.033 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:04:06 smithi167 sudo[145171]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122959]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122959]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122959]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:07.770+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff2a0098170 con 0x7ff2ac074ee0 2023-10-06T21:04:08.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122977]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:08.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:08.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122977]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:08.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 sudo[122977]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:08.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 ceph-mon[98490]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:08.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:08.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:07 smithi078 ceph-mon[134896]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:08.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:07 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:08.254 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145174]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145174]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145174]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145177]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:08.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145177]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 sudo[145177]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:08.748 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144862]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-06T21:04:08.749 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:08.749 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144862]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:08.749 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144862]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: Cluster is now healthy 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/868062716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]': finished 2023-10-06T21:04:09.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T21:04:09.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:09.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:09.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:08 smithi167 ceph-mon[134494]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: Cluster is now healthy 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:04:09.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/868062716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]': finished 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:09.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:08 smithi026 ceph-mon[98490]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144865]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144865]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 sudo[144865]: pam_unix(sudo:session): session closed for user root 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-06T21:04:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: Cluster is now healthy 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/868062716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]: dispatch 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d5d2ee-19a4-407b-bec4-92dd3e7c684a"}]': finished 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: osdmap e22: 4 total, 3 up, 4 in 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:09.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:08 smithi078 ceph-mon[134896]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-06T21:04:10.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/311662333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:10.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/311662333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:10.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/311662333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:11.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:11 smithi167 ceph-mon[134494]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:11.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:11 smithi026 ceph-mon[98490]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:11.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:11 smithi078 ceph-mon[134896]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:13.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:13 smithi026 ceph-mon[98490]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:13.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:13 smithi078 ceph-mon[134896]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:13.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:13 smithi167 ceph-mon[134494]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:15 smithi167 ceph-mon[134494]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-06T21:04:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:15 smithi026 ceph-mon[98490]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-06T21:04:15.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:15 smithi078 ceph-mon[134896]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 43 KiB/s wr, 4 op/s 2023-10-06T21:04:17.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:17 smithi026 ceph-mon[98490]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:04:17.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:04:17.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:17 smithi167 ceph-mon[134494]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:04:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:04:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:17.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:17 smithi078 ceph-mon[134896]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-10-06T21:04:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-06T21:04:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:18 smithi167 ceph-mon[134494]: Deploying daemon osd.3 on smithi026 2023-10-06T21:04:18.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:18 smithi026 ceph-mon[98490]: Deploying daemon osd.3 on smithi026 2023-10-06T21:04:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:18 smithi078 ceph-mon[134896]: Deploying daemon osd.3 on smithi026 2023-10-06T21:04:19.531 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:19 smithi026 ceph-mon[98490]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1018 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:04:19.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:19 smithi167 ceph-mon[134494]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1018 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:04:19.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:19 smithi078 ceph-mon[134896]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1018 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-06T21:04:21.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:21 smithi026 ceph-mon[98490]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:21.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:21 smithi167 ceph-mon[134494]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:21.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:21 smithi078 ceph-mon[134896]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:23.773 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:23 smithi026 ceph-mon[98490]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:23.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:23.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:23 smithi167 ceph-mon[134494]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:23.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:23.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:23 smithi078 ceph-mon[134896]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:23.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:23.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:25.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:25.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-10-06T21:04:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.343+0000 7ff29b7fe700 1 -- 172.21.15.26:0/869157866 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2ac02d260 con 0x7ff2940a1470 2023-10-06T21:04:26.346 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2023-10-06T21:04:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 msgr2=0x7ff2940a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 0x7ff2940a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0752c0 tx=0x7ff29c009260 comp rx=0 tx=0).stop 2023-10-06T21:04:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 msgr2=0x7ff2ac19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0004710 tx=0x7ff2a0009580 comp rx=0 tx=0).stop 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 shutdown_connections 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff2940a1470 0x7ff2940a3930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac10d560 0x7ff2ac1a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2ac073910 0x7ff2ac19d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 --2- 172.21.15.26:0/869157866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2ac074ee0 0x7ff2ac19d850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 >> 172.21.15.26:0/869157866 conn(0x7ff2ac0fc520 msgr2=0x7ff2ac0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.345+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 shutdown_connections 2023-10-06T21:04:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:04:26.346+0000 7ff2b0f43700 1 -- 172.21.15.26:0/869157866 wait complete. 2023-10-06T21:04:26.902 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-06T21:04:26.906 INFO:tasks.cephadm:Deploying osd.4 on smithi078 with /dev/vg_nvme/lv_3... 2023-10-06T21:04:26.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:04:26.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:04:27.475 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:04:27.556 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.556 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.556 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:27 smithi026 ceph-mon[98490]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:27.556 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:27.556 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:27 smithi026 ceph-mon[98490]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:04:27.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:27 smithi078 ceph-mon[134896]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:27 smithi078 ceph-mon[134896]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:04:27.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:27.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:27 smithi167 ceph-mon[134494]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:27.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:27.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:27 smithi167 ceph-mon[134494]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-06T21:04:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-mon[98490]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:04:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-mon[98490]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:04:28.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-mon[98490]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:04:28.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:28.574 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3[125269]: 2023-10-06T21:04:28.318+0000 7f88e6bd4700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T21:04:28.574 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:04:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3[125269]: 2023-10-06T21:04:28.327+0000 7f88e31ca700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:04:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:28 smithi078 ceph-mon[134896]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:04:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:28 smithi078 ceph-mon[134896]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:04:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:28 smithi078 ceph-mon[134896]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:04:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:28.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:28 smithi167 ceph-mon[134494]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-06T21:04:28.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:28 smithi167 ceph-mon[134494]: osdmap e23: 4 total, 3 up, 4 in 2023-10-06T21:04:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:28 smithi167 ceph-mon[134494]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:04:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:28.986 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:04:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:04:28.989 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2023-10-06T21:04:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2023-10-06T21:04:28.989 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116896 s, 897 MB/s 2023-10-06T21:04:28.989 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2023-10-06T21:04:29.334 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2023-10-06T21:04:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:29 smithi078 ceph-mon[134896]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:04:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:29 smithi078 ceph-mon[134896]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:04:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:29 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:29 smithi078 ceph-mon[134896]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:29 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:29 smithi167 ceph-mon[134494]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:04:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:29 smithi167 ceph-mon[134494]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:04:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:29 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:29 smithi167 ceph-mon[134494]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:29 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:29 smithi026 ceph-mon[98490]: from='osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:04:29.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:29 smithi026 ceph-mon[98490]: osdmap e24: 4 total, 3 up, 4 in 2023-10-06T21:04:29.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:29 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:29 smithi026 ceph-mon[98490]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:29.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:29 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:29.893 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:04:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:04:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:04:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] boot 2023-10-06T21:04:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T21:04:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:30 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:04:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:04:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] boot 2023-10-06T21:04:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T21:04:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:30 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:30.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:04:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:04:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: osd.3 [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] boot 2023-10-06T21:04:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: osdmap e25: 4 total, 4 up, 4 in 2023-10-06T21:04:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:30 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:04:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 -- 172.21.15.78:0/3274565412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b880ffe70 msgr2=0x7f1b88100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 --2- 172.21.15.78:0/3274565412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b880ffe70 0x7f1b88100290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009990 tx=0x7f1b7c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 -- 172.21.15.78:0/3274565412 shutdown_connections 2023-10-06T21:04:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 --2- 172.21.15.78:0/3274565412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b88101990 0x7f1b881082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 --2- 172.21.15.78:0/3274565412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b880ffe70 0x7f1b88100290 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 --2- 172.21.15.78:0/3274565412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88100fd0 0x7f1b88101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 -- 172.21.15.78:0/3274565412 >> 172.21.15.78:0/3274565412 conn(0x7f1b880fb3f0 msgr2=0x7f1b880fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 -- 172.21.15.78:0/3274565412 shutdown_connections 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.118+0000 7f1b8f553700 1 -- 172.21.15.78:0/3274565412 wait complete. 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 Processor -- start 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 -- start start 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880ffe70 0x7f1b88196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88101990 0x7f1b8819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8819e8e0 con 0x7f1b880ffe70 2023-10-06T21:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8819ea20 con 0x7f1b88100fd0 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8f553700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8819ebb0 con 0x7f1b88101990 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40366/0 (socket says 172.21.15.78:40366) 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 -- 172.21.15.78:0/200331395 learned_addr learned my addr 172.21.15.78:0/200331395 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8daf0700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88101990 0x7f1b8819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 -- 172.21.15.78:0/200331395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88101990 msgr2=0x7f1b8819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88101990 0x7f1b8819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 -- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880ffe70 msgr2=0x7f1b88196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880ffe70 0x7f1b88196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.119+0000 7f1b8caee700 1 -- 172.21.15.78:0/200331395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0095f0 con 0x7f1b88100fd0 2023-10-06T21:04:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b8caee700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400b530 tx=0x7f1b8400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84011830 con 0x7f1b88100fd0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b84011e50 con 0x7f1b88100fd0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8819ee90 con 0x7f1b88100fd0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400f2e0 con 0x7f1b88100fd0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.120+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8819f3d0 con 0x7f1b88100fd0 2023-10-06T21:04:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.121+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8810b780 con 0x7f1b88100fd0 2023-10-06T21:04:31.127 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.122+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1b84010380 con 0x7f1b88100fd0 2023-10-06T21:04:31.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.122+0000 7f1b7a7fc700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 0x7f1b740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:31.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.122+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f1b840cb5e0 con 0x7f1b88100fd0 2023-10-06T21:04:31.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.122+0000 7f1b8d2ef700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 0x7f1b740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:31.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.123+0000 7f1b8d2ef700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 0x7f1b740a39c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b5c0 tx=0x7f1b7c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:31.128 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.127+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1b84095e40 con 0x7f1b88100fd0 2023-10-06T21:04:31.325 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:31.324+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b880690d0 con 0x7f1b740a1500 2023-10-06T21:04:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:04:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi026 2023-10-06T21:04:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:31.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:31 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi026 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:31 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: osdmap e26: 4 total, 4 up, 4 in 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi026 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:31 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 8636M 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:32.441 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 8636M 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:32.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 8636M 2023-10-06T21:04:32.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:32.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:32.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: osdmap e27: 4 total, 4 up, 4 in 2023-10-06T21:04:32.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:32.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:33.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:33 smithi078 ceph-mon[134896]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:33.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:33 smithi078 ceph-mon[134896]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:04:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:33 smithi167 ceph-mon[134494]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:33.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:33 smithi167 ceph-mon[134494]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:04:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:33 smithi026 ceph-mon[98490]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:33 smithi026 ceph-mon[98490]: osdmap e28: 4 total, 4 up, 4 in 2023-10-06T21:04:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:04:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/3324124392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]': finished 2023-10-06T21:04:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T21:04:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:34 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:34.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:04:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/3324124392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]': finished 2023-10-06T21:04:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T21:04:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:34 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: osdmap e29: 4 total, 4 up, 4 in 2023-10-06T21:04:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/3324124392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]: dispatch 2023-10-06T21:04:34.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22b765a7-0ed8-452b-a897-6b5a7c720e59"}]': finished 2023-10-06T21:04:34.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: osdmap e30: 5 total, 4 up, 5 in 2023-10-06T21:04:34.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:34 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:35.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:35 smithi078 ceph-mon[134896]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:04:35.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:35 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/1924552075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:35.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:35 smithi167 ceph-mon[134494]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:04:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:35 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/1924552075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:35 smithi026 ceph-mon[98490]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:04:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:35 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/1924552075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:37.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:37 smithi078 ceph-mon[134896]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-06T21:04:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:37 smithi167 ceph-mon[134494]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-06T21:04:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:37 smithi026 ceph-mon[98490]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-10-06T21:04:39.032 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:38 smithi078 ceph-mon[134896]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:04:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:38 smithi167 ceph-mon[134494]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:04:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:38 smithi026 ceph-mon[98490]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:04:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:41 smithi078 ceph-mon[134896]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:04:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:04:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:41 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:41.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:41 smithi167 ceph-mon[134494]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:04:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:41 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:04:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:41 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:41 smithi026 ceph-mon[98490]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:04:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:41 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-06T21:04:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:41 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:42.532 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:42 smithi078 ceph-mon[134896]: Deploying daemon osd.4 on smithi078 2023-10-06T21:04:42.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:42 smithi167 ceph-mon[134494]: Deploying daemon osd.4 on smithi078 2023-10-06T21:04:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:42 smithi026 ceph-mon[98490]: Deploying daemon osd.4 on smithi078 2023-10-06T21:04:43.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:43 smithi078 ceph-mon[134896]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:43.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:43 smithi167 ceph-mon[134494]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:43 smithi026 ceph-mon[98490]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:45 smithi167 ceph-mon[134494]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:45.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:45 smithi026 ceph-mon[98490]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:45 smithi078 ceph-mon[134896]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:47.768 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:47.769 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.769 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.769 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:47.769 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:47.769 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:47.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:47.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:47.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.792 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 4 on host 'smithi078' 2023-10-06T21:04:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.789+0000 7f1b7a7fc700 1 -- 172.21.15.78:0/200331395 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b880690d0 con 0x7f1b740a1500 2023-10-06T21:04:47.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.791+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 msgr2=0x7f1b740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:47.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.791+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 0x7f1b740a39c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b5c0 tx=0x7f1b7c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:04:47.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.791+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 msgr2=0x7f1b88197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:47.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.791+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400b530 tx=0x7f1b8400b840 comp rx=0 tx=0).stop 2023-10-06T21:04:47.794 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 shutdown_connections 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b88101990 0x7f1b8819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1b740a1500 0x7f1b740a39c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880ffe70 0x7f1b88196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 --2- 172.21.15.78:0/200331395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b88100fd0 0x7f1b88197290 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 >> 172.21.15.78:0/200331395 conn(0x7f1b880fb3f0 msgr2=0x7f1b880689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 shutdown_connections 2023-10-06T21:04:47.795 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:04:47.792+0000 7f1b8f553700 1 -- 172.21.15.78:0/200331395 wait complete. 2023-10-06T21:04:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:48.180 DEBUG:teuthology.orchestra.run.smithi078:osd.4> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-06T21:04:48.183 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2023-10-06T21:04:48.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-06T21:04:48.212 INFO:journalctl@ceph.osd.4.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:04:48.757 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:04:49.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:48 smithi167 ceph-mon[134494]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:49.071 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.071 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.071 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:49.071 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:48 smithi078 ceph-mon[134896]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:49.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:04:49.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:48 smithi026 ceph-mon[98490]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:49.370 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:04:49 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:04:49.066+0000 7fda907d9640 -1 osd.4 0 log_to_monitors true 2023-10-06T21:04:50.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:49 smithi167 ceph-mon[134494]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:49 smithi167 ceph-mon[134494]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:49 smithi026 ceph-mon[98490]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:49 smithi026 ceph-mon[98490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:49 smithi078 ceph-mon[134896]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:49 smithi078 ceph-mon[134896]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-06T21:04:50.339 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-06T21:04:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-06T21:04:50.340 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-10-06T21:04:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-10-06T21:04:50.340 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127491 s, 822 MB/s 2023-10-06T21:04:50.340 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-10-06T21:04:50.715 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2023-10-06T21:04:50.800 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:04:50.800 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:04:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:04:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:50 smithi167 ceph-mon[134494]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-06T21:04:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: from='osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:04:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:51.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:50 smithi026 ceph-mon[98490]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: osdmap e31: 5 total, 4 up, 5 in 2023-10-06T21:04:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:04:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-mon[134896]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:51.121 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:04:50.801+0000 7fda8564f700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T21:04:51.121 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:04:50 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:04:50.807+0000 7fda7dc3d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:04:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:04:52.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi078 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 8584M 2023-10-06T21:04:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:52.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:52.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:04:52.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:04:52.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi078 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 8584M 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:52.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:52.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: osdmap e32: 5 total, 4 up, 5 in 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi078 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 8584M 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:04:52.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:04:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 -- 172.21.15.167:0/4008677758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 msgr2=0x7f43b4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/4008677758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b4101450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009990 tx=0x7f43b0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 -- 172.21.15.167:0/4008677758 shutdown_connections 2023-10-06T21:04:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/4008677758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 0x7f43b41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/4008677758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 0x7f43b4100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/4008677758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b4101450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 -- 172.21.15.167:0/4008677758 >> 172.21.15.167:0/4008677758 conn(0x7f43b40fb3f0 msgr2=0x7f43b40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 -- 172.21.15.167:0/4008677758 shutdown_connections 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.339+0000 7f43bbe2e700 1 -- 172.21.15.167:0/4008677758 wait complete. 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 Processor -- start 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 -- start start 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 0x7f43b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 0x7f43b419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419e7d0 con 0x7f43b40ffe70 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419e940 con 0x7f43b4101990 2023-10-06T21:04:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43bbe2e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b419eb00 con 0x7f43b4100fd0 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43b93c9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43b93c9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59116/0 (socket says 172.21.15.167:59116) 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43b93c9700 1 -- 172.21.15.167:0/1905316538 learned_addr learned my addr 172.21.15.167:0/1905316538 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43ba3cb700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 0x7f43b419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.340+0000 7f43b9bca700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 0x7f43b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 -- 172.21.15.167:0/1905316538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 msgr2=0x7f43b419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 0x7f43b419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 -- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 msgr2=0x7f43b4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 0x7f43b4069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 -- 172.21.15.167:0/1905316538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00095f0 con 0x7f43b4100fd0 2023-10-06T21:04:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43b93c9700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f43b0004730 tx=0x7f43b0004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b001d070 con 0x7f43b4100fd0 2023-10-06T21:04:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43b0004c90 con 0x7f43b4100fd0 2023-10-06T21:04:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b419ed80 con 0x7f43b4100fd0 2023-10-06T21:04:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0005570 con 0x7f43b4100fd0 2023-10-06T21:04:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.341+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b419f190 con 0x7f43b4100fd0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.342+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b410b760 con 0x7f43b4100fd0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.343+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f43b000fc70 con 0x7f43b4100fd0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.343+0000 7f43aaffd700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 0x7f43a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.344+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f43b00cd060 con 0x7f43b4100fd0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.344+0000 7f43b9bca700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 0x7f43a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.344+0000 7f43b9bca700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 0x7f43a00a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006fd0 tx=0x7f43a4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:04:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.348+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f43b005b100 con 0x7f43b4100fd0 2023-10-06T21:04:52.547 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:04:52.544+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f43b40612f0 con 0x7f43a00a1480 2023-10-06T21:04:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:04:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:04:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] boot 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:53.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:52 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] boot 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:52 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:53.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: osd.4 [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] boot 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: osdmap e33: 5 total, 5 up, 5 in 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:04:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:52 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:04:54.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:53 smithi167 ceph-mon[134494]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:54.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:53 smithi167 ceph-mon[134494]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:04:54.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:53 smithi026 ceph-mon[98490]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:54.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:53 smithi026 ceph-mon[98490]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:04:54.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:53 smithi078 ceph-mon[134896]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:04:54.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:53 smithi078 ceph-mon[134896]: osdmap e34: 5 total, 5 up, 5 in 2023-10-06T21:04:55.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:54 smithi026 ceph-mon[98490]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:55.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:54 smithi078 ceph-mon[134896]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:54 smithi167 ceph-mon[134494]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/2268640741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]': finished 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:04:56.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:55 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/397307710' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:56.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/2268640741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]': finished 2023-10-06T21:04:56.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T21:04:56.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:04:56.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:55 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/397307710' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:56.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/2268640741' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]: dispatch 2023-10-06T21:04:56.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c5d6f6e-9881-41e0-859d-2e8702b70d22"}]': finished 2023-10-06T21:04:56.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: osdmap e35: 6 total, 5 up, 6 in 2023-10-06T21:04:56.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:04:56.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:55 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/397307710' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:04:57.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:56 smithi078 ceph-mon[134896]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:57.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:56 smithi167 ceph-mon[134494]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:57.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:56 smithi026 ceph-mon[98490]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:59.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:04:58 smithi167 ceph-mon[134494]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:04:58 smithi026 ceph-mon[98490]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:04:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:04:58 smithi078 ceph-mon[134896]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:01.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:01 smithi167 ceph-mon[134494]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:01.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:01 smithi026 ceph-mon[98490]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:01.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:01 smithi078 ceph-mon[134896]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:05:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:02 smithi167 ceph-mon[134494]: Deploying daemon osd.5 on smithi167 2023-10-06T21:05:02.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:05:02.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:02.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:02 smithi026 ceph-mon[98490]: Deploying daemon osd.5 on smithi167 2023-10-06T21:05:02.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-06T21:05:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:02 smithi078 ceph-mon[134896]: Deploying daemon osd.5 on smithi167 2023-10-06T21:05:03.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:03 smithi167 ceph-mon[134494]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:03.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:03 smithi026 ceph-mon[98490]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:03.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:03 smithi078 ceph-mon[134896]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:05.645 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:05 smithi167 ceph-mon[134494]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:05.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:05 smithi026 ceph-mon[98490]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:05.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:05 smithi078 ceph-mon[134896]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:07.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:07.003 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:06 smithi167 ceph-mon[134494]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:06 smithi078 ceph-mon[134896]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:06 smithi026 ceph-mon[98490]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:08.414 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2023-10-06T21:05:08.414 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.410+0000 7f43aaffd700 1 -- 172.21.15.167:0/1905316538 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f43b40612f0 con 0x7f43a00a1480 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.412+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 msgr2=0x7f43a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.412+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 0x7f43a00a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006fd0 tx=0x7f43a4008040 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 msgr2=0x7f43b419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f43b0004730 tx=0x7f43b0004760 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 shutdown_connections 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43b4101990 0x7f43b419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f43a00a1480 0x7f43a00a3940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43b40ffe70 0x7f43b4069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 --2- 172.21.15.167:0/1905316538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b4100fd0 0x7f43b419b9e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 >> 172.21.15.167:0/1905316538 conn(0x7f43b40fb3f0 msgr2=0x7f43b40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:05:08.416 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 shutdown_connections 2023-10-06T21:05:08.417 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:05:08.413+0000 7f43bbe2e700 1 -- 172.21.15.167:0/1905316538 wait complete. 2023-10-06T21:05:08.857 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-06T21:05:08.859 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2023-10-06T21:05:08.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:05:08.903 INFO:journalctl@ceph.osd.5.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:05:09.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:09.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:09.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:05:10.033 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:05:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:05:09.799+0000 7f6c31e4d640 -1 osd.5 0 log_to_monitors true 2023-10-06T21:05:10.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:10 smithi026 ceph-mon[98490]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:10.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:10 smithi026 ceph-mon[98490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:10.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:10 smithi078 ceph-mon[134896]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:10.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:10 smithi078 ceph-mon[134896]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:10.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:10 smithi167 ceph-mon[134494]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:10.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:10 smithi167 ceph-mon[134494]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-06T21:05:11.242 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:05:11.125+0000 7f6c26cc3700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T21:05:11.242 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:05:11.133+0000 7f6c22ab8700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.243 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:11 smithi167 ceph-mon[134494]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:11.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:05:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:05:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:11 smithi078 ceph-mon[134896]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:11.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-06T21:05:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: osdmap e36: 6 total, 5 up, 6 in 2023-10-06T21:05:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: from='osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:05:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:11 smithi026 ceph-mon[98490]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0317366 s, 330 MB/s 2023-10-06T21:05:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-10-06T21:05:12.110 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2023-10-06T21:05:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:05:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:05:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi167 2023-10-06T21:05:12.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 8584M 2023-10-06T21:05:12.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:12.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:12.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:12 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi167 2023-10-06T21:05:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 8584M 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:12.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:12 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:05:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: osdmap e37: 6 total, 5 up, 6 in 2023-10-06T21:05:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi167 2023-10-06T21:05:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 8584M 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:12 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:05:13.416 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:05:13.416 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:05:13.416 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] boot 2023-10-06T21:05:13.416 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T21:05:13.417 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:13.417 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:13.417 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:13 smithi026 ceph-mon[98490]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:05:13.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] boot 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:13.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:13 smithi167 ceph-mon[134494]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:05:13.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: osd.5 [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] boot 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: osdmap e38: 6 total, 6 up, 6 in 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:13 smithi078 ceph-mon[134896]: osdmap e39: 6 total, 6 up, 6 in 2023-10-06T21:05:14.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 -- 172.21.15.26:0/3142481806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 msgr2=0x7f4124101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:14.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 --2- 172.21.15.26:0/3142481806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124101390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f41140099c0 tx=0x7f4114009cd0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 -- 172.21.15.26:0/3142481806 shutdown_connections 2023-10-06T21:05:14.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 --2- 172.21.15.26:0/3142481806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102b10 0x7f41241093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 --2- 172.21.15.26:0/3142481806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124101390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 --2- 172.21.15.26:0/3142481806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102170 0x7f41241025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.396+0000 7f4129469700 1 -- 172.21.15.26:0/3142481806 >> 172.21.15.26:0/3142481806 conn(0x7f41240fc4d0 msgr2=0x7f41240fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:05:14.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.397+0000 7f4129469700 1 -- 172.21.15.26:0/3142481806 shutdown_connections 2023-10-06T21:05:14.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.397+0000 7f4129469700 1 -- 172.21.15.26:0/3142481806 wait complete. 2023-10-06T21:05:14.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.398+0000 7f4129469700 1 Processor -- start 2023-10-06T21:05:14.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.398+0000 7f4129469700 1 -- start start 2023-10-06T21:05:14.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:14.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 0x7f4124197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102b10 0x7f412419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419e790 con 0x7f4124100f70 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419e8d0 con 0x7f4124102170 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4129469700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412419ea90 con 0x7f4124102b10 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56742/0 (socket says 172.21.15.26:56742) 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 -- 172.21.15.26:0/878569365 learned_addr learned my addr 172.21.15.26:0/878569365 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f41237fe700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102b10 0x7f412419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:14.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f41227fc700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 0x7f4124197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:14.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 -- 172.21.15.26:0/878569365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102b10 msgr2=0x7f412419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:14.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102b10 0x7f412419e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 -- 172.21.15.26:0/878569365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 msgr2=0x7f4124197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:14.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 0x7f4124197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:14.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.399+0000 7f4122ffd700 1 -- 172.21.15.26:0/878569365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140095f0 con 0x7f4124100f70 2023-10-06T21:05:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f41227fc700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 0x7f4124197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:05:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f4122ffd700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4114009fd0 tx=0x7f4114009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:05:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411401d070 con 0x7f4124100f70 2023-10-06T21:05:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4114005030 con 0x7f4124100f70 2023-10-06T21:05:14.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412419ed10 con 0x7f4124100f70 2023-10-06T21:05:14.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114004b30 con 0x7f4124100f70 2023-10-06T21:05:14.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.400+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412419f180 con 0x7f4124100f70 2023-10-06T21:05:14.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.401+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4124108550 con 0x7f4124100f70 2023-10-06T21:05:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.403+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f41140051a0 con 0x7f4124100f70 2023-10-06T21:05:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.404+0000 7f410bfff700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 0x7f410c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.405+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f41140ccc70 con 0x7f4124100f70 2023-10-06T21:05:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.405+0000 7f41227fc700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 0x7f410c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:14.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.405+0000 7f41227fc700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 0x7f410c0a39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4118009cb0 tx=0x7f4118009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:05:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.409+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4114097160 con 0x7f4124100f70 2023-10-06T21:05:14.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:14.626+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f412410c850 con 0x7f410c0a1500 2023-10-06T21:05:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:14 smithi026 ceph-mon[98490]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:05:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:14 smithi026 ceph-mon[98490]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:14 smithi078 ceph-mon[134896]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:05:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:14 smithi078 ceph-mon[134896]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:15.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:14 smithi167 ceph-mon[134494]: osdmap e40: 6 total, 6 up, 6 in 2023-10-06T21:05:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:14 smithi167 ceph-mon[134494]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:16.041 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:15 smithi026 ceph-mon[98490]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:16.041 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:15 smithi026 ceph-mon[98490]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:05:16.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:15 smithi078 ceph-mon[134896]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:16.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:15 smithi078 ceph-mon[134896]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:05:16.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:15 smithi167 ceph-mon[134494]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:16.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:15 smithi167 ceph-mon[134494]: osdmap e41: 6 total, 6 up, 6 in 2023-10-06T21:05:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:16 smithi026 ceph-mon[98490]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:05:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:16 smithi026 ceph-mon[98490]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:16 smithi078 ceph-mon[134896]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:05:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:16 smithi078 ceph-mon[134896]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:17.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:16 smithi167 ceph-mon[134494]: osdmap e42: 6 total, 6 up, 6 in 2023-10-06T21:05:17.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:16 smithi167 ceph-mon[134494]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:18.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:17 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]: dispatch 2023-10-06T21:05:18.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:17 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]': finished 2023-10-06T21:05:18.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:17 smithi026 ceph-mon[98490]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T21:05:18.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:17 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]: dispatch 2023-10-06T21:05:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:17 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]': finished 2023-10-06T21:05:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:17 smithi078 ceph-mon[134896]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T21:05:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:18.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:17 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]: dispatch 2023-10-06T21:05:18.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:17 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1474743791' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba40219-6497-43c6-a864-79afb928c52a"}]': finished 2023-10-06T21:05:18.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:17 smithi167 ceph-mon[134494]: osdmap e43: 7 total, 6 up, 7 in 2023-10-06T21:05:18.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:18 smithi026 ceph-mon[98490]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-06T21:05:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:18 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1339024076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:18 smithi078 ceph-mon[134896]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-06T21:05:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:18 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1339024076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:19.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:18 smithi167 ceph-mon[134494]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-10-06T21:05:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:18 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1339024076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:21.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:21 smithi026 ceph-mon[98490]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:05:21.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:21 smithi167 ceph-mon[134494]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:05:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:21 smithi078 ceph-mon[134896]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:05:23.499 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:23 smithi026 ceph-mon[98490]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:05:23.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:23 smithi167 ceph-mon[134494]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:05:23.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:23 smithi078 ceph-mon[134896]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:05:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:25 smithi026 ceph-mon[98490]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:05:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:05:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:25.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:25 smithi167 ceph-mon[134494]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:05:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:05:25.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:25.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:25 smithi078 ceph-mon[134896]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:05:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-06T21:05:25.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:26.464 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:26 smithi026 ceph-mon[98490]: Deploying daemon osd.6 on smithi026 2023-10-06T21:05:26.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:26 smithi167 ceph-mon[134494]: Deploying daemon osd.6 on smithi026 2023-10-06T21:05:26.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:26 smithi078 ceph-mon[134896]: Deploying daemon osd.6 on smithi026 2023-10-06T21:05:27.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:27 smithi026 ceph-mon[98490]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:05:27.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:27 smithi167 ceph-mon[134494]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:05:27.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:27 smithi078 ceph-mon[134896]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:05:29.014 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:28 smithi026 ceph-mon[98490]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-06T21:05:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:28 smithi078 ceph-mon[134896]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-06T21:05:29.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:28 smithi167 ceph-mon[134494]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-06T21:05:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:31 smithi167 ceph-mon[134494]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:31.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:31 smithi026 ceph-mon[98490]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:31.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:31 smithi078 ceph-mon[134896]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:32.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.719 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:32 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:32.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:32 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:32.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:32.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:32 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:33 smithi167 ceph-mon[134494]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:33 smithi026 ceph-mon[98490]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:33.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:33 smithi078 ceph-mon[134896]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:35.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.039+0000 7f410bfff700 1 -- 172.21.15.26:0/878569365 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f412410c850 con 0x7f410c0a1500 2023-10-06T21:05:35.042 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2023-10-06T21:05:35.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 msgr2=0x7f410c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:35.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 0x7f410c0a39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4118009cb0 tx=0x7f4118009260 comp rx=0 tx=0).stop 2023-10-06T21:05:35.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 msgr2=0x7f4124196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:35.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4114009fd0 tx=0x7f4114009580 comp rx=0 tx=0).stop 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 shutdown_connections 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4124102b10 0x7f412419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f410c0a1500 0x7f410c0a39c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4124100f70 0x7f4124196c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 --2- 172.21.15.26:0/878569365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4124102170 0x7f4124197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.041+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 >> 172.21.15.26:0/878569365 conn(0x7f41240fc4d0 msgr2=0x7f4124105690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.042+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 shutdown_connections 2023-10-06T21:05:35.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:05:35.042+0000 7f4129469700 1 -- 172.21.15.26:0/878569365 wait complete. 2023-10-06T21:05:35.579 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-06T21:05:35.583 INFO:tasks.cephadm:Deploying osd.7 on smithi078 with /dev/vg_nvme/lv_2... 2023-10-06T21:05:35.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:05:35.658 INFO:journalctl@ceph.osd.6.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:05:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:35 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:35 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:35 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:36.166 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:05:36.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:36 smithi026 ceph-mon[98490]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:36.472 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:36 smithi026 ceph-mon[98490]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:36 smithi167 ceph-mon[134494]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:36.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:36 smithi167 ceph-mon[134494]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:36.790 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:36 smithi078 ceph-mon[134896]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:36.790 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:36 smithi078 ceph-mon[134896]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-06T21:05:37.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:37.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:05:37.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:05:37.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:37.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:37.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:37 smithi078 ceph-mon[134896]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:05:37.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:05:37.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:37.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:37.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:37 smithi167 ceph-mon[134494]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124022 s, 845 MB/s 2023-10-06T21:05:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2023-10-06T21:05:37.824 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:05:37.444+0000 7fa0f8336700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T21:05:37.824 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:05:37.456+0000 7fa0f1926700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: osdmap e44: 7 total, 6 up, 7 in 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: from='osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:37 smithi026 ceph-mon[98490]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:05:38.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:38.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:05:38.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:05:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:05:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:05:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:05:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: osdmap e45: 7 total, 6 up, 7 in 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:39.534 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:39.534 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] boot 2023-10-06T21:05:39.534 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:39.535 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:39 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] boot 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:39.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:39.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:39 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-06T21:05:39.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: osd.6 [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] boot 2023-10-06T21:05:39.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: osdmap e46: 7 total, 7 up, 7 in 2023-10-06T21:05:39.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:39.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:39 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 -- 172.21.15.78:0/3775864012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80101990 msgr2=0x7f2d801082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/3775864012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80101990 0x7f2d801082b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800b0a0 tx=0x7f2d7800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 -- 172.21.15.78:0/3775864012 shutdown_connections 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/3775864012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80101990 0x7f2d801082b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/3775864012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d800ffe70 0x7f2d80100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/3775864012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80100fd0 0x7f2d80101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 -- 172.21.15.78:0/3775864012 >> 172.21.15.78:0/3775864012 conn(0x7f2d800fb3f0 msgr2=0x7f2d800fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.991+0000 7f2d87c6f700 1 -- 172.21.15.78:0/3775864012 shutdown_connections 2023-10-06T21:05:39.995 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 -- 172.21.15.78:0/3775864012 wait complete. 2023-10-06T21:05:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 Processor -- start 2023-10-06T21:05:39.996 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 -- start start 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d800ffe70 0x7f2d80069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 0x7f2d8019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019e8d0 con 0x7f2d80101990 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019ea40 con 0x7f2d80100fd0 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.992+0000 7f2d87c6f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019ec00 con 0x7f2d800ffe70 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:39.998 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40478/0 (socket says 172.21.15.78:40478) 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 -- 172.21.15.78:0/707530410 learned_addr learned my addr 172.21.15.78:0/707530410 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 -- 172.21.15.78:0/707530410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d800ffe70 msgr2=0x7f2d80069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d85a0b700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d800ffe70 0x7f2d80069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d800ffe70 0x7f2d80069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8620c700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 0x7f2d8019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 -- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 msgr2=0x7f2d8019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 0x7f2d8019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 -- 172.21.15.78:0/707530410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009d00 con 0x7f2d80100fd0 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8620c700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 0x7f2d8019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d8520a700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00d950 tx=0x7f2d7c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c009b40 con 0x7f2d80100fd0 2023-10-06T21:05:39.999 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7c009ca0 con 0x7f2d80100fd0 2023-10-06T21:05:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8019eee0 con 0x7f2d80100fd0 2023-10-06T21:05:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00f620 con 0x7f2d80100fd0 2023-10-06T21:05:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.993+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8019f350 con 0x7f2d80100fd0 2023-10-06T21:05:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.994+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80107400 con 0x7f2d80100fd0 2023-10-06T21:05:40.000 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.995+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2d7c010460 con 0x7f2d80100fd0 2023-10-06T21:05:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.996+0000 7f2d72ffd700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 0x7f2d6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:05:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.996+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f2d7c0cc370 con 0x7f2d80100fd0 2023-10-06T21:05:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.996+0000 7f2d85a0b700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 0x7f2d6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:05:40.001 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.997+0000 7f2d85a0b700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 0x7f2d6c0a39c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005950 tx=0x7f2d74005cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:05:40.002 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:39.999+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d7c096640 con 0x7f2d80100fd0 2023-10-06T21:05:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:40.197+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d8002d2d0 con 0x7f2d6c0a1500 2023-10-06T21:05:40.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi026 2023-10-06T21:05:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 5757M 2023-10-06T21:05:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:05:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi026 2023-10-06T21:05:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 5757M 2023-10-06T21:05:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:05:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:40.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi026 2023-10-06T21:05:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 5757M 2023-10-06T21:05:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: osdmap e47: 7 total, 7 up, 7 in 2023-10-06T21:05:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:05:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:05:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:41.575 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:41 smithi078 ceph-mon[134896]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:41.575 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:41 smithi078 ceph-mon[134896]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:41.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:41 smithi167 ceph-mon[134494]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:41.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:41 smithi167 ceph-mon[134494]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:41 smithi026 ceph-mon[98490]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:05:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:41 smithi026 ceph-mon[98490]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:43.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/1303848992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]': finished 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:05:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:43 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/3142025833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/1303848992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]': finished 2023-10-06T21:05:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T21:05:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:05:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:43 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/3142025833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/1303848992' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]: dispatch 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28a181f7-1d70-4d23-a95c-3fd0309f720d"}]': finished 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: osdmap e48: 8 total, 7 up, 8 in 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:05:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:43 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/3142025833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:05:45.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:45 smithi167 ceph-mon[134494]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:45 smithi026 ceph-mon[98490]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:45.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:45 smithi078 ceph-mon[134896]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:46.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:46 smithi167 ceph-mon[134494]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:46 smithi026 ceph-mon[98490]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:46.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:46 smithi078 ceph-mon[134896]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:48 smithi078 ceph-mon[134896]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:48 smithi026 ceph-mon[98490]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:49.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:48 smithi167 ceph-mon[134494]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:05:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:49 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:49 smithi026 ceph-mon[98490]: Deploying daemon osd.7 on smithi078 2023-10-06T21:05:50.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:05:50.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:49 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:50.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:49 smithi078 ceph-mon[134896]: Deploying daemon osd.7 on smithi078 2023-10-06T21:05:50.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-06T21:05:50.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:49 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:50.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:49 smithi167 ceph-mon[134494]: Deploying daemon osd.7 on smithi078 2023-10-06T21:05:50.853 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:50 smithi078 ceph-mon[134896]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:50 smithi026 ceph-mon[98490]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:51.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:50 smithi167 ceph-mon[134494]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:53.673 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:53 smithi078 ceph-mon[134896]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:53 smithi167 ceph-mon[134494]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:53 smithi026 ceph-mon[98490]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:54.988 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:54.988 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:54.988 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:54.988 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:54 smithi078 ceph-mon[134896]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:55.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:55.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:55.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:55.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:54 smithi026 ceph-mon[98490]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:55.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:54 smithi167 ceph-mon[134494]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:56.464 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 7 on host 'smithi078' 2023-10-06T21:05:56.464 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.461+0000 7f2d72ffd700 1 -- 172.21.15.78:0/707530410 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2d8002d2d0 con 0x7f2d6c0a1500 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 msgr2=0x7f2d6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 0x7f2d6c0a39c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005950 tx=0x7f2d74005cf0 comp rx=0 tx=0).stop 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 msgr2=0x7f2d8019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00d950 tx=0x7f2d7c00dd10 comp rx=0 tx=0).stop 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 shutdown_connections 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2d6c0a1500 0x7f2d6c0a39c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d80101990 0x7f2d8019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d800ffe70 0x7f2d80069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:56.466 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 --2- 172.21.15.78:0/707530410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d80100fd0 0x7f2d8019baf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:05:56.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 >> 172.21.15.78:0/707530410 conn(0x7f2d800fb3f0 msgr2=0x7f2d80104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:05:56.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 shutdown_connections 2023-10-06T21:05:56.467 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:05:56.463+0000 7f2d87c6f700 1 -- 172.21.15.78:0/707530410 wait complete. 2023-10-06T21:05:56.871 DEBUG:teuthology.orchestra.run.smithi078:osd.7> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-06T21:05:56.873 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2023-10-06T21:05:56.874 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-06T21:05:56.905 INFO:journalctl@ceph.osd.7.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:05:57.444 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:05:57.560 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.561 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:57.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:57.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:57.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:05:57.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:05:58.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:05:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:05:57.807+0000 7f163159b640 -1 osd.7 0 log_to_monitors true 2023-10-06T21:05:58.779 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-06T21:05:58.780 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-06T21:05:58.780 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-10-06T21:05:58.780 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-10-06T21:05:58.780 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126402 s, 830 MB/s 2023-10-06T21:05:58.780 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-10-06T21:05:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:58 smithi026 ceph-mon[98490]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:58 smithi026 ceph-mon[98490]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:58 smithi026 ceph-mon[98490]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:59.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:58 smithi167 ceph-mon[134494]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:58 smithi167 ceph-mon[134494]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:58 smithi167 ceph-mon[134494]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:59.087 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2023-10-06T21:05:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:58 smithi078 ceph-mon[134896]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:58 smithi078 ceph-mon[134896]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-06T21:05:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:58 smithi078 ceph-mon[134896]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:05:59.648 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:06:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:06:00.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:00.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:00.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:05:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:06:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:06:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:05:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-06T21:06:00.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: osdmap e49: 8 total, 7 up, 8 in 2023-10-06T21:06:00.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:00.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:00.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:00.125 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:05:59.799+0000 7f1626411700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T21:06:00.125 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:05:59 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:05:59.810+0000 7f161e9ff700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:06:00.708 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 -- 172.21.15.167:0/3748761301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e901021a0 msgr2=0x7f3e90102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:00.708 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 --2- 172.21.15.167:0/3748761301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e901021a0 0x7f3e90102600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009990 tx=0x7f3e80009ca0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 -- 172.21.15.167:0/3748761301 shutdown_connections 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 --2- 172.21.15.167:0/3748761301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e90102b40 0x7f3e901093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 --2- 172.21.15.167:0/3748761301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90100fa0 0x7f3e901013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 --2- 172.21.15.167:0/3748761301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e901021a0 0x7f3e90102600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 -- 172.21.15.167:0/3748761301 >> 172.21.15.167:0/3748761301 conn(0x7f3e900fc540 msgr2=0x7f3e900fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 -- 172.21.15.167:0/3748761301 shutdown_connections 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 -- 172.21.15.167:0/3748761301 wait complete. 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.705+0000 7f3e961b5700 1 Processor -- start 2023-10-06T21:06:00.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 -- start start 2023-10-06T21:06:00.711 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:00.711 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e901021a0 0x7f3e90197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:00.711 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 0x7f3e9019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019e7a0 con 0x7f3e901021a0 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019e8e0 con 0x7f3e90102b40 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e961b5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019eaa0 con 0x7f3e90100fa0 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59220/0 (socket says 172.21.15.167:59220) 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 -- 172.21.15.167:0/2103660953 learned_addr learned my addr 172.21.15.167:0/2103660953 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8ffff700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 0x7f3e9019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 -- 172.21.15.167:0/2103660953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 msgr2=0x7f3e9019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:00.712 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 0x7f3e9019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8effd700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e901021a0 0x7f3e90197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 -- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e901021a0 msgr2=0x7f3e90197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e901021a0 0x7f3e90197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 -- 172.21.15.167:0/2103660953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800095f0 con 0x7f3e90100fa0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8ffff700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 0x7f3e9019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8f7fe700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c8f0 tx=0x7f3e8400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.706+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84004220 con 0x7f3e90100fa0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.707+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e84007dc0 con 0x7f3e90100fa0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.707+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9019ed80 con 0x7f3e90100fa0 2023-10-06T21:06:00.713 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.707+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84003d10 con 0x7f3e90100fa0 2023-10-06T21:06:00.714 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.707+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9019f170 con 0x7f3e90100fa0 2023-10-06T21:06:00.714 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.707+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90107ee0 con 0x7f3e90100fa0 2023-10-06T21:06:00.714 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.709+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3e84004380 con 0x7f3e90100fa0 2023-10-06T21:06:00.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.709+0000 7f3e8cff9700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 0x7f3e780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:00.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.709+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f3e840cc080 con 0x7f3e90100fa0 2023-10-06T21:06:00.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.710+0000 7f3e8effd700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 0x7f3e780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:00.715 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.710+0000 7f3e8effd700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 0x7f3e780a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005b40 tx=0x7f3e80005ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:00.716 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.713+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e84096360 con 0x7f3e90100fa0 2023-10-06T21:06:00.917 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:00.914+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e9002d320 con 0x7f3e780a1480 2023-10-06T21:06:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi078 2023-10-06T21:06:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 5722M 2023-10-06T21:06:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:06:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:06:01.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:00 smithi026 ceph-mon[98490]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi078 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 5722M 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:00 smithi078 ceph-mon[134896]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:06:01.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi078 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 5722M 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: osdmap e50: 8 total, 7 up, 8 in 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:00 smithi167 ceph-mon[134494]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] boot 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:01.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:02.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:06:02.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:06:02.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] boot 2023-10-06T21:06:02.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T21:06:02.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:02.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:02.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:02.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:02.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: osd.7 [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] boot 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: osdmap e51: 8 total, 8 up, 8 in 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:02.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:02 smithi026 ceph-mon[98490]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:06:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:02 smithi026 ceph-mon[98490]: pgmap v171: 1 pgs: 1 unknown; 0 B data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:02 smithi078 ceph-mon[134896]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:06:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:02 smithi078 ceph-mon[134896]: pgmap v171: 1 pgs: 1 unknown; 0 B data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:03.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:02 smithi167 ceph-mon[134494]: osdmap e52: 8 total, 8 up, 8 in 2023-10-06T21:06:03.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:02 smithi167 ceph-mon[134494]: pgmap v171: 1 pgs: 1 unknown; 0 B data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/1260977191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]': finished 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: osdmap e54: 9 total, 8 up, 9 in 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:03 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/256146802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:06:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/1260977191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]': finished 2023-10-06T21:06:04.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: osdmap e54: 9 total, 8 up, 9 in 2023-10-06T21:06:04.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:04.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:03 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/256146802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:04.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: osdmap e53: 8 total, 8 up, 8 in 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/1260977191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]: dispatch 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd2b8aa3-49f9-4263-825a-bd720b77e281"}]': finished 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: osdmap e54: 9 total, 8 up, 9 in 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:04.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:03 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/256146802' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:05.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:04 smithi078 ceph-mon[134896]: pgmap v174: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:05.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:04 smithi167 ceph-mon[134494]: pgmap v174: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:05.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:04 smithi026 ceph-mon[98490]: pgmap v174: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:06.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:05 smithi078 ceph-mon[134896]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T21:06:06.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:06.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:05 smithi167 ceph-mon[134494]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T21:06:06.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:06.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:05 smithi026 ceph-mon[98490]: osdmap e55: 9 total, 8 up, 9 in 2023-10-06T21:06:06.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:06 smithi078 ceph-mon[134896]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T21:06:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:06 smithi078 ceph-mon[134896]: pgmap v177: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:07.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:06 smithi167 ceph-mon[134494]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T21:06:07.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:07.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:06 smithi167 ceph-mon[134494]: pgmap v177: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:06 smithi026 ceph-mon[98490]: osdmap e56: 9 total, 8 up, 9 in 2023-10-06T21:06:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:06 smithi026 ceph-mon[98490]: pgmap v177: 1 pgs: 1 unknown; 0 B data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:08 smithi026 ceph-mon[98490]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:08 smithi078 ceph-mon[134896]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:09.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:08 smithi167 ceph-mon[134494]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:10.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:06:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:09 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:10.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:06:10.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:09 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:10.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-06T21:06:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:09 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:10 smithi026 ceph-mon[98490]: Deploying daemon osd.8 on smithi167 2023-10-06T21:06:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:10 smithi026 ceph-mon[98490]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:11.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:10 smithi078 ceph-mon[134896]: Deploying daemon osd.8 on smithi167 2023-10-06T21:06:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:10 smithi078 ceph-mon[134896]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:11.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:10 smithi167 ceph-mon[134494]: Deploying daemon osd.8 on smithi167 2023-10-06T21:06:11.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:10 smithi167 ceph-mon[134494]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:13.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:13 smithi167 ceph-mon[134494]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:13 smithi026 ceph-mon[98490]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:13.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:13 smithi078 ceph-mon[134896]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:14 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:14 smithi167 ceph-mon[134494]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:14 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:14 smithi026 ceph-mon[98490]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:15.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:14 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:15.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:14 smithi078 ceph-mon[134896]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:16.129 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.127+0000 7f3e8cff9700 1 -- 172.21.15.167:0/2103660953 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3e9002d320 con 0x7f3e780a1480 2023-10-06T21:06:16.129 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 msgr2=0x7f3e780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 0x7f3e780a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005b40 tx=0x7f3e80005ad0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 msgr2=0x7f3e90196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c8f0 tx=0x7f3e8400ccb0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 shutdown_connections 2023-10-06T21:06:16.132 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e90102b40 0x7f3e9019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e90100fa0 0x7f3e90196c50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f3e780a1480 0x7f3e780a3940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 --2- 172.21.15.167:0/2103660953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e901021a0 0x7f3e90197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.129+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 >> 172.21.15.167:0/2103660953 conn(0x7f3e900fc540 msgr2=0x7f3e900fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.130+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 shutdown_connections 2023-10-06T21:06:16.133 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:06:16.130+0000 7f3e961b5700 1 -- 172.21.15.167:0/2103660953 wait complete. 2023-10-06T21:06:16.539 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-06T21:06:16.542 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2023-10-06T21:06:16.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:06:16.573 INFO:journalctl@ceph.osd.8.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:17.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:16 smithi167 ceph-mon[134494]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:17.285 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:06:17.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:16 smithi026 ceph-mon[98490]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:17.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:17.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:17.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:16 smithi078 ceph-mon[134896]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:17.799 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:06:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:06:17.593+0000 7f676b00d640 -1 osd.8 0 log_to_monitors true 2023-10-06T21:06:18.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:18 smithi078 ceph-mon[134896]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:18.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:18 smithi078 ceph-mon[134896]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:18.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:18 smithi026 ceph-mon[98490]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:18.522 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:18 smithi026 ceph-mon[98490]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:18.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:18 smithi167 ceph-mon[134494]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:18.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:18 smithi167 ceph-mon[134494]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0297509 s, 352 MB/s 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2023-10-06T21:06:19.262 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-10-06T21:06:19.282 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:06:19.134+0000 7f675fe83700 -1 osd.8 0 waiting for initial osdmap 2023-10-06T21:06:19.282 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:06:19.146+0000 7f6759473700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:19.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:19 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.398 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.399 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.400 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:19.400 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:19.400 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:19 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-06T21:06:19.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: osdmap e57: 9 total, 8 up, 9 in 2023-10-06T21:06:19.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:19.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:19.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:19.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:19 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:19.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2023-10-06T21:06:20.503 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:06:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi167 2023-10-06T21:06:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 5722M 2023-10-06T21:06:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:06:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:06:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:20 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi167 2023-10-06T21:06:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 5722M 2023-10-06T21:06:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:06:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:06:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:20.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:20 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:20.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi167 2023-10-06T21:06:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 5722M 2023-10-06T21:06:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:06:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: osdmap e58: 9 total, 8 up, 9 in 2023-10-06T21:06:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:20 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] boot 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:21.301 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:21 smithi026 ceph-mon[98490]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:21.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:06:21.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:06:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] boot 2023-10-06T21:06:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T21:06:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:21 smithi167 ceph-mon[134494]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: osd.8 [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] boot 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: osdmap e59: 9 total, 9 up, 9 in 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:06:21.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:21 smithi078 ceph-mon[134896]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:21.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 -- 172.21.15.26:0/3908182885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c108e40 msgr2=0x7f8f4c10b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 --2- 172.21.15.26:0/3908182885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c108e40 0x7f8f4c10b2d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b0a0 tx=0x7f8f4400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 -- 172.21.15.26:0/3908182885 shutdown_connections 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 --2- 172.21.15.26:0/3908182885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c108e40 0x7f8f4c10b2d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 --2- 172.21.15.26:0/3908182885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c074ee0 0x7f8f4c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 --2- 172.21.15.26:0/3908182885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c073910 0x7f8f4c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.872+0000 7f8f5126a700 1 -- 172.21.15.26:0/3908182885 >> 172.21.15.26:0/3908182885 conn(0x7f8f4c0fc4d0 msgr2=0x7f8f4c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.873+0000 7f8f5126a700 1 -- 172.21.15.26:0/3908182885 shutdown_connections 2023-10-06T21:06:21.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.873+0000 7f8f5126a700 1 -- 172.21.15.26:0/3908182885 wait complete. 2023-10-06T21:06:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.874+0000 7f8f5126a700 1 Processor -- start 2023-10-06T21:06:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.874+0000 7f8f5126a700 1 -- start start 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c074ee0 0x7f8f4c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c108e40 0x7f8f4c1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a0ac0 con 0x7f8f4c073910 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a0c00 con 0x7f8f4c108e40 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f5126a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1a0d90 con 0x7f8f4c074ee0 2023-10-06T21:06:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f4affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34100/0 (socket says 172.21.15.26:34100) 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f4affd700 1 -- 172.21.15.26:0/4236334372 learned_addr learned my addr 172.21.15.26:0/4236334372 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f4a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c074ee0 0x7f8f4c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.875+0000 7f8f4b7fe700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c108e40 0x7f8f4c1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 -- 172.21.15.26:0/4236334372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c074ee0 msgr2=0x7f8f4c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c074ee0 0x7f8f4c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 -- 172.21.15.26:0/4236334372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c108e40 msgr2=0x7f8f4c1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c108e40 0x7f8f4c1a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 -- 172.21.15.26:0/4236334372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009d00 con 0x7f8f4c073910 2023-10-06T21:06:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f4affd700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9a0 tx=0x7f8f3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c009790 con 0x7f8f4c073910 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c00c810 con 0x7f8f4c073910 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c1a1070 con 0x7f8f4c073910 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0054c0 con 0x7f8f4c073910 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.876+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c1a15c0 con 0x7f8f4c073910 2023-10-06T21:06:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.877+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c1074f0 con 0x7f8f4c073910 2023-10-06T21:06:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.879+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8f3c005620 con 0x7f8f4c073910 2023-10-06T21:06:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.881+0000 7f8f33fff700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.881+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f8f3c014070 con 0x7f8f4c073910 2023-10-06T21:06:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.881+0000 7f8f4a7fc700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.882+0000 7f8f4a7fc700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 0x7f8f340a39c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c074c10 tx=0x7f8f4000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:21.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:21.883+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f3c096720 con 0x7f8f4c073910 2023-10-06T21:06:22.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:22.104+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f4c02d230 con 0x7f8f340a1500 2023-10-06T21:06:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:22 smithi167 ceph-mon[134494]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:06:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:22 smithi026 ceph-mon[98490]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:06:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:22.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:22.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:22 smithi078 ceph-mon[134896]: osdmap e60: 9 total, 9 up, 9 in 2023-10-06T21:06:22.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:22.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:22.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:23.480 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:23 smithi026 ceph-mon[98490]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:23.480 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:23 smithi026 ceph-mon[98490]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:06:23.480 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:23 smithi026 ceph-mon[98490]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:23.480 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:23 smithi026 ceph-mon[98490]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:06:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:23 smithi167 ceph-mon[134494]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:23 smithi167 ceph-mon[134494]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:06:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:23 smithi167 ceph-mon[134494]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:23 smithi167 ceph-mon[134494]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:06:23.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:23 smithi078 ceph-mon[134896]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:23 smithi078 ceph-mon[134896]: osdmap e61: 9 total, 9 up, 9 in 2023-10-06T21:06:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:23 smithi078 ceph-mon[134896]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:23 smithi078 ceph-mon[134896]: osdmap e62: 9 total, 9 up, 9 in 2023-10-06T21:06:25.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:24 smithi026 ceph-mon[98490]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:06:25.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:24 smithi026 ceph-mon[98490]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:25.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:24 smithi078 ceph-mon[134896]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:06:25.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:24 smithi078 ceph-mon[134896]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:25.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:24 smithi167 ceph-mon[134494]: osdmap e63: 9 total, 9 up, 9 in 2023-10-06T21:06:25.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:24 smithi167 ceph-mon[134494]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:26.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:25 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4116862702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:25 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:25 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]': finished 2023-10-06T21:06:26.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:25 smithi026 ceph-mon[98490]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T21:06:26.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:26.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:25 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4116862702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:25 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:25 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]': finished 2023-10-06T21:06:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:25 smithi078 ceph-mon[134896]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T21:06:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:26.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:25 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4116862702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:25 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]: dispatch 2023-10-06T21:06:26.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:25 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc3a0395-6144-480c-aaf7-d2f8f8f22f3d"}]': finished 2023-10-06T21:06:26.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:25 smithi167 ceph-mon[134494]: osdmap e64: 10 total, 9 up, 10 in 2023-10-06T21:06:26.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:26 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1116807886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:26 smithi026 ceph-mon[98490]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:26 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1116807886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:26 smithi078 ceph-mon[134896]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:26 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1116807886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:26 smithi167 ceph-mon[134494]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:28 smithi026 ceph-mon[98490]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:06:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:28 smithi078 ceph-mon[134896]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:06:29.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:28 smithi167 ceph-mon[134494]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-06T21:06:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:31 smithi167 ceph-mon[134494]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:06:31.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:31 smithi026 ceph-mon[98490]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:06:31.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:31 smithi078 ceph-mon[134896]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-10-06T21:06:33.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:33 smithi026 ceph-mon[98490]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:06:33.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:33 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:06:33.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:33 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:33 smithi167 ceph-mon[134494]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:06:33.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:33 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:06:33.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:33 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:33.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:33 smithi078 ceph-mon[134896]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-06T21:06:33.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:33 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-06T21:06:33.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:33 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:34 smithi167 ceph-mon[134494]: Deploying daemon osd.9 on smithi026 2023-10-06T21:06:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:34 smithi026 ceph-mon[98490]: Deploying daemon osd.9 on smithi026 2023-10-06T21:06:34.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:34 smithi078 ceph-mon[134896]: Deploying daemon osd.9 on smithi026 2023-10-06T21:06:35.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:35 smithi026 ceph-mon[98490]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:06:35.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:35 smithi167 ceph-mon[134494]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:06:35.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:35 smithi078 ceph-mon[134896]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-06T21:06:37.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:37 smithi026 ceph-mon[98490]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-06T21:06:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:37 smithi167 ceph-mon[134494]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-06T21:06:37.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:37 smithi078 ceph-mon[134896]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-10-06T21:06:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:38 smithi026 ceph-mon[98490]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T21:06:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:38 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:38 smithi078 ceph-mon[134896]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T21:06:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:38 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:38 smithi167 ceph-mon[134494]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-06T21:06:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:39.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:38 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:41.694 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:41 smithi026 ceph-mon[98490]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:41.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:41 smithi167 ceph-mon[134494]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:41.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:41 smithi078 ceph-mon[134896]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.023+0000 7f8f33fff700 1 -- 172.21.15.26:0/4236334372 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8f4c02d230 con 0x7f8f340a1500 2023-10-06T21:06:42.026 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 msgr2=0x7f8f340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 0x7f8f340a39c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c074c10 tx=0x7f8f4000b410 comp rx=0 tx=0).stop 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 msgr2=0x7f8f4c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00e9a0 tx=0x7f8f3c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 shutdown_connections 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f4c108e40 0x7f8f4c1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f340a1500 0x7f8f340a39c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:42.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f4c073910 0x7f8f4c198f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 --2- 172.21.15.26:0/4236334372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f4c074ee0 0x7f8f4c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 >> 172.21.15.26:0/4236334372 conn(0x7f8f4c0fc4d0 msgr2=0x7f8f4c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:06:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 shutdown_connections 2023-10-06T21:06:42.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:06:42.026+0000 7f8f5126a700 1 -- 172.21.15.26:0/4236334372 wait complete. 2023-10-06T21:06:42.528 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-06T21:06:42.533 INFO:tasks.cephadm:Deploying osd.10 on smithi078 with /dev/vg_nvme/lv_1... 2023-10-06T21:06:42.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:06:42.608 INFO:journalctl@ceph.osd.9.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:06:42.823 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:06:42.693+0000 7f101fc73640 -1 osd.9 0 log_to_monitors true 2023-10-06T21:06:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:06:43.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.247 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:43.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:42 smithi026 ceph-mon[98490]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:42 smithi167 ceph-mon[134494]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:06:43.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:43.329 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:43.330 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:42 smithi078 ceph-mon[134896]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-06T21:06:44.297 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:06:44.097+0000 7f1014ae9700 -1 osd.9 0 waiting for initial osdmap 2023-10-06T21:06:44.297 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:06:44.113+0000 7f100e8da700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:06:44.298 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-mon[98490]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:06:44.298 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-mon[98490]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:06:44.298 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-mon[98490]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:06:44.298 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:44 smithi078 ceph-mon[134896]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:06:44.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:44 smithi078 ceph-mon[134896]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:06:44.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:44 smithi078 ceph-mon[134896]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:06:44.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:44.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:44 smithi167 ceph-mon[134494]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-06T21:06:44.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:44 smithi167 ceph-mon[134494]: osdmap e65: 10 total, 9 up, 10 in 2023-10-06T21:06:44.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:44 smithi167 ceph-mon[134494]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-10-06T21:06:44.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:44.568 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:06:44.569 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:06:44.569 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2023-10-06T21:06:44.569 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2023-10-06T21:06:44.569 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012655 s, 829 MB/s 2023-10-06T21:06:44.569 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2023-10-06T21:06:44.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2023-10-06T21:06:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:45 smithi078 ceph-mon[134896]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:06:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:45 smithi078 ceph-mon[134896]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:06:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:45 smithi078 ceph-mon[134896]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:06:45.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:45 smithi167 ceph-mon[134494]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:06:45.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:45 smithi167 ceph-mon[134494]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:06:45.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:45 smithi167 ceph-mon[134494]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:45.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:45 smithi026 ceph-mon[98490]: from='osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-10-06T21:06:45.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:45 smithi026 ceph-mon[98490]: osdmap e66: 10 total, 9 up, 10 in 2023-10-06T21:06:45.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:45.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:45 smithi026 ceph-mon[98490]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-06T21:06:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:46 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:06:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:46 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:06:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:46 smithi078 ceph-mon[134896]: osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] boot 2023-10-06T21:06:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:46 smithi078 ceph-mon[134896]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T21:06:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:46 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:46.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:46 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:06:46.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:46 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:06:46.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:46 smithi167 ceph-mon[134494]: osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] boot 2023-10-06T21:06:46.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:46 smithi167 ceph-mon[134494]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T21:06:46.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:46 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:46 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:06:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:46 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:06:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:46 smithi026 ceph-mon[98490]: osd.9 [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] boot 2023-10-06T21:06:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:46 smithi026 ceph-mon[98490]: osdmap e67: 10 total, 10 up, 10 in 2023-10-06T21:06:46.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:46 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:06:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 -- 172.21.15.78:0/2191598206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101a20 msgr2=0x7fd6d0108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:46.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2191598206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101a20 0x7fd6d0108340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b0a0 tx=0x7fd6c800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.723 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 -- 172.21.15.78:0/2191598206 shutdown_connections 2023-10-06T21:06:46.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2191598206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101a20 0x7fd6d0108340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2191598206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00ffe70 0x7fd6d0100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.724 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2191598206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d0101060 0x7fd6d01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 -- 172.21.15.78:0/2191598206 >> 172.21.15.78:0/2191598206 conn(0x7fd6d00fb480 msgr2=0x7fd6d00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:06:46.725 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.716+0000 7fd6d848b700 1 -- 172.21.15.78:0/2191598206 shutdown_connections 2023-10-06T21:06:46.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 -- 172.21.15.78:0/2191598206 wait complete. 2023-10-06T21:06:46.726 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 Processor -- start 2023-10-06T21:06:46.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 -- start start 2023-10-06T21:06:46.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00ffe70 0x7fd6d0104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:46.727 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:46.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d0101a20 0x7fd6d0102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:46.728 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0103340 con 0x7fd6d00ffe70 2023-10-06T21:06:46.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01034b0 con 0x7fd6d0101060 2023-10-06T21:06:46.729 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d848b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a0d50 con 0x7fd6d0101a20 2023-10-06T21:06:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d5a26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d5a26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40622/0 (socket says 172.21.15.78:40622) 2023-10-06T21:06:46.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.717+0000 7fd6d5a26700 1 -- 172.21.15.78:0/2791451371 learned_addr learned my addr 172.21.15.78:0/2791451371 (peer_addr_for_me v2:172.21.15.78:0/0) 2023-10-06T21:06:46.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 -- 172.21.15.78:0/2791451371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d0101a20 msgr2=0x7fd6d0102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:46.731 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d0101a20 0x7fd6d0102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 -- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00ffe70 msgr2=0x7fd6d0104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:06:46.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00ffe70 0x7fd6d0104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:06:46.732 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 -- 172.21.15.78:0/2791451371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009d00 con 0x7fd6d0101060 2023-10-06T21:06:46.733 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d5a26700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00c970 tx=0x7fd6cc00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc004220 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6cc007d60 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01a0fd0 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc003d00 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.718+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a14f0 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.719+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d0107e80 con 0x7fd6d0101060 2023-10-06T21:06:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.720+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd6cc004380 con 0x7fd6d0101060 2023-10-06T21:06:46.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.720+0000 7fd6c37fe700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 0x7fd6bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:06:46.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.720+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd6cc0cc420 con 0x7fd6d0101060 2023-10-06T21:06:46.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.721+0000 7fd6d6227700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 0x7fd6bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:06:46.735 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.721+0000 7fd6d6227700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 0x7fd6bc0a39c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0100bd0 tx=0x7fd6c4005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:06:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.724+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd6cc096280 con 0x7fd6d0101060 2023-10-06T21:06:46.924 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:06:46.922+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d002d320 con 0x7fd6bc0a1500 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi026 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 4318M 2023-10-06T21:06:47.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:47.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:47 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi026 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 4318M 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:47.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:47.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:47 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: osdmap e68: 10 total, 10 up, 10 in 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi026 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 4318M 2023-10-06T21:06:47.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:06:47.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:47 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:48.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:48 smithi078 ceph-mon[134896]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:48.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:48 smithi167 ceph-mon[134494]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:48.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:48 smithi026 ceph-mon[98490]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:06:49.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:49 smithi167 ceph-mon[134494]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:49.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:49 smithi026 ceph-mon[98490]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:49.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:49 smithi078 ceph-mon[134896]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/2730064997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]': finished 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:06:50.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:50 smithi167 ceph-mon[134494]: from='client.? 172.21.15.78:0/4096413240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/2730064997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]': finished 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:06:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.78:0/4096413240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/2730064997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]: dispatch 2023-10-06T21:06:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bda1e71-4772-49f7-8d68-cc56c7f174ba"}]': finished 2023-10-06T21:06:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: osdmap e69: 11 total, 10 up, 11 in 2023-10-06T21:06:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:06:50.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:50 smithi078 ceph-mon[134896]: from='client.? 172.21.15.78:0/4096413240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:06:51.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:51 smithi167 ceph-mon[134494]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:51 smithi026 ceph-mon[98490]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:51.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:51 smithi078 ceph-mon[134896]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:53.591 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:53 smithi078 ceph-mon[134896]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:53 smithi167 ceph-mon[134494]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:53.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:53 smithi026 ceph-mon[98490]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:55.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:55 smithi078 ceph-mon[134896]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:55.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:55 smithi167 ceph-mon[134494]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:55 smithi026 ceph-mon[98490]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:56.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:06:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:56.785 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:06:56.785 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-06T21:06:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:06:57.678 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:57 smithi078 ceph-mon[134896]: Deploying daemon osd.10 on smithi078 2023-10-06T21:06:57.678 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:57 smithi078 ceph-mon[134896]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:57.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:57 smithi167 ceph-mon[134494]: Deploying daemon osd.10 on smithi078 2023-10-06T21:06:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:57 smithi167 ceph-mon[134494]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:57.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:57 smithi026 ceph-mon[98490]: Deploying daemon osd.10 on smithi078 2023-10-06T21:06:57.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:57 smithi026 ceph-mon[98490]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:06:58 smithi026 ceph-mon[98490]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:06:58 smithi078 ceph-mon[134896]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:06:59.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:06:58 smithi167 ceph-mon[134494]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:01.707 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:01 smithi078 ceph-mon[134896]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:01.708 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:01.708 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:01.708 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:01 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:01.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:01 smithi167 ceph-mon[134494]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:01.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:01.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:01.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:01 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:01.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:01 smithi026 ceph-mon[98490]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:01.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:01.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:01.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:01 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.587+0000 7fd6c37fe700 1 -- 172.21.15.78:0/2791451371 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd6d002d320 con 0x7fd6bc0a1500 2023-10-06T21:07:03.590 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 10 on host 'smithi078' 2023-10-06T21:07:03.591 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 msgr2=0x7fd6bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 0x7fd6bc0a39c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0100bd0 tx=0x7fd6c4005d20 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 msgr2=0x7fd6d0102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00c970 tx=0x7fd6cc00cd30 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 shutdown_connections 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d0101a20 0x7fd6d0102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd6bc0a1500 0x7fd6bc0a39c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6d00ffe70 0x7fd6d0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 --2- 172.21.15.78:0/2791451371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6d0101060 0x7fd6d0102680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 >> 172.21.15.78:0/2791451371 conn(0x7fd6d00fb480 msgr2=0x7fd6d00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 shutdown_connections 2023-10-06T21:07:03.592 INFO:teuthology.orchestra.run.smithi078.stderr:2023-10-06T21:07:03.589+0000 7fd6d848b700 1 -- 172.21.15.78:0/2791451371 wait complete. 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:03.746 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:03 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:03.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:03 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:03.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:03.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:03 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:03.992 DEBUG:teuthology.orchestra.run.smithi078:osd.10> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-06T21:07:03.995 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2023-10-06T21:07:03.995 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-06T21:07:04.025 INFO:journalctl@ceph.osd.10.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:07:04.566 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:07:04.687 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:07:04.489+0000 7f68346cc640 -1 osd.10 0 log_to_monitors true 2023-10-06T21:07:04.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:04.687 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:04.688 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:04 smithi078 ceph-mon[134896]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:04.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:04.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:04.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:04.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:04 smithi167 ceph-mon[134494]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:04.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:04.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:04 smithi026 ceph-mon[98490]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-06T21:07:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:05 smithi026 ceph-mon[98490]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:07:05.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:05 smithi026 ceph-mon[98490]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:05.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:05 smithi026 ceph-mon[98490]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:07:05.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:05.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:05 smithi026 ceph-mon[98490]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:05.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-mon[134896]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:07:05.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-mon[134896]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:05.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-mon[134896]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:07:05.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:05.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-mon[134896]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:05.873 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:07:05.551+0000 7f6829542700 -1 osd.10 0 waiting for initial osdmap 2023-10-06T21:07:05.873 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:07:05 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:07:05.566+0000 7f6824b36700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:07:06.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:05 smithi167 ceph-mon[134494]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-06T21:07:06.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:05 smithi167 ceph-mon[134494]: from='osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:06.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:05 smithi167 ceph-mon[134494]: osdmap e70: 11 total, 10 up, 11 in 2023-10-06T21:07:06.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:05 smithi167 ceph-mon[134494]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2023-10-06T21:07:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-06T21:07:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-06T21:07:06.034 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2023-10-06T21:07:06.035 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2023-10-06T21:07:06.035 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146034 s, 718 MB/s 2023-10-06T21:07:06.035 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2023-10-06T21:07:06.423 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi078 2023-10-06T21:07:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 4292M 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:06.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:06.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:07:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:07:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi078 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 4292M 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:06.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: osdmap e71: 11 total, 10 up, 11 in 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi078 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 4292M 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:06.982 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:07:07.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:07:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:07:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] boot 2023-10-06T21:07:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T21:07:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:07 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:07.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:07:07.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:07:07.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:07.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] boot 2023-10-06T21:07:07.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T21:07:07.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:07 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: osd.10 [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] boot 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: osdmap e72: 11 total, 11 up, 11 in 2023-10-06T21:07:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:07 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:07:08.185 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 -- 172.21.15.167:0/763178755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57880ffe70 msgr2=0x7f5788100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 --2- 172.21.15.167:0/763178755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57880ffe70 0x7f5788100290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5778009990 tx=0x7f5778009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 -- 172.21.15.167:0/763178755 shutdown_connections 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 --2- 172.21.15.167:0/763178755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f57881082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 --2- 172.21.15.167:0/763178755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57880ffe70 0x7f5788100290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 --2- 172.21.15.167:0/763178755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5788100fd0 0x7f5788101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 -- 172.21.15.167:0/763178755 >> 172.21.15.167:0/763178755 conn(0x7f57880fb3f0 msgr2=0x7f57880fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 -- 172.21.15.167:0/763178755 shutdown_connections 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.183+0000 7f57900be700 1 -- 172.21.15.167:0/763178755 wait complete. 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 Processor -- start 2023-10-06T21:07:08.186 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 -- start start 2023-10-06T21:07:08.188 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 0x7f5788196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:08.188 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:08.188 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f578819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:08.188 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578819e790 con 0x7f57880ffe70 2023-10-06T21:07:08.188 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578819e8d0 con 0x7f5788101990 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f57900be700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578819ea90 con 0x7f5788100fd0 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59380/0 (socket says 172.21.15.167:59380) 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 -- 172.21.15.167:0/566358930 learned_addr learned my addr 172.21.15.167:0/566358930 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578e65b700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f578819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578de5a700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 0x7f5788196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 -- 172.21.15.167:0/566358930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 msgr2=0x7f578819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f578819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 -- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 msgr2=0x7f5788196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 0x7f5788196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:08.189 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578d659700 1 -- 172.21.15.167:0/566358930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780095f0 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578e65b700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f578819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.184+0000 7f578de5a700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 0x7f5788196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f578d659700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f578400b530 tx=0x7f578400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011830 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5784011e50 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578819ed70 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400f2e0 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.185+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578819f1c0 con 0x7f5788100fd0 2023-10-06T21:07:08.190 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.186+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5788107f50 con 0x7f5788100fd0 2023-10-06T21:07:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.187+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5784010580 con 0x7f5788100fd0 2023-10-06T21:07:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.187+0000 7f577effd700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 0x7f57740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.188+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f57840cc820 con 0x7f5788100fd0 2023-10-06T21:07:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.188+0000 7f578de5a700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 0x7f57740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.189+0000 7f578de5a700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 0x7f57740a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c00 tx=0x7f5778005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:08.193 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.191+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5784096410 con 0x7f5788100fd0 2023-10-06T21:07:08.389 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:08.387+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f578802d360 con 0x7f57740a1480 2023-10-06T21:07:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:07:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:07:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:07:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:07:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:09.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: osdmap e73: 11 total, 11 up, 11 in 2023-10-06T21:07:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-06T21:07:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-06T21:07:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:09.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:09 smithi167 ceph-mon[134494]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:07:09.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:09 smithi026 ceph-mon[98490]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:07:09.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:09 smithi078 ceph-mon[134896]: osdmap e74: 11 total, 11 up, 11 in 2023-10-06T21:07:10.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:10 smithi078 ceph-mon[134896]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:07:10.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:10 smithi078 ceph-mon[134896]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:10 smithi167 ceph-mon[134494]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:07:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:10 smithi167 ceph-mon[134494]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:10 smithi026 ceph-mon[98490]: osdmap e75: 11 total, 11 up, 11 in 2023-10-06T21:07:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:10 smithi026 ceph-mon[98490]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:11.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/3594871099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]': finished 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:11 smithi078 ceph-mon[134896]: from='client.? 172.21.15.167:0/2599473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/3594871099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]': finished 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:11 smithi167 ceph-mon[134494]: from='client.? 172.21.15.167:0/2599473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:07:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: osdmap e76: 11 total, 11 up, 11 in 2023-10-06T21:07:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/3594871099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]: dispatch 2023-10-06T21:07:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d023d04f-16f7-4e4c-8fef-e2ac8e7db619"}]': finished 2023-10-06T21:07:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: osdmap e77: 12 total, 11 up, 12 in 2023-10-06T21:07:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:11 smithi026 ceph-mon[98490]: from='client.? 172.21.15.167:0/2599473814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-06T21:07:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:12 smithi167 ceph-mon[134494]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:07:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:12 smithi026 ceph-mon[98490]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:07:13.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:12 smithi078 ceph-mon[134896]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-06T21:07:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:15 smithi167 ceph-mon[134494]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T21:07:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:15 smithi026 ceph-mon[98490]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T21:07:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:15 smithi078 ceph-mon[134896]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-10-06T21:07:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:17 smithi167 ceph-mon[134494]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T21:07:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:07:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:17 smithi026 ceph-mon[98490]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T21:07:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:07:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:17.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:17 smithi078 ceph-mon[134896]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-10-06T21:07:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-06T21:07:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:18 smithi167 ceph-mon[134494]: Deploying daemon osd.11 on smithi167 2023-10-06T21:07:18.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:18 smithi026 ceph-mon[98490]: Deploying daemon osd.11 on smithi167 2023-10-06T21:07:18.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:18 smithi078 ceph-mon[134896]: Deploying daemon osd.11 on smithi167 2023-10-06T21:07:19.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:19 smithi167 ceph-mon[134494]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:07:19.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:19 smithi026 ceph-mon[98490]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:07:19.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:19 smithi078 ceph-mon[134896]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-06T21:07:21.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:21 smithi167 ceph-mon[134494]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-06T21:07:21.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:21 smithi026 ceph-mon[98490]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-06T21:07:21.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:21 smithi078 ceph-mon[134896]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-10-06T21:07:22.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:22.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:22.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:22.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:22 smithi167 ceph-mon[134494]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T21:07:22.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:22 smithi078 ceph-mon[134896]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T21:07:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:23.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:23.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:22 smithi026 ceph-mon[98490]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-10-06T21:07:24.257 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.255+0000 7f577effd700 1 -- 172.21.15.167:0/566358930 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f578802d360 con 0x7f57740a1480 2023-10-06T21:07:24.257 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 msgr2=0x7f57740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 0x7f57740a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c00 tx=0x7f5778005fb0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 msgr2=0x7f5788197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f578400b530 tx=0x7f578400b8f0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 shutdown_connections 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5788101990 0x7f578819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.260 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57740a1480 0x7f57740a3940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.261 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57880ffe70 0x7f5788196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.261 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 --2- 172.21.15.167:0/566358930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5788100fd0 0x7f5788197180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:24.261 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 >> 172.21.15.167:0/566358930 conn(0x7f57880fb3f0 msgr2=0x7f57880fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:24.261 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 shutdown_connections 2023-10-06T21:07:24.261 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:24.258+0000 7f57900be700 1 -- 172.21.15.167:0/566358930 wait complete. 2023-10-06T21:07:24.659 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-06T21:07:24.662 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-06T21:07:24.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T21:07:24.690 INFO:journalctl@ceph.osd.11.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:07:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:07:25.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-mon[134494]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:25 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:25 smithi026 ceph-mon[98490]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:25.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:25 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:25.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:25 smithi078 ceph-mon[134896]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:25.782 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:07:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:07:25.617+0000 7f58af6ff640 -1 osd.11 0 log_to_monitors true 2023-10-06T21:07:26.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:26 smithi078 ceph-mon[134896]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:26 smithi078 ceph-mon[134896]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:26 smithi026 ceph-mon[98490]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.694 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:26 smithi026 ceph-mon[98490]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:26 smithi167 ceph-mon[134494]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:26 smithi167 ceph-mon[134494]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 -- 172.21.15.26:0/1486144700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 msgr2=0x7fb74c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 --2- 172.21.15.26:0/1486144700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1025d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb7400099c0 tx=0x7fb740009cd0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 -- 172.21.15.26:0/1486144700 shutdown_connections 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 --2- 172.21.15.26:0/1486144700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c102b10 0x7fb74c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 --2- 172.21.15.26:0/1486144700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c100f70 0x7fb74c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 --2- 172.21.15.26:0/1486144700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1025d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 -- 172.21.15.26:0/1486144700 >> 172.21.15.26:0/1486144700 conn(0x7fb74c0fc4d0 msgr2=0x7fb74c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 -- 172.21.15.26:0/1486144700 shutdown_connections 2023-10-06T21:07:26.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.942+0000 7fb75208c700 1 -- 172.21.15.26:0/1486144700 wait complete. 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 Processor -- start 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 -- start start 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c100f70 0x7fb74c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1a0a00 con 0x7fb74c102170 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1a0b40 con 0x7fb74c100f70 2023-10-06T21:07:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.943+0000 7fb75208c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1a0cd0 con 0x7fb74c102b10 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:53136/0 (socket says 172.21.15.26:53136) 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c100f70 0x7fb74c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 -- 172.21.15.26:0/1650179961 learned_addr learned my addr 172.21.15.26:0/1650179961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 -- 172.21.15.26:0/1650179961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c100f70 msgr2=0x7fb74c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c100f70 0x7fb74c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 -- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 msgr2=0x7fb74c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 -- 172.21.15.26:0/1650179961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400095f0 con 0x7fb74c102b10 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74affd700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.944+0000 7fb74bfff700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb74400ef60 tx=0x7fb74400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.945+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744018070 con 0x7fb74c102b10 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.945+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb74400f040 con 0x7fb74c102b10 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.945+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c1a0f50 con 0x7fb74c102b10 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.945+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400ccc0 con 0x7fb74c102b10 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.945+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c1a1370 con 0x7fb74c102b10 2023-10-06T21:07:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.946+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c1935b0 con 0x7fb74c102b10 2023-10-06T21:07:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.948+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb744003a00 con 0x7fb74c102b10 2023-10-06T21:07:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.949+0000 7fb748ff9700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 0x7fb7340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.949+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fb7440cc3a0 con 0x7fb74c102b10 2023-10-06T21:07:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.949+0000 7fb74b7fe700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 0x7fb7340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.950+0000 7fb74b7fe700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 0x7fb7340a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb74c101fd0 tx=0x7fb73c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:26.951+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb744095fc0 con 0x7fb74c102b10 2023-10-06T21:07:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.164+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb74c10c7c0 con 0x7fb74c102b10 2023-10-06T21:07:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.165+0000 7fb748ff9700 1 -- 172.21.15.26:0/1650179961 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fb7440995e0 con 0x7fb74c102b10 2023-10-06T21:07:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.167+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 msgr2=0x7fb7340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.167+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 0x7fb7340a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb74c101fd0 tx=0x7fb73c009260 comp rx=0 tx=0).stop 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.167+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 msgr2=0x7fb74c1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.167+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb74400ef60 tx=0x7fb74400ef90 comp rx=0 tx=0).stop 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.167+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 shutdown_connections 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb74c102b10 0x7fb74c1a0430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb74c100f70 0x7fb74c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb7340a1480 0x7fb7340a3940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 --2- 172.21.15.26:0/1650179961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74c102170 0x7fb74c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 >> 172.21.15.26:0/1650179961 conn(0x7fb74c0fc4d0 msgr2=0x7fb74c107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 shutdown_connections 2023-10-06T21:07:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:27.168+0000 7fb75208c700 1 -- 172.21.15.26:0/1650179961 wait complete. 2023-10-06T21:07:27.361 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:07:27.338+0000 7f58a4575700 -1 osd.11 0 waiting for initial osdmap 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1650179961' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:07:27.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1650179961' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:27.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:27.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:27.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:27 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: osdmap e78: 12 total, 11 up, 12 in 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1650179961' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:27.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:27 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:27.699 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1696626426,"num_in_osds":12,"osd_in_since":1696626430,"num_remapped_pgs":0} 2023-10-06T21:07:27.782 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:07:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:07:27.357+0000 7f589fb69700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: Detected new or changed devices on smithi167 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 4292M 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:28.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:28.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:28 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: Detected new or changed devices on smithi167 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 4292M 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:28.771 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:28.772 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:28.772 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:28.772 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:28 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: Detected new or changed devices on smithi167 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 4292M 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: osdmap e79: 12 total, 11 up, 12 in 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:28 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:29.447 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:07:29.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: purged_snaps scrub starts 2023-10-06T21:07:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: purged_snaps scrub ok 2023-10-06T21:07:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] boot 2023-10-06T21:07:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T21:07:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:29 smithi078 ceph-mon[134896]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:29.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: purged_snaps scrub starts 2023-10-06T21:07:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: purged_snaps scrub ok 2023-10-06T21:07:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] boot 2023-10-06T21:07:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T21:07:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:29.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:29 smithi026 ceph-mon[98490]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:29.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: purged_snaps scrub starts 2023-10-06T21:07:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: purged_snaps scrub ok 2023-10-06T21:07:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: osd.11 [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] boot 2023-10-06T21:07:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: osdmap e80: 12 total, 12 up, 12 in 2023-10-06T21:07:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:07:29.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:29 smithi167 ceph-mon[134494]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-10-06T21:07:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:30 smithi026 ceph-mon[98490]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:07:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:30 smithi078 ceph-mon[134896]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:07:30.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:30 smithi167 ceph-mon[134494]: osdmap e81: 12 total, 12 up, 12 in 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.957+0000 7f022bfff700 1 -- 172.21.15.26:0/3308737095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c100dd0 msgr2=0x7f022c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.957+0000 7f022bfff700 1 --2- 172.21.15.26:0/3308737095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c100dd0 0x7f022c1011f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f021c0099c0 tx=0x7f021c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 -- 172.21.15.26:0/3308737095 shutdown_connections 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 --2- 172.21.15.26:0/3308737095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c102970 0x7f022c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 --2- 172.21.15.26:0/3308737095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c100dd0 0x7f022c1011f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 --2- 172.21.15.26:0/3308737095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c101fd0 0x7f022c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 -- 172.21.15.26:0/3308737095 >> 172.21.15.26:0/3308737095 conn(0x7f022c0fc390 msgr2=0x7f022c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 -- 172.21.15.26:0/3308737095 shutdown_connections 2023-10-06T21:07:30.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.958+0000 7f022bfff700 1 -- 172.21.15.26:0/3308737095 wait complete. 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 Processor -- start 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 -- start start 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c101fd0 0x7f022c197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 0x7f022c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c19e660 con 0x7f022c102970 2023-10-06T21:07:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c19e7a0 con 0x7f022c101fd0 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.959+0000 7f022bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c19e930 con 0x7f022c100dd0 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41712/0 (socket says 172.21.15.26:41712) 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 -- 172.21.15.26:0/841928332 learned_addr learned my addr 172.21.15.26:0/841928332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022b7fe700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 0x7f022c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022a7fc700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c101fd0 0x7f022c197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 -- 172.21.15.26:0/841928332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c101fd0 msgr2=0x7f022c197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c101fd0 0x7f022c197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 -- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 msgr2=0x7f022c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 0x7f022c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 -- 172.21.15.26:0/841928332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0095f0 con 0x7f022c100dd0 2023-10-06T21:07:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022b7fe700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 0x7f022c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.960+0000 7f022affd700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.961+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c01d070 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.961+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c19ebb0 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.961+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c19f020 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.961+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021c005030 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.961+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c004a90 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.962+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c10c620 con 0x7f022c100dd0 2023-10-06T21:07:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.964+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f021c0051a0 con 0x7f022c100dd0 2023-10-06T21:07:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.966+0000 7f0213fff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 0x7f02140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.966+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f021c0cd7e0 con 0x7f022c100dd0 2023-10-06T21:07:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.966+0000 7f022a7fc700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 0x7f02140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:30.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.966+0000 7f022a7fc700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 0x7f02140a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0220009c60 tx=0x7f0220009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:30.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:30.967+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f021c0972d0 con 0x7f022c100dd0 2023-10-06T21:07:31.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.177+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f022c19f1b0 con 0x7f022c100dd0 2023-10-06T21:07:31.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.178+0000 7f0213fff700 1 -- 172.21.15.26:0/841928332 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f021c09a8f0 con 0x7f022c100dd0 2023-10-06T21:07:31.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 msgr2=0x7f02140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 0x7f02140a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0220009c60 tx=0x7f0220009260 comp rx=0 tx=0).stop 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 msgr2=0x7f022c196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c00f690 comp rx=0 tx=0).stop 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 shutdown_connections 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f02140a1480 0x7f02140a3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f022c102970 0x7f022c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f022c100dd0 0x7f022c196ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 --2- 172.21.15.26:0/841928332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f022c101fd0 0x7f022c197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:31.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 >> 172.21.15.26:0/841928332 conn(0x7f022c0fc390 msgr2=0x7f022c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:31.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 shutdown_connections 2023-10-06T21:07:31.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:31.181+0000 7f022bfff700 1 -- 172.21.15.26:0/841928332 wait complete. 2023-10-06T21:07:31.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:31 smithi026 ceph-mon[98490]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:31.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:31 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/841928332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:31.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:31 smithi078 ceph-mon[134896]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:31.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:31 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/841928332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:31.630 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1696626448,"num_in_osds":12,"osd_in_since":1696626430,"num_remapped_pgs":0} 2023-10-06T21:07:31.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:07:31.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:31 smithi167 ceph-mon[134494]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:31.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:31 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/841928332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-06T21:07:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:07:33.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:33 smithi026 ceph-mon[98490]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:33.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:33 smithi078 ceph-mon[134896]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:33 smithi167 ceph-mon[134494]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:33.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 -- 172.21.15.26:0/786688869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458106f90 msgr2=0x7fd458107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:33.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 --2- 172.21.15.26:0/786688869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458106f90 0x7fd458107370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4540099c0 tx=0x7fd454009cd0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 -- 172.21.15.26:0/786688869 shutdown_connections 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 --2- 172.21.15.26:0/786688869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458101940 0x7fd45810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 --2- 172.21.15.26:0/786688869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458106f90 0x7fd458107370 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 --2- 172.21.15.26:0/786688869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458100f80 0x7fd458101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 -- 172.21.15.26:0/786688869 >> 172.21.15.26:0/786688869 conn(0x7fd4580780a0 msgr2=0x7fd4580784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 -- 172.21.15.26:0/786688869 shutdown_connections 2023-10-06T21:07:33.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.864+0000 7fd45f70b700 1 -- 172.21.15.26:0/786688869 wait complete. 2023-10-06T21:07:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.865+0000 7fd45f70b700 1 Processor -- start 2023-10-06T21:07:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.865+0000 7fd45f70b700 1 -- start start 2023-10-06T21:07:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458100f80 0x7fd4581991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:33.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458106f90 0x7fd45819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e1a0 con 0x7fd458101940 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e310 con 0x7fd458100f80 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45f70b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819e4d0 con 0x7fd458106f90 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45cca6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45cca6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52668/0 (socket says 172.21.15.26:52668) 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45cca6700 1 -- 172.21.15.26:0/83223651 learned_addr learned my addr 172.21.15.26:0/83223651 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45d4a7700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458100f80 0x7fd4581991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:33.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.866+0000 7fd45dca8700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458106f90 0x7fd45819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 -- 172.21.15.26:0/83223651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458106f90 msgr2=0x7fd45819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458106f90 0x7fd45819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 -- 172.21.15.26:0/83223651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458100f80 msgr2=0x7fd4581991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458100f80 0x7fd4581991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 -- 172.21.15.26:0/83223651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540095f0 con 0x7fd458101940 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45cca6700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd44800e9a0 tx=0x7fd44800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448009790 con 0x7fd458101940 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd448010430 con 0x7fd458101940 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819e7b0 con 0x7fd458101940 2023-10-06T21:07:33.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448018680 con 0x7fd458101940 2023-10-06T21:07:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.867+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4581a6010 con 0x7fd458101940 2023-10-06T21:07:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.869+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45804ebb0 con 0x7fd458101940 2023-10-06T21:07:33.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.871+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd4480105a0 con 0x7fd458101940 2023-10-06T21:07:33.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.872+0000 7fd44e7fc700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 0x7fd4440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:33.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.872+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd448014070 con 0x7fd458101940 2023-10-06T21:07:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.872+0000 7fd45d4a7700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 0x7fd4440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:33.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.873+0000 7fd45d4a7700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 0x7fd4440a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd454009990 tx=0x7fd454005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:33.876+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd4480964b0 con 0x7fd458101940 2023-10-06T21:07:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.087+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd458111080 con 0x7fd458101940 2023-10-06T21:07:34.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.088+0000 7fd44e7fc700 1 -- 172.21.15.26:0/83223651 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18371 (secure 0 0 0) 0x7fd448099ad0 con 0x7fd458101940 2023-10-06T21:07:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:34.091 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:07:29.352305+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:07:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 msgr2=0x7fd4440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 0x7fd4440a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd454009990 tx=0x7fd454005fb0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 msgr2=0x7fd458199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd44800e9a0 tx=0x7fd44800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 shutdown_connections 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd458106f90 0x7fd45819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd458100f80 0x7fd4581991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd4440a1500 0x7fd4440a39c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 --2- 172.21.15.26:0/83223651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458101940 0x7fd458199720 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.090+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 >> 172.21.15.26:0/83223651 conn(0x7fd4580780a0 msgr2=0x7fd458073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.091+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 shutdown_connections 2023-10-06T21:07:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:34.091+0000 7fd45f70b700 1 -- 172.21.15.26:0/83223651 wait complete. 2023-10-06T21:07:34.551 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:07:34.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-06T21:07:34.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:34 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/83223651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:34 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/83223651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:34 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/83223651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:35.291 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:07:35.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:35 smithi026 ceph-mon[98490]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:35.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:35 smithi167 ceph-mon[134494]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:35 smithi078 ceph-mon[134896]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:36.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 -- 172.21.15.26:0/584150032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 msgr2=0x7f74d010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:36.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/584150032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d010ddb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b0d0 tx=0x7f74c800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 -- 172.21.15.26:0/584150032 shutdown_connections 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/584150032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d010ddb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/584150032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 0x7f74d0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/584150032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 -- 172.21.15.26:0/584150032 >> 172.21.15.26:0/584150032 conn(0x7f74d00780a0 msgr2=0x7f74d00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 -- 172.21.15.26:0/584150032 shutdown_connections 2023-10-06T21:07:36.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.792+0000 7f74d5ecc700 1 -- 172.21.15.26:0/584150032 wait complete. 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.793+0000 7f74d5ecc700 1 Processor -- start 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.793+0000 7f74d5ecc700 1 -- start start 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 0x7f74d019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019e300 con 0x7f74d0101940 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019e470 con 0x7f74d0106f90 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74d5ecc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d019e630 con 0x7f74d0100f80 2023-10-06T21:07:36.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74ceffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74cf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74cf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41752/0 (socket says 172.21.15.26:41752) 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74cf7fe700 1 -- 172.21.15.26:0/824620088 learned_addr learned my addr 172.21.15.26:0/824620088 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.794+0000 7f74cffff700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 0x7f74d019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 -- 172.21.15.26:0/824620088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 msgr2=0x7f74d019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 0x7f74d019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 -- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 msgr2=0x7f74d0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d0199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 -- 172.21.15.26:0/824620088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8009d00 con 0x7f74d0100f80 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74ceffd700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74cf7fe700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d950 tx=0x7f74c000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:36.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000f890 con 0x7f74d0100f80 2023-10-06T21:07:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d019e910 con 0x7f74d0100f80 2023-10-06T21:07:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74c0010460 con 0x7f74d0100f80 2023-10-06T21:07:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000e610 con 0x7f74d0100f80 2023-10-06T21:07:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.795+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d01a6170 con 0x7f74d0100f80 2023-10-06T21:07:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.796+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d004ebb0 con 0x7f74d0100f80 2023-10-06T21:07:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.798+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f74c00108d0 con 0x7f74d0100f80 2023-10-06T21:07:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.800+0000 7f74ccff9700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 0x7f74b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.800+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74c00cc5d0 con 0x7f74d0100f80 2023-10-06T21:07:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.800+0000 7f74ceffd700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 0x7f74b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.800+0000 7f74ceffd700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 0x7f74b80a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74d019a960 tx=0x7f74c400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:36.801+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f74c0096030 con 0x7f74d0100f80 2023-10-06T21:07:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.010+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74d019a6a0 con 0x7f74d0100f80 2023-10-06T21:07:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.010+0000 7f74ccff9700 1 -- 172.21.15.26:0/824620088 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74c0099650 con 0x7f74d0100f80 2023-10-06T21:07:37.013 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:07:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 msgr2=0x7f74b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 0x7f74b80a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74d019a960 tx=0x7f74c400b410 comp rx=0 tx=0).stop 2023-10-06T21:07:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 msgr2=0x7f74d0199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d950 tx=0x7f74c000dc60 comp rx=0 tx=0).stop 2023-10-06T21:07:37.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 shutdown_connections 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74d0106f90 0x7f74d019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74d0100f80 0x7f74d0199340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f74b80a1480 0x7f74b80a3940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 --2- 172.21.15.26:0/824620088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d0101940 0x7f74d0199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 >> 172.21.15.26:0/824620088 conn(0x7f74d00780a0 msgr2=0x7f74d010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 shutdown_connections 2023-10-06T21:07:37.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:37.013+0000 7f74d5ecc700 1 -- 172.21.15.26:0/824620088 wait complete. 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.a on smithi026 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.d on smithi026 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.g on smithi026 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.j on smithi026 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.b on smithi078 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.e on smithi078 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.h on smithi078 2023-10-06T21:07:37.471 INFO:tasks.cephadm:Adding mds.k on smithi078 2023-10-06T21:07:37.472 INFO:tasks.cephadm:Adding mds.c on smithi167 2023-10-06T21:07:37.472 INFO:tasks.cephadm:Adding mds.f on smithi167 2023-10-06T21:07:37.472 INFO:tasks.cephadm:Adding mds.i on smithi167 2023-10-06T21:07:37.472 INFO:tasks.cephadm:Adding mds.l on smithi167 2023-10-06T21:07:37.472 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2023-10-06T21:07:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:37 smithi167 ceph-mon[134494]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:37.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:37 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/824620088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:37 smithi026 ceph-mon[98490]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:37 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/824620088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:37.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:37 smithi078 ceph-mon[134896]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:37.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:37 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/824620088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:38.046 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:07:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:38 smithi026 ceph-mon[98490]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:39.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:38 smithi167 ceph-mon[134494]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:38 smithi078 ceph-mon[134896]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:39.244 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/4292128862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 msgr2=0x7f5ee4103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:39.244 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/4292128862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4103df0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009990 tx=0x7f5ed4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.244 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/4292128862 shutdown_connections 2023-10-06T21:07:39.244 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/4292128862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee4106c60 0x7f5ee41090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.245 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/4292128862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4103df0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.245 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/4292128862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ee4104330 0x7f5ee4106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.245 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/4292128862 >> 172.21.15.167:0/4292128862 conn(0x7f5ee40fb3f0 msgr2=0x7f5ee40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:39.245 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/4292128862 shutdown_connections 2023-10-06T21:07:39.245 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.241+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/4292128862 wait complete. 2023-10-06T21:07:39.246 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 Processor -- start 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 -- start start 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee4104330 0x7f5ee40ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ee4106c60 0x7f5ee40ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4100670 con 0x7f5ee4104330 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41007e0 con 0x7f5ee4106c60 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5eeb5c0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41a0cf0 con 0x7f5ee4101a00 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59504/0 (socket says 172.21.15.167:59504) 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 -- 172.21.15.167:0/3402529259 learned_addr learned my addr 172.21.15.167:0/3402529259 (peer_addr_for_me v2:172.21.15.167:0/0) 2023-10-06T21:07:39.247 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 -- 172.21.15.167:0/3402529259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ee4106c60 msgr2=0x7f5ee40ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ee4106c60 0x7f5ee40ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 -- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee4104330 msgr2=0x7f5ee40ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee4104330 0x7f5ee40ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 -- 172.21.15.167:0/3402529259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40095f0 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.242+0000 7f5ee935c700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000c00 tx=0x7f5ed400f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.243+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed401d070 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.243+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed400fc00 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.243+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee41a0f70 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.243+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4017650 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.243+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee41a13e0 con 0x7f5ee4101a00 2023-10-06T21:07:39.248 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.244+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee40fcf70 con 0x7f5ee4101a00 2023-10-06T21:07:39.249 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.245+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5ed4022470 con 0x7f5ee4101a00 2023-10-06T21:07:39.249 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.245+0000 7f5eda7fc700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 0x7f5ed00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:39.249 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.245+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ed40cddd0 con 0x7f5ee4101a00 2023-10-06T21:07:39.249 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.246+0000 7f5ee8b5b700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 0x7f5ed00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:39.250 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.246+0000 7f5ee8b5b700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 0x7f5ed00a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0005950 tx=0x7f5ee00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:39.251 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.248+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5ed4097960 con 0x7f5ee4101a00 2023-10-06T21:07:39.448 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.445+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ee402d3d0 con 0x7f5ed00a1500 2023-10-06T21:07:39.456 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.454+0000 7f5eda7fc700 1 -- 172.21.15.167:0/3402529259 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5ee402d3d0 con 0x7f5ed00a1500 2023-10-06T21:07:39.457 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 msgr2=0x7f5ed00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 0x7f5ed00a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0005950 tx=0x7f5ee00058e0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 msgr2=0x7f5ee4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000c00 tx=0x7f5ed400f710 comp rx=0 tx=0).stop 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 shutdown_connections 2023-10-06T21:07:39.459 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ee4106c60 0x7f5ee40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ee4101a00 0x7f5ee4101360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5ed00a1500 0x7f5ed00a39c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 --2- 172.21.15.167:0/3402529259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee4104330 0x7f5ee40ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 >> 172.21.15.167:0/3402529259 conn(0x7f5ee40fb3f0 msgr2=0x7f5ee4102690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 shutdown_connections 2023-10-06T21:07:39.460 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T21:07:39.456+0000 7f5eeb5c0700 1 -- 172.21.15.167:0/3402529259 wait complete. 2023-10-06T21:07:39.801 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-06T21:07:39.806 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-06T21:07:39.852 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-06T21:07:39.892 INFO:journalctl@ceph.mds.a.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:07:39.895 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-06T21:07:39.929 INFO:journalctl@ceph.mds.d.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:07:39.932 DEBUG:teuthology.orchestra.run.smithi078:mds.b> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-06T21:07:39.935 DEBUG:teuthology.orchestra.run.smithi078:mds.e> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-06T21:07:39.965 INFO:journalctl@ceph.mds.b.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:07:39.966 DEBUG:teuthology.orchestra.run.smithi078:mds.h> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-06T21:07:39.971 INFO:journalctl@ceph.mds.g.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:07:39.991 INFO:journalctl@ceph.mds.e.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:07:39.992 DEBUG:teuthology.orchestra.run.smithi078:mds.k> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-06T21:07:40.005 INFO:journalctl@ceph.mds.j.smithi026.stdout:-- Logs begin at Fri 2023-10-06 20:43:21 UTC. -- 2023-10-06T21:07:40.017 INFO:journalctl@ceph.mds.h.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:07:40.018 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-06T21:07:40.021 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-06T21:07:40.043 INFO:journalctl@ceph.mds.k.smithi078.stdout:-- Logs begin at Fri 2023-10-06 20:39:28 UTC. -- 2023-10-06T21:07:40.049 INFO:journalctl@ceph.mds.c.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:07:40.050 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-06T21:07:40.076 INFO:journalctl@ceph.mds.f.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:07:40.077 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-06T21:07:40.103 INFO:journalctl@ceph.mds.i.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:07:40.104 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-06T21:07:40.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:07:40.130 INFO:journalctl@ceph.mds.l.smithi167.stdout:-- Logs begin at Fri 2023-10-06 20:39:41 UTC. -- 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 -- 172.21.15.26:0/1541553393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 msgr2=0x7fab14102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/1541553393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab14102500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab04009990 tx=0x7fab04009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 -- 172.21.15.26:0/1541553393 shutdown_connections 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/1541553393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab14109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/1541553393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab141012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/1541553393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab14102500 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 -- 172.21.15.26:0/1541553393 >> 172.21.15.26:0/1541553393 conn(0x7fab140fc420 msgr2=0x7fab140fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 -- 172.21.15.26:0/1541553393 shutdown_connections 2023-10-06T21:07:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.315+0000 7fab1b9ff700 1 -- 172.21.15.26:0/1541553393 wait complete. 2023-10-06T21:07:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.316+0000 7fab1b9ff700 1 Processor -- start 2023-10-06T21:07:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.316+0000 7fab1b9ff700 1 -- start start 2023-10-06T21:07:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab14074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14073f80 con 0x7fab14102080 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140740f0 con 0x7fab14100ea0 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1b9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141a0c70 con 0x7fab14102a40 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1979b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab14074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52700/0 (socket says 172.21.15.26:52700) 2023-10-06T21:07:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 -- 172.21.15.26:0/3623666139 learned_addr learned my addr 172.21.15.26:0/3623666139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab1979b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab14074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33878/0 (socket says 172.21.15.26:33878) 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab19f9c700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 -- 172.21.15.26:0/3623666139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 msgr2=0x7fab140737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab140737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 -- 172.21.15.26:0/3623666139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 msgr2=0x7fab14074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab14074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 -- 172.21.15.26:0/3623666139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040095f0 con 0x7fab14102080 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab19f9c700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.317+0000 7fab18f9a700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b680 tx=0x7fab04005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.318+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0401d070 con 0x7fab14102080 2023-10-06T21:07:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.318+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab141a0ef0 con 0x7fab14102080 2023-10-06T21:07:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.318+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab04021410 con 0x7fab14102080 2023-10-06T21:07:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.318+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04017650 con 0x7fab14102080 2023-10-06T21:07:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.318+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab141a12b0 con 0x7fab14102080 2023-10-06T21:07:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.319+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab14107e50 con 0x7fab14102080 2023-10-06T21:07:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.321+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fab04021580 con 0x7fab14102080 2023-10-06T21:07:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.323+0000 7fab0a7fc700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 0x7fab000a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.323+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fab040ce450 con 0x7fab14102080 2023-10-06T21:07:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.323+0000 7fab1979b700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 0x7fab000a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.323+0000 7fab1979b700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 0x7fab000a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab14101bf0 tx=0x7fab1000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.325+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fab04097f40 con 0x7fab14102080 2023-10-06T21:07:40.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.538+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab1410c760 con 0x7fab14102080 2023-10-06T21:07:40.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.539+0000 7fab0a7fc700 1 -- 172.21.15.26:0/3623666139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18371 (secure 0 0 0) 0x7fab0409b560 con 0x7fab14102080 2023-10-06T21:07:40.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:40.542 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:07:29.352305+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:07:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 msgr2=0x7fab000a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 0x7fab000a3990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab14101bf0 tx=0x7fab1000b410 comp rx=0 tx=0).stop 2023-10-06T21:07:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 msgr2=0x7fab140732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b680 tx=0x7fab04005740 comp rx=0 tx=0).stop 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 shutdown_connections 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab14102a40 0x7fab140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab14100ea0 0x7fab14074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fab000a14d0 0x7fab000a3990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 --2- 172.21.15.26:0/3623666139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab14102080 0x7fab140732a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.541+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 >> 172.21.15.26:0/3623666139 conn(0x7fab140fc420 msgr2=0x7fab140fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.542+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 shutdown_connections 2023-10-06T21:07:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.542+0000 7fab1b9ff700 1 -- 172.21.15.26:0/3623666139 wait complete. 2023-10-06T21:07:40.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:07:40.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:07:40.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:40.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:40 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1182624810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 msgr2=0x7f8af41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1182624810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af41012c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80098f0 tx=0x7f8ae8009c00 comp rx=0 tx=0).stop 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1182624810 shutdown_connections 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1182624810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 0x7f8af4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1182624810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af41012c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1182624810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 0x7f8af4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1182624810 >> 172.21.15.26:0/1182624810 conn(0x7f8af40fc460 msgr2=0x7f8af40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:40.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1182624810 shutdown_connections 2023-10-06T21:07:40.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.770+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1182624810 wait complete. 2023-10-06T21:07:40.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.771+0000 7f8afb1ff700 1 Processor -- start 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.771+0000 7f8afb1ff700 1 -- start start 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 0x7f8af41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 0x7f8af41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41a0a30 con 0x7f8af4100ea0 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41a0b70 con 0x7f8af4102080 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8afb1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41a0d00 con 0x7f8af4102a40 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af979c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 0x7f8af41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52716/0 (socket says 172.21.15.26:52716) 2023-10-06T21:07:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 -- 172.21.15.26:0/1660049465 learned_addr learned my addr 172.21.15.26:0/1660049465 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af3fff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 0x7f8af41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 -- 172.21.15.26:0/1660049465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 msgr2=0x7f8af41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 0x7f8af41a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 -- 172.21.15.26:0/1660049465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 msgr2=0x7f8af41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 0x7f8af41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.772+0000 7f8af8f9b700 1 -- 172.21.15.26:0/1660049465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec009d00 con 0x7f8af4100ea0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8af8f9b700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009fd0 tx=0x7f8ae800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae801c070 con 0x7f8af4100ea0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae800f820 con 0x7f8af4100ea0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8009520 con 0x7f8af4100ea0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8003a70 con 0x7f8af4100ea0 2023-10-06T21:07:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.773+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41a11c0 con 0x7f8af4100ea0 2023-10-06T21:07:40.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.774+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af410c7d0 con 0x7f8af4100ea0 2023-10-06T21:07:40.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.776+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ae8020030 con 0x7f8af4100ea0 2023-10-06T21:07:40.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.778+0000 7f8af1ffb700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 0x7f8ae40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.778+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ae80cdc00 con 0x7f8af4100ea0 2023-10-06T21:07:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.778+0000 7f8af3fff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 0x7f8ae40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.779+0000 7f8af3fff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 0x7f8ae40a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8af4100bc0 tx=0x7f8ae0007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:40.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.781+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ae80976f0 con 0x7f8af4100ea0 2023-10-06T21:07:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:40 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:40 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:40.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.989+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af41a1350 con 0x7f8af4100ea0 2023-10-06T21:07:40.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.990+0000 7f8af1ffb700 1 -- 172.21.15.26:0/1660049465 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ae809ad10 con 0x7f8af4100ea0 2023-10-06T21:07:40.993 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:07:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 msgr2=0x7f8ae40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 0x7f8ae40a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8af4100bc0 tx=0x7f8ae0007400 comp rx=0 tx=0).stop 2023-10-06T21:07:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 msgr2=0x7f8af4198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009fd0 tx=0x7f8ae800b790 comp rx=0 tx=0).stop 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 shutdown_connections 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8af4102a40 0x7f8af41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8ae40a1480 0x7f8ae40a3940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100ea0 0x7f8af4198eb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 --2- 172.21.15.26:0/1660049465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af4102080 0x7f8af41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.992+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 >> 172.21.15.26:0/1660049465 conn(0x7f8af40fc460 msgr2=0x7f8af40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.993+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 shutdown_connections 2023-10-06T21:07:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:40.993+0000 7f8afb1ff700 1 -- 172.21.15.26:0/1660049465 wait complete. 2023-10-06T21:07:41.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 -- 172.21.15.26:0/3093223678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 msgr2=0x7f0700102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 --2- 172.21.15.26:0/3093223678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700102500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009990 tx=0x7f06f4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 -- 172.21.15.26:0/3093223678 shutdown_connections 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 --2- 172.21.15.26:0/3093223678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700102a40 0x7f0700109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 --2- 172.21.15.26:0/3093223678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700100ea0 0x7f07001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 --2- 172.21.15.26:0/3093223678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700102500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 -- 172.21.15.26:0/3093223678 >> 172.21.15.26:0/3093223678 conn(0x7f07000fc420 msgr2=0x7f07000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.215+0000 7f07061ff700 1 -- 172.21.15.26:0/3093223678 shutdown_connections 2023-10-06T21:07:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.216+0000 7f07061ff700 1 -- 172.21.15.26:0/3093223678 wait complete. 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.216+0000 7f07061ff700 1 Processor -- start 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 -- start start 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700100ea0 0x7f0700196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070019e770 con 0x7f0700102a40 2023-10-06T21:07:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070019e8e0 con 0x7f0700100ea0 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f07061ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070019eaa0 con 0x7f0700102080 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06feffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06feffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54898/0 (socket says 172.21.15.26:54898) 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39202/0 (socket says 172.21.15.26:39202) 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06feffd700 1 -- 172.21.15.26:0/650395026 learned_addr learned my addr 172.21.15.26:0/650395026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.217+0000 7f06ff7fe700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700100ea0 0x7f0700196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 -- 172.21.15.26:0/650395026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 msgr2=0x7f0700197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 -- 172.21.15.26:0/650395026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700100ea0 msgr2=0x7f0700196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700100ea0 0x7f0700196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 -- 172.21.15.26:0/650395026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f40095f0 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fffff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06f800efc0 tx=0x7f06f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8018070 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070019ed20 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06f800f040 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800cd40 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.218+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070019f180 con 0x7f0700102a40 2023-10-06T21:07:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.219+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070010c760 con 0x7f0700102a40 2023-10-06T21:07:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.221+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f06f8003a00 con 0x7f0700102a40 2023-10-06T21:07:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.222+0000 7f06fcff9700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 0x7f06e80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.223+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06f80cc500 con 0x7f0700102a40 2023-10-06T21:07:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.223+0000 7f06ff7fe700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 0x7f06e80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.223+0000 7f06ff7fe700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 0x7f06e80a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0700198100 tx=0x7f06f0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.227+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f06f8095ff0 con 0x7f0700102a40 2023-10-06T21:07:41.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.491+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0700107e20 con 0x7f0700102a40 2023-10-06T21:07:41.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.491+0000 7f06fcff9700 1 -- 172.21.15.26:0/650395026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f06f8099610 con 0x7f0700102a40 2023-10-06T21:07:41.494 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:41.494 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-06T21:07:41.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 msgr2=0x7f06e80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:41.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 0x7f06e80a3990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0700198100 tx=0x7f06f0009260 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 msgr2=0x7f070019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06f800efc0 tx=0x7f06f800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 shutdown_connections 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06e80a14d0 0x7f06e80a3990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0700102a40 0x7f070019e1a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0700100ea0 0x7f0700196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 --2- 172.21.15.26:0/650395026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0700102080 0x7f0700197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 >> 172.21.15.26:0/650395026 conn(0x7f07000fc420 msgr2=0x7f07000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 shutdown_connections 2023-10-06T21:07:41.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:41.494+0000 7f07061ff700 1 -- 172.21.15.26:0/650395026 wait complete. 2023-10-06T21:07:41.498 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 1 2023-10-06T21:07:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:41 smithi167 ceph-mon[134494]: Deploying daemon mds.l on smithi167 2023-10-06T21:07:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:41 smithi167 ceph-mon[134494]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:41 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3623666139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:41 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1660049465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:41.783 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:41 smithi167 systemd[1]: Starting Ceph mds.l for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:41 smithi026 ceph-mon[98490]: Deploying daemon mds.l on smithi167 2023-10-06T21:07:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:41 smithi026 ceph-mon[98490]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:41.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:41 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3623666139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:41.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:41 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1660049465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:41 smithi078 ceph-mon[134896]: Deploying daemon mds.l on smithi167 2023-10-06T21:07:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:41 smithi078 ceph-mon[134896]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:41 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3623666139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:07:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:41 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1660049465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:07:42.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:41 smithi167 podman[170057]: 2023-10-06T21:07:42.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:42 smithi167 bash[170057]: a1f7060d066f93cc98228e8a433c90b3acfbfbeccc580bc5a12be1acac4c190f 2023-10-06T21:07:42.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mds[170097]: -- 172.21.15.167:0/511583261 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fb01e51e0 con 0x558fb01ba800 2023-10-06T21:07:42.136 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-l[170073]: starting mds.l at 2023-10-06T21:07:42.460 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:07:42 smithi167 systemd[1]: Started Ceph mds.l for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:42.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/650395026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6832/4048987,v1:172.21.15.167:6833/4048987] up:boot 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: fsmap 1 up:standby 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:42 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/650395026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6832/4048987,v1:172.21.15.167:6833/4048987] up:boot 2023-10-06T21:07:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: fsmap 1 up:standby 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:42 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:42.863 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/650395026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:42.863 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6832/4048987,v1:172.21.15.167:6833/4048987] up:boot 2023-10-06T21:07:42.863 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: fsmap 1 up:standby 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:42.864 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:42 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:43.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 -- 172.21.15.26:0/4023197960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102080 msgr2=0x7f4f30102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 --2- 172.21.15.26:0/4023197960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102080 0x7f4f30102500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009990 tx=0x7f4f1c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 -- 172.21.15.26:0/4023197960 shutdown_connections 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 --2- 172.21.15.26:0/4023197960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102a40 0x7f4f30109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 --2- 172.21.15.26:0/4023197960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f301012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 --2- 172.21.15.26:0/4023197960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102080 0x7f4f30102500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 -- 172.21.15.26:0/4023197960 >> 172.21.15.26:0/4023197960 conn(0x7f4f300fc420 msgr2=0x7f4f300fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 -- 172.21.15.26:0/4023197960 shutdown_connections 2023-10-06T21:07:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.728+0000 7f4f36bff700 1 -- 172.21.15.26:0/4023197960 wait complete. 2023-10-06T21:07:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 Processor -- start 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 -- start start 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102080 0x7f4f30197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3019e770 con 0x7f4f30100ea0 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3019e8b0 con 0x7f4f30102a40 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.729+0000 7f4f36bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3019ea40 con 0x7f4f30102080 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3499b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 -- 172.21.15.26:0/1092085338 learned_addr learned my addr 172.21.15.26:0/1092085338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3499b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39214/0 (socket says 172.21.15.26:39214) 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f2ffff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102080 0x7f4f30197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 -- 172.21.15.26:0/1092085338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102080 msgr2=0x7f4f30197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102080 0x7f4f30197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 -- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 msgr2=0x7f4f30196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 -- 172.21.15.26:0/1092085338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0095f0 con 0x7f4f30102a40 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3499b700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f3519c700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f28007ee0 tx=0x7f4f2800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28017070 con 0x7f4f30102a40 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f2800f040 con 0x7f4f30102a40 2023-10-06T21:07:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28005740 con 0x7f4f30102a40 2023-10-06T21:07:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.730+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3019ed20 con 0x7f4f30102a40 2023-10-06T21:07:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.731+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3019f1d0 con 0x7f4f30102a40 2023-10-06T21:07:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.732+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30108460 con 0x7f4f30102a40 2023-10-06T21:07:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.734+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4f2800dc60 con 0x7f4f30102a40 2023-10-06T21:07:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.735+0000 7f4f2dffb700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 0x7f4f200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.735+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f280cbff0 con 0x7f4f30102a40 2023-10-06T21:07:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.735+0000 7f4f3499b700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 0x7f4f200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.736+0000 7f4f3499b700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 0x7f4f200a3a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f30101bf0 tx=0x7f4f24009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.737+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4f28095a90 con 0x7f4f30102a40 2023-10-06T21:07:43.753 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:43 smithi078 ceph-mon[134896]: Deploying daemon mds.k on smithi078 2023-10-06T21:07:43.753 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:43 smithi078 ceph-mon[134896]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:43.753 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:43 smithi078 systemd[1]: Starting Ceph mds.k for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:43.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:43 smithi167 ceph-mon[134494]: Deploying daemon mds.k on smithi078 2023-10-06T21:07:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:43 smithi167 ceph-mon[134494]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:43 smithi026 ceph-mon[98490]: Deploying daemon mds.k on smithi078 2023-10-06T21:07:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:43 smithi026 ceph-mon[98490]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.998+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f3002d230 con 0x7f4f30102a40 2023-10-06T21:07:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:43.999+0000 7f4f2dffb700 1 -- 172.21.15.26:0/1092085338 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1319 (secure 0 0 0) 0x7f4f280990b0 con 0x7f4f30102a40 2023-10-06T21:07:44.002 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:44.002 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-06T21:07:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.001+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 msgr2=0x7f4f200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.001+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 0x7f4f200a3a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f30101bf0 tx=0x7f4f24009260 comp rx=0 tx=0).stop 2023-10-06T21:07:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.001+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 msgr2=0x7f4f3019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.001+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f28007ee0 tx=0x7f4f2800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.001+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 shutdown_connections 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f30102a40 0x7f4f3019e1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f200a1550 0x7f4f200a3a10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100ea0 0x7f4f30196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 --2- 172.21.15.26:0/1092085338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f30102080 0x7f4f30197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 >> 172.21.15.26:0/1092085338 conn(0x7f4f300fc420 msgr2=0x7f4f301055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 shutdown_connections 2023-10-06T21:07:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:44.002+0000 7f4f36bff700 1 -- 172.21.15.26:0/1092085338 wait complete. 2023-10-06T21:07:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2023-10-06T21:07:44.297 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:44 smithi078 podman[170833]: 2023-10-06T21:07:44.297 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:44 smithi078 bash[170833]: dcf57726f0ea6086f119e441b010391b5a8470ab9030491ceb193b44ffd6a66d 2023-10-06T21:07:44.620 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mds[170873]: -- 172.21.15.78:0/519754606 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0bd7531e0 con 0x55d0bd729000 2023-10-06T21:07:44.621 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-k[170849]: starting mds.k at 2023-10-06T21:07:44.621 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:07:44 smithi078 systemd[1]: Started Ceph mds.k for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1092085338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:44.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:44.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1092085338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1092085338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:44.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:44.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:44.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:45.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:45 smithi026 ceph-mon[98490]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:45.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:45 smithi026 ceph-mon[98490]: Deploying daemon mds.j on smithi026 2023-10-06T21:07:45.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:45 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6832/3813856521,v1:172.21.15.78:6833/3813856521] up:boot 2023-10-06T21:07:45.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:45 smithi026 ceph-mon[98490]: fsmap 2 up:standby 2023-10-06T21:07:45.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:07:45.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:45 smithi167 ceph-mon[134494]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:45 smithi167 ceph-mon[134494]: Deploying daemon mds.j on smithi026 2023-10-06T21:07:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:45 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6832/3813856521,v1:172.21.15.78:6833/3813856521] up:boot 2023-10-06T21:07:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:45 smithi167 ceph-mon[134494]: fsmap 2 up:standby 2023-10-06T21:07:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:07:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:45 smithi078 ceph-mon[134896]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:45 smithi078 ceph-mon[134896]: Deploying daemon mds.j on smithi026 2023-10-06T21:07:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:45 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6832/3813856521,v1:172.21.15.78:6833/3813856521] up:boot 2023-10-06T21:07:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:45 smithi078 ceph-mon[134896]: fsmap 2 up:standby 2023-10-06T21:07:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:07:46.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 -- 172.21.15.26:0/4013979214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0073910 msgr2=0x7f9ed0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/4013979214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0073910 0x7f9ed0073d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009990 tx=0x7f9ebc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 -- 172.21.15.26:0/4013979214 shutdown_connections 2023-10-06T21:07:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/4013979214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0108db0 0x7f9ed010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/4013979214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/4013979214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0073910 0x7f9ed0073d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 -- 172.21.15.26:0/4013979214 >> 172.21.15.26:0/4013979214 conn(0x7f9ed00fc420 msgr2=0x7f9ed00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 -- 172.21.15.26:0/4013979214 shutdown_connections 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.463+0000 7f9ed71ff700 1 -- 172.21.15.26:0/4013979214 wait complete. 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.465+0000 7f9ed71ff700 1 Processor -- start 2023-10-06T21:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.465+0000 7f9ed71ff700 1 -- start start 2023-10-06T21:07:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.465+0000 7f9ed71ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.465+0000 7f9ed71ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.465+0000 7f9ed71ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0108db0 0x7f9ed01a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed71ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a0960 con 0x7f9ed0074ef0 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed71ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a0aa0 con 0x7f9ed0108db0 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed71ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a0c30 con 0x7f9ed0073910 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54930/0 (socket says 172.21.15.26:54930) 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed579c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0108db0 0x7f9ed01a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 -- 172.21.15.26:0/2736642976 learned_addr learned my addr 172.21.15.26:0/2736642976 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39224/0 (socket says 172.21.15.26:39224) 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 -- 172.21.15.26:0/2736642976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0108db0 msgr2=0x7f9ed01a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0108db0 0x7f9ed01a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 -- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 msgr2=0x7f9ed0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ed4f9b700 1 -- 172.21.15.26:0/2736642976 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0095f0 con 0x7f9ed0073910 2023-10-06T21:07:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.466+0000 7f9ecffff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.467+0000 7f9ed4f9b700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9a0 tx=0x7f9ec400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.468+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4009790 con 0x7f9ed0073910 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.468+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec4010430 con 0x7f9ed0073910 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.468+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed01a0f10 con 0x7f9ed0073910 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.468+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4018680 con 0x7f9ed0073910 2023-10-06T21:07:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.468+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed01a1340 con 0x7f9ed0073910 2023-10-06T21:07:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.469+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed0107460 con 0x7f9ed0073910 2023-10-06T21:07:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.471+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9ec40105a0 con 0x7f9ed0073910 2023-10-06T21:07:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.472+0000 7f9ecdffb700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 0x7f9ec00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.472+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ec4014070 con 0x7f9ed0073910 2023-10-06T21:07:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.473+0000 7f9ecffff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 0x7f9ec00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.473+0000 7f9ecffff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 0x7f9ec00a3a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b2e0 tx=0x7f9ebc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:46.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.477+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9ec4096480 con 0x7f9ed0073910 2023-10-06T21:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.751+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed002d020 con 0x7f9ed0073910 2023-10-06T21:07:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.753+0000 7f9ecdffb700 1 -- 172.21.15.26:0/2736642976 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2110 (secure 0 0 0) 0x7f9ec4099aa0 con 0x7f9ed0073910 2023-10-06T21:07:46.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:46.755 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 msgr2=0x7f9ec00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 0x7f9ec00a3a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b2e0 tx=0x7f9ebc005fb0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 msgr2=0x7f9ed0198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9a0 tx=0x7f9ec400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 shutdown_connections 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed0108db0 0x7f9ed01a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed0073910 0x7f9ed0198de0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9ec00a1550 0x7f9ec00a3a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 --2- 172.21.15.26:0/2736642976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0074ef0 0x7f9ed0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 >> 172.21.15.26:0/2736642976 conn(0x7f9ed00fc420 msgr2=0x7f9ed00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 shutdown_connections 2023-10-06T21:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:46.755+0000 7f9ed71ff700 1 -- 172.21.15.26:0/2736642976 wait complete. 2023-10-06T21:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2023-10-06T21:07:47.073 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:46 smithi026 systemd[1]: Starting Ceph mds.j for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:47.376 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:47 smithi026 podman[148530]: 2023-10-06T21:07:47.376 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:47 smithi026 bash[148530]: 693fc5ce939d30780c7b7116394a41347098d44ee443ced8c17cf8d09522947a 2023-10-06T21:07:47.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:47 smithi167 ceph-mon[134494]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:47.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:47 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2736642976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:47 smithi026 ceph-mon[98490]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:47.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:47 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2736642976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:47.823 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:47 smithi026 ceph-mds[148570]: -- 172.21.15.26:0/937582585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561893f9b1e0 con 0x561893f71000 2023-10-06T21:07:47.824 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:47 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-j[148546]: starting mds.j at 2023-10-06T21:07:47.824 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:07:47 smithi026 systemd[1]: Started Ceph mds.j for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:47.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:47 smithi078 ceph-mon[134896]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:47 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2736642976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:48.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6834/4105384407,v1:172.21.15.26:6835/4105384407] up:boot 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: fsmap 3 up:standby 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:48.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:48 smithi026 ceph-mon[98490]: Deploying daemon mds.d on smithi026 2023-10-06T21:07:48.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6834/4105384407,v1:172.21.15.26:6835/4105384407] up:boot 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: fsmap 3 up:standby 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:48.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:48.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:48.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:48 smithi167 ceph-mon[134494]: Deploying daemon mds.d on smithi026 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6834/4105384407,v1:172.21.15.26:6835/4105384407] up:boot 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: fsmap 3 up:standby 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:48.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:48.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:48 smithi078 ceph-mon[134896]: Deploying daemon mds.d on smithi026 2023-10-06T21:07:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/81008002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84104270 msgr2=0x7f8b84106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/81008002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84104270 0x7f8b84106660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009990 tx=0x7f8b78009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/81008002 shutdown_connections 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/81008002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84106ba0 0x7f8b84109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/81008002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84101940 0x7f8b84103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/81008002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84104270 0x7f8b84106660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/81008002 >> 172.21.15.26:0/81008002 conn(0x7f8b840fb310 msgr2=0x7f8b840fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.991+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/81008002 shutdown_connections 2023-10-06T21:07:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.992+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/81008002 wait complete. 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.992+0000 7f8b8a5ff700 1 Processor -- start 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 -- start start 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84101940 0x7f8b84198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84104270 0x7f8b84199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841a0880 con 0x7f8b84106ba0 2023-10-06T21:07:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841a09f0 con 0x7f8b84104270 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b8a5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841a0bb0 con 0x7f8b84101940 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b88b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b88b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39260/0 (socket says 172.21.15.26:39260) 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b88b9c700 1 -- 172.21.15.26:0/3238073334 learned_addr learned my addr 172.21.15.26:0/3238073334 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b83fff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84101940 0x7f8b84198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.993+0000 7f8b837fe700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84104270 0x7f8b84199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 -- 172.21.15.26:0/3238073334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84101940 msgr2=0x7f8b84198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84101940 0x7f8b84198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 -- 172.21.15.26:0/3238073334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84104270 msgr2=0x7f8b84199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84104270 0x7f8b84199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 -- 172.21.15.26:0/3238073334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780095f0 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b88b9c700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00ef30 tx=0x7f8b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c018070 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b841a0e30 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00cd00 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.994+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b841a1200 con 0x7f8b84106ba0 2023-10-06T21:07:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.995+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b84108770 con 0x7f8b84106ba0 2023-10-06T21:07:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.997+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8b7c003a00 con 0x7f8b84106ba0 2023-10-06T21:07:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.999+0000 7f8b817fa700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 0x7f8b6c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.999+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b7c0cc410 con 0x7f8b84106ba0 2023-10-06T21:07:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:48.999+0000 7f8b83fff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 0x7f8b6c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.000+0000 7f8b83fff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 0x7f8b6c0a3990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b8419a210 tx=0x7f8b74009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:49.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.003+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8b7c095f00 con 0x7f8b84106ba0 2023-10-06T21:07:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.283+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b8410c6b0 con 0x7f8b84106ba0 2023-10-06T21:07:49.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.284+0000 7f8b817fa700 1 -- 172.21.15.26:0/3238073334 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2901 (secure 0 0 0) 0x7f8b7c099520 con 0x7f8b84106ba0 2023-10-06T21:07:49.287 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:49.287 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:07:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 msgr2=0x7f8b6c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 0x7f8b6c0a3990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b8419a210 tx=0x7f8b74009260 comp rx=0 tx=0).stop 2023-10-06T21:07:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 msgr2=0x7f8b841a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00ef30 tx=0x7f8b7c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 shutdown_connections 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8b6c0a14d0 0x7f8b6c0a3990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b84106ba0 0x7f8b841a02b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b84101940 0x7f8b84198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 --2- 172.21.15.26:0/3238073334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b84104270 0x7f8b84199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.286+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 >> 172.21.15.26:0/3238073334 conn(0x7f8b840fb310 msgr2=0x7f8b840fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.287+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 shutdown_connections 2023-10-06T21:07:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:49.287+0000 7f8b8a5ff700 1 -- 172.21.15.26:0/3238073334 wait complete. 2023-10-06T21:07:49.291 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2023-10-06T21:07:49.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:49 smithi026 ceph-mon[98490]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:49.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:49 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3238073334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:49.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:49 smithi167 ceph-mon[134494]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:49.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:49 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3238073334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:49 smithi078 ceph-mon[134896]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:49 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3238073334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:50.073 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:49 smithi026 systemd[1]: Starting Ceph mds.d for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:50.435 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:50 smithi026 podman[148918]: 2023-10-06T21:07:50.436 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:50 smithi026 bash[148918]: 88b0cef1984d7e8e03e395149305da384698e944aa10296dccc53d0adfea6112 2023-10-06T21:07:50.436 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:50 smithi026 ceph-mds[148958]: -- 172.21.15.26:0/3248279824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596dd4671e0 con 0x5596dd43c800 2023-10-06T21:07:50.436 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:50 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-d[148934]: starting mds.d at 2023-10-06T21:07:50.823 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:07:50 smithi026 systemd[1]: Started Ceph mds.d for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:51.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 -- 172.21.15.26:0/1536735009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 msgr2=0x7feb94109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 --2- 172.21.15.26:0/1536735009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb94109360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b0a0 tx=0x7feb8c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 -- 172.21.15.26:0/1536735009 shutdown_connections 2023-10-06T21:07:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 --2- 172.21.15.26:0/1536735009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb94109360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 --2- 172.21.15.26:0/1536735009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 --2- 172.21.15.26:0/1536735009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 0x7feb94102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 -- 172.21.15.26:0/1536735009 >> 172.21.15.26:0/1536735009 conn(0x7feb940fc420 msgr2=0x7feb940fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 -- 172.21.15.26:0/1536735009 shutdown_connections 2023-10-06T21:07:51.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.529+0000 7feb995ff700 1 -- 172.21.15.26:0/1536735009 wait complete. 2023-10-06T21:07:51.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.531+0000 7feb995ff700 1 Processor -- start 2023-10-06T21:07:51.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.531+0000 7feb995ff700 1 -- start start 2023-10-06T21:07:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 0x7feb940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94073f80 con 0x7feb94100ea0 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940740f0 con 0x7feb94102a40 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb995ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb941a0d30 con 0x7feb94102080 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36572/0 (socket says 172.21.15.26:36572) 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 -- 172.21.15.26:0/1757464832 learned_addr learned my addr 172.21.15.26:0/1757464832 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb927fc700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 0x7feb940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb937fe700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 -- 172.21.15.26:0/1757464832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 msgr2=0x7feb940732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 0x7feb940732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 -- 172.21.15.26:0/1757464832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 msgr2=0x7feb940737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb940737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.532+0000 7feb92ffd700 1 -- 172.21.15.26:0/1757464832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c009d00 con 0x7feb94100ea0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb92ffd700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb8400e970 tx=0x7feb8400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84009790 con 0x7feb94100ea0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb8400c7e0 con 0x7feb94100ea0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84003c70 con 0x7feb94100ea0 2023-10-06T21:07:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb941a0fb0 con 0x7feb94100ea0 2023-10-06T21:07:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.533+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb941a1380 con 0x7feb94100ea0 2023-10-06T21:07:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.534+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9410c7d0 con 0x7feb94100ea0 2023-10-06T21:07:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.536+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feb8400cc50 con 0x7feb94100ea0 2023-10-06T21:07:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.537+0000 7feb7bfff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 0x7feb7c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.537+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb84014070 con 0x7feb94100ea0 2023-10-06T21:07:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.538+0000 7feb927fc700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 0x7feb7c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.538+0000 7feb927fc700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 0x7feb7c0a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feb94100bc0 tx=0x7feb8800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.541+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feb84096530 con 0x7feb94100ea0 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6836/2894864282,v1:172.21.15.26:6837/2894864282] up:boot 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: fsmap 4 up:standby 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:51.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:51 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:51.794 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:51.794 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6836/2894864282,v1:172.21.15.26:6837/2894864282] up:boot 2023-10-06T21:07:51.794 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: fsmap 4 up:standby 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:51.795 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:51 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.800+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb941a18e0 con 0x7feb94100ea0 2023-10-06T21:07:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.801+0000 7feb7bfff700 1 -- 172.21.15.26:0/1757464832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7feb84003dd0 con 0x7feb94100ea0 2023-10-06T21:07:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:51.804 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-06T21:07:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 msgr2=0x7feb7c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 0x7feb7c0a3990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feb94100bc0 tx=0x7feb8800b410 comp rx=0 tx=0).stop 2023-10-06T21:07:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 msgr2=0x7feb94074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:51.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb8400e970 tx=0x7feb8400ec80 comp rx=0 tx=0).stop 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 shutdown_connections 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb94102a40 0x7feb940737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7feb7c0a14d0 0x7feb7c0a3990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb94100ea0 0x7feb94074c60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 --2- 172.21.15.26:0/1757464832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb94102080 0x7feb940732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.803+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 >> 172.21.15.26:0/1757464832 conn(0x7feb940fc420 msgr2=0x7feb940fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.804+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 shutdown_connections 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:51.804+0000 7feb995ff700 1 -- 172.21.15.26:0/1757464832 wait complete. 2023-10-06T21:07:51.807 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 5 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6836/2894864282,v1:172.21.15.26:6837/2894864282] up:boot 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: fsmap 4 up:standby 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:51.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:51.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:51.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:51 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:52.371 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 systemd[1]: Starting Ceph mds.e for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:52.678 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 podman[171176]: 2023-10-06T21:07:52.678 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:52 smithi078 ceph-mon[134896]: Deploying daemon mds.e on smithi078 2023-10-06T21:07:52.679 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:52 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1757464832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:52 smithi167 ceph-mon[134494]: Deploying daemon mds.e on smithi078 2023-10-06T21:07:52.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:52 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1757464832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:52 smithi026 ceph-mon[98490]: Deploying daemon mds.e on smithi078 2023-10-06T21:07:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:52 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1757464832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:53.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 ceph-mds[171216]: -- 172.21.15.78:0/2204394483 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648743b71e0 con 0x56487438cc00 2023-10-06T21:07:53.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 bash[171176]: 573aac2361ab5c6c7523504e08c7ee932901e07489beb52b8cd256458d9c1b06 2023-10-06T21:07:53.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: starting mds.e at 2023-10-06T21:07:53.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:07:52 smithi078 systemd[1]: Started Ceph mds.e for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:53.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:53 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:53.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:53.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:53.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:53 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:53.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:53.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:53 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 -- 172.21.15.26:0/1765793830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 msgr2=0x7fe03c103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 --2- 172.21.15.26:0/1765793830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c103d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe038009990 tx=0x7fe038009ca0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 -- 172.21.15.26:0/1765793830 shutdown_connections 2023-10-06T21:07:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 --2- 172.21.15.26:0/1765793830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 --2- 172.21.15.26:0/1765793830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c103d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 --2- 172.21.15.26:0/1765793830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 -- 172.21.15.26:0/1765793830 >> 172.21.15.26:0/1765793830 conn(0x7fe03c0fb3b0 msgr2=0x7fe03c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 -- 172.21.15.26:0/1765793830 shutdown_connections 2023-10-06T21:07:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.036+0000 7fe0439ff700 1 -- 172.21.15.26:0/1765793830 wait complete. 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.037+0000 7fe0439ff700 1 Processor -- start 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.037+0000 7fe0439ff700 1 -- start start 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.037+0000 7fe0439ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe0439ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe0439ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe0439ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c100630 con 0x7fe03c106c00 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe0439ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1007a0 con 0x7fe03c1019a0 2023-10-06T21:07:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe0439ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1a0b80 con 0x7fe03c1042d0 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe041f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe04179b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe041f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36588/0 (socket says 172.21.15.26:36588) 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe041f9c700 1 -- 172.21.15.26:0/2772612947 learned_addr learned my addr 172.21.15.26:0/2772612947 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 -- 172.21.15.26:0/2772612947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 msgr2=0x7fe03c101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 -- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 msgr2=0x7fe03c0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 -- 172.21.15.26:0/2772612947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034009c40 con 0x7fe03c1042d0 2023-10-06T21:07:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe041f9c700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.038+0000 7fe040f9a700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c0ff950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e9f0 tx=0x7fe02c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.039+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c00c9c0 con 0x7fe03c1042d0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.039+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0380095f0 con 0x7fe03c1042d0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.039+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe02c010430 con 0x7fe03c1042d0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.039+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c018680 con 0x7fe03c1042d0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.039+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c1a1190 con 0x7fe03c1042d0 2023-10-06T21:07:54.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.040+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c0fcf30 con 0x7fe03c1042d0 2023-10-06T21:07:54.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.042+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe02c010860 con 0x7fe03c1042d0 2023-10-06T21:07:54.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.043+0000 7fe0327fc700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 0x7fe0280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:54.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.043+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe02c014070 con 0x7fe03c1042d0 2023-10-06T21:07:54.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.043+0000 7fe04179b700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 0x7fe0280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:54.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.044+0000 7fe04179b700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 0x7fe0280a3990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b2e0 tx=0x7fe038005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:54.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.045+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe02c096450 con 0x7fe03c1042d0 2023-10-06T21:07:54.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.301+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe03c02d270 con 0x7fe03c1042d0 2023-10-06T21:07:54.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.302+0000 7fe0327fc700 1 -- 172.21.15.26:0/2772612947 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7fe02c099a70 con 0x7fe03c1042d0 2023-10-06T21:07:54.304 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:54.305 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-06T21:07:54.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 msgr2=0x7fe0280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:54.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 0x7fe0280a3990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b2e0 tx=0x7fe038005fb0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 msgr2=0x7fe03c0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:54.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c0ff950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00e9f0 tx=0x7fe02c00ed00 comp rx=0 tx=0).stop 2023-10-06T21:07:54.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 shutdown_connections 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0280a14d0 0x7fe0280a3990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe03c106c00 0x7fe03c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe03c1019a0 0x7fe03c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 --2- 172.21.15.26:0/2772612947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c1042d0 0x7fe03c0ff950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.304+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 >> 172.21.15.26:0/2772612947 conn(0x7fe03c0fb3b0 msgr2=0x7fe03c102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.305+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 shutdown_connections 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:54.305+0000 7fe0439ff700 1 -- 172.21.15.26:0/2772612947 wait complete. 2023-10-06T21:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 6 2023-10-06T21:07:54.502 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 systemd[1]: Starting Ceph mds.f for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:54.782 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 podman[170406]: 2023-10-06T21:07:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mon[134494]: Deploying daemon mds.f on smithi167 2023-10-06T21:07:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6834/4281425672,v1:172.21.15.78:6835/4281425672] up:boot 2023-10-06T21:07:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mon[134494]: fsmap 5 up:standby 2023-10-06T21:07:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:07:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2772612947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:54 smithi026 ceph-mon[98490]: Deploying daemon mds.f on smithi167 2023-10-06T21:07:54.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:54 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6834/4281425672,v1:172.21.15.78:6835/4281425672] up:boot 2023-10-06T21:07:54.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:54 smithi026 ceph-mon[98490]: fsmap 5 up:standby 2023-10-06T21:07:54.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:54 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:07:54.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:54 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2772612947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:54 smithi078 ceph-mon[134896]: Deploying daemon mds.f on smithi167 2023-10-06T21:07:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:54 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6834/4281425672,v1:172.21.15.78:6835/4281425672] up:boot 2023-10-06T21:07:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:54 smithi078 ceph-mon[134896]: fsmap 5 up:standby 2023-10-06T21:07:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:54 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:07:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:54 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2772612947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:55.282 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-mds[170446]: -- 172.21.15.167:0/437260282 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621ac0bd1e0 con 0x5621ac093000 2023-10-06T21:07:55.283 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-f[170422]: starting mds.f at 2023-10-06T21:07:55.283 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 bash[170406]: 7bfd323c5f0c31f8ead93e98c76110675a548ff48e2e67f0e507d4cf9f4a6050 2023-10-06T21:07:55.283 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:07:54 smithi167 systemd[1]: Started Ceph mds.f for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:55 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:55.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:55.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:55 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:55 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:56.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:56.477 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:56 smithi167 systemd[1]: Starting Ceph mds.c for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 -- 172.21.15.26:0/3149878789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 msgr2=0x7f6288109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 --2- 172.21.15.26:0/3149878789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f6288109360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f628000b0a0 tx=0x7f628000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 -- 172.21.15.26:0/3149878789 shutdown_connections 2023-10-06T21:07:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 --2- 172.21.15.26:0/3149878789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f6288109360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 --2- 172.21.15.26:0/3149878789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288100ea0 0x7f62881012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 --2- 172.21.15.26:0/3149878789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288102080 0x7f6288102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 -- 172.21.15.26:0/3149878789 >> 172.21.15.26:0/3149878789 conn(0x7f62880fc420 msgr2=0x7f62880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:56.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 -- 172.21.15.26:0/3149878789 shutdown_connections 2023-10-06T21:07:56.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.540+0000 7f628cbff700 1 -- 172.21.15.26:0/3149878789 wait complete. 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.541+0000 7f628cbff700 1 Processor -- start 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.541+0000 7f628cbff700 1 -- start start 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 0x7f6288198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288102080 0x7f6288199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f62881a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881a0a80 con 0x7f6288100ea0 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881a0bc0 con 0x7f6288102080 2023-10-06T21:07:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628cbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881a0d50 con 0x7f6288102a40 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 0x7f6288198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f6286d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f62881a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f6286d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f62881a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45742/0 (socket says 172.21.15.26:45742) 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f6286d9d700 1 -- 172.21.15.26:0/2084841805 learned_addr learned my addr 172.21.15.26:0/2084841805 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f6285d9b700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288102080 0x7f6288199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 -- 172.21.15.26:0/2084841805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 msgr2=0x7f62881a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f62881a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 -- 172.21.15.26:0/2084841805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288102080 msgr2=0x7f6288199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288102080 0x7f6288199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.542+0000 7f628659c700 1 -- 172.21.15.26:0/2084841805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280009d00 con 0x7f6288100ea0 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f628659c700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 0x7f6288198f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f627800d720 tx=0x7f627800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627800f690 con 0x7f6288100ea0 2023-10-06T21:07:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62881a0fd0 con 0x7f6288100ea0 2023-10-06T21:07:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f627800fcb0 con 0x7f6288100ea0 2023-10-06T21:07:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278010e60 con 0x7f6288100ea0 2023-10-06T21:07:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.543+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62881a1450 con 0x7f6288100ea0 2023-10-06T21:07:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.544+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628810c760 con 0x7f6288100ea0 2023-10-06T21:07:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.546+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f627800f7f0 con 0x7f6288100ea0 2023-10-06T21:07:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.549+0000 7f62777fe700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 0x7f62700a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.549+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62780cc8a0 con 0x7f6288100ea0 2023-10-06T21:07:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.549+0000 7f6285d9b700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 0x7f62700a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.550+0000 7f6285d9b700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 0x7f62700a3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6288100bc0 tx=0x7f627c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:56.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.552+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6278096300 con 0x7f6288100ea0 2023-10-06T21:07:56.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:56 smithi167 ceph-mon[134494]: Deploying daemon mds.c on smithi167 2023-10-06T21:07:56.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:56 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6834/3142312308,v1:172.21.15.167:6835/3142312308] up:boot 2023-10-06T21:07:56.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:56 smithi167 ceph-mon[134494]: fsmap 6 up:standby 2023-10-06T21:07:56.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:56 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:07:56.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.811+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f628810c970 con 0x7f6288100ea0 2023-10-06T21:07:56.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.812+0000 7f62777fe700 1 -- 172.21.15.26:0/2084841805 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5277 (secure 0 0 0) 0x7f6278099920 con 0x7f6288100ea0 2023-10-06T21:07:56.815 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:56.815 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-06T21:07:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.814+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 msgr2=0x7f62700a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.814+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 0x7f62700a3990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6288100bc0 tx=0x7f627c009260 comp rx=0 tx=0).stop 2023-10-06T21:07:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.814+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 msgr2=0x7f6288198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:56.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.814+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 0x7f6288198f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f627800d720 tx=0x7f627800da30 comp rx=0 tx=0).stop 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 shutdown_connections 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6288102a40 0x7f62881a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f62700a14d0 0x7f62700a3990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6288100ea0 0x7f6288198f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 --2- 172.21.15.26:0/2084841805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6288102080 0x7f6288199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 >> 172.21.15.26:0/2084841805 conn(0x7f62880fc420 msgr2=0x7f62881079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 shutdown_connections 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:56.815+0000 7f628cbff700 1 -- 172.21.15.26:0/2084841805 wait complete. 2023-10-06T21:07:56.818 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2023-10-06T21:07:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:56 smithi026 ceph-mon[98490]: Deploying daemon mds.c on smithi167 2023-10-06T21:07:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:56 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6834/3142312308,v1:172.21.15.167:6835/3142312308] up:boot 2023-10-06T21:07:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:56 smithi026 ceph-mon[98490]: fsmap 6 up:standby 2023-10-06T21:07:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:56 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:07:56.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:56 smithi078 ceph-mon[134896]: Deploying daemon mds.c on smithi167 2023-10-06T21:07:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:56 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6834/3142312308,v1:172.21.15.167:6835/3142312308] up:boot 2023-10-06T21:07:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:56 smithi078 ceph-mon[134896]: fsmap 6 up:standby 2023-10-06T21:07:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:56 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:07:57.032 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:56 smithi167 podman[170745]: 2023-10-06T21:07:57.506 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mds[170785]: -- 172.21.15.167:0/2113110682 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a1f34d11e0 con 0x55a1f34a7000 2023-10-06T21:07:57.507 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: starting mds.c at 2023-10-06T21:07:57.507 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:57 smithi167 bash[170745]: fa74e3dda64c4b0e9bcea065c1af140bb97776f4c7c54f21237d20cb8138598f 2023-10-06T21:07:57.507 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:07:57 smithi167 systemd[1]: Started Ceph mds.c for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:07:57.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2084841805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:57 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:57.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2084841805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:57 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:57.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:57.846 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2084841805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:07:57.847 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:57 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:07:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:58 smithi167 ceph-mon[134494]: Deploying daemon mds.h on smithi078 2023-10-06T21:07:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:58 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6836/2946984451,v1:172.21.15.167:6837/2946984451] up:boot 2023-10-06T21:07:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:58 smithi167 ceph-mon[134494]: fsmap 7 up:standby 2023-10-06T21:07:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:58 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:07:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:58 smithi167 ceph-mon[134494]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:58.785 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:58 smithi078 ceph-mon[134896]: Deploying daemon mds.h on smithi078 2023-10-06T21:07:58.785 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:58 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6836/2946984451,v1:172.21.15.167:6837/2946984451] up:boot 2023-10-06T21:07:58.785 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:58 smithi078 ceph-mon[134896]: fsmap 7 up:standby 2023-10-06T21:07:58.786 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:58 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:07:58.786 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:58 smithi078 ceph-mon[134896]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:58 smithi026 ceph-mon[98490]: Deploying daemon mds.h on smithi078 2023-10-06T21:07:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:58 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6836/2946984451,v1:172.21.15.167:6837/2946984451] up:boot 2023-10-06T21:07:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:58 smithi026 ceph-mon[98490]: fsmap 7 up:standby 2023-10-06T21:07:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:58 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:07:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:58 smithi026 ceph-mon[98490]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:07:58.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:07:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 -- 172.21.15.26:0/1132403675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0102a40 msgr2=0x7fe9b0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/1132403675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0102a40 0x7fe9b0109360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b0a0 tx=0x7fe9a800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 -- 172.21.15.26:0/1132403675 shutdown_connections 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/1132403675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0102a40 0x7fe9b0109360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/1132403675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0100ea0 0x7fe9b01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/1132403675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102080 0x7fe9b0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 -- 172.21.15.26:0/1132403675 >> 172.21.15.26:0/1132403675 conn(0x7fe9b00fc420 msgr2=0x7fe9b00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 -- 172.21.15.26:0/1132403675 shutdown_connections 2023-10-06T21:07:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.048+0000 7fe9b5bff700 1 -- 172.21.15.26:0/1132403675 wait complete. 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.049+0000 7fe9b5bff700 1 Processor -- start 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 -- start start 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 0x7fe9b0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102a40 0x7fe9b01a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01a0a70 con 0x7fe9b0102a40 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01a0bb0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9b5bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b01a0d40 con 0x7fe9b0102080 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9af7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9af7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49022/0 (socket says 172.21.15.26:49022) 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9af7fe700 1 -- 172.21.15.26:0/2511659858 learned_addr learned my addr 172.21.15.26:0/2511659858 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9affff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102a40 0x7fe9b01a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9aeffd700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 0x7fe9b0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9af7fe700 1 -- 172.21.15.26:0/2511659858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 msgr2=0x7fe9b0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.050+0000 7fe9af7fe700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 0x7fe9b0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9af7fe700 1 -- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102a40 msgr2=0x7fe9b01a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9af7fe700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102a40 0x7fe9b01a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9af7fe700 1 -- 172.21.15.26:0/2511659858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8009d00 con 0x7fe9b0100ea0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9aeffd700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 0x7fe9b0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9af7fe700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000e970 tx=0x7fe9a000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0009790 con 0x7fe9b0100ea0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9a000eec0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b01a0fc0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0003cf0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.051+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b01a1490 con 0x7fe9b0100ea0 2023-10-06T21:07:59.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.052+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b010c760 con 0x7fe9b0100ea0 2023-10-06T21:07:59.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.054+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe9a000ccd0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.055+0000 7fe9acff9700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 0x7fe9980a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:07:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.056+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9a0014070 con 0x7fe9b0100ea0 2023-10-06T21:07:59.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.056+0000 7fe9aeffd700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 0x7fe9980a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:07:59.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.056+0000 7fe9aeffd700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 0x7fe9980a3a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0100bc0 tx=0x7fe9a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:07:59.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.059+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe9a0096560 con 0x7fe9b0100ea0 2023-10-06T21:07:59.115 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:58 smithi078 systemd[1]: Starting Ceph mds.h for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.319+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9b010c9a0 con 0x7fe9b0100ea0 2023-10-06T21:07:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.320+0000 7fe9acff9700 1 -- 172.21.15.26:0/2511659858 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6071 (secure 0 0 0) 0x7fe9a0099b80 con 0x7fe9b0100ea0 2023-10-06T21:07:59.322 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:07:59.322 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 msgr2=0x7fe9980a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 0x7fe9980a3a10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0100bc0 tx=0x7fe9a4009260 comp rx=0 tx=0).stop 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 msgr2=0x7fe9b0198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000e970 tx=0x7fe9a000ec80 comp rx=0 tx=0).stop 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 shutdown_connections 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe9980a1550 0x7fe9980a3a10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9b0102a40 0x7fe9b01a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9b0100ea0 0x7fe9b0198ef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 --2- 172.21.15.26:0/2511659858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b0102080 0x7fe9b0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 >> 172.21.15.26:0/2511659858 conn(0x7fe9b00fc420 msgr2=0x7fe9b01079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 shutdown_connections 2023-10-06T21:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:07:59.322+0000 7fe9b5bff700 1 -- 172.21.15.26:0/2511659858 wait complete. 2023-10-06T21:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-10-06T21:07:59.370 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:59 smithi078 podman[171515]: 2023-10-06T21:07:59.829 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mds[171555]: -- 172.21.15.78:0/3546317860 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f453d71e0 con 0x557f453ad000 2023-10-06T21:07:59.829 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:59 smithi078 bash[171515]: 9cc262cd94e8df63b0f817c478e07cb3da9dd3e1e3e322a9d1d276883cade199 2023-10-06T21:07:59.829 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: starting mds.h at 2023-10-06T21:07:59.829 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:07:59 smithi078 systemd[1]: Started Ceph mds.h for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:08:00.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2511659858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:07:59 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2511659858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:00.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:00.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:07:59 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2511659858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:00.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:00.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:00.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:07:59 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:01.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:00 smithi167 ceph-mon[134494]: Deploying daemon mds.i on smithi167 2023-10-06T21:08:01.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:00 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:boot 2023-10-06T21:08:01.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:00 smithi167 ceph-mon[134494]: fsmap 8 up:standby 2023-10-06T21:08:01.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:00 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:08:01.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:00 smithi167 ceph-mon[134494]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:00 smithi078 ceph-mon[134896]: Deploying daemon mds.i on smithi167 2023-10-06T21:08:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:00 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:boot 2023-10-06T21:08:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:00 smithi078 ceph-mon[134896]: fsmap 8 up:standby 2023-10-06T21:08:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:00 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:08:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:00 smithi078 ceph-mon[134896]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:01.283 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 systemd[1]: Starting Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:08:01.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:00 smithi026 ceph-mon[98490]: Deploying daemon mds.i on smithi167 2023-10-06T21:08:01.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:00 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:boot 2023-10-06T21:08:01.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:00 smithi026 ceph-mon[98490]: fsmap 8 up:standby 2023-10-06T21:08:01.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:00 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:08:01.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:00 smithi026 ceph-mon[98490]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:01.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 -- 172.21.15.26:0/283977650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 msgr2=0x7f76581012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 --2- 172.21.15.26:0/283977650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f76581012c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7648009990 tx=0x7f7648009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 -- 172.21.15.26:0/283977650 shutdown_connections 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 --2- 172.21.15.26:0/283977650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102a40 0x7f7658109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 --2- 172.21.15.26:0/283977650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f76581012c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 --2- 172.21.15.26:0/283977650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102080 0x7f7658102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 -- 172.21.15.26:0/283977650 >> 172.21.15.26:0/283977650 conn(0x7f76580fc420 msgr2=0x7f76580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 -- 172.21.15.26:0/283977650 shutdown_connections 2023-10-06T21:08:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.558+0000 7f765d1ff700 1 -- 172.21.15.26:0/283977650 wait complete. 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 Processor -- start 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 -- start start 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102080 0x7f7658199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102a40 0x7f76581a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a0980 con 0x7f7658102a40 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a0ac0 con 0x7f7658102080 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765d1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a0c50 con 0x7f7658100ea0 2023-10-06T21:08:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f7656d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f7656d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:56494/0 (socket says 172.21.15.26:56494) 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f7656d9d700 1 -- 172.21.15.26:0/1984254444 learned_addr learned my addr 172.21.15.26:0/1984254444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.559+0000 7f765759e700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102a40 0x7f76581a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f765659c700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102080 0x7f7658199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 -- 172.21.15.26:0/1984254444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102080 msgr2=0x7f7658199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102080 0x7f7658199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 -- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102a40 msgr2=0x7f76581a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102a40 0x7f76581a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 -- 172.21.15.26:0/1984254444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480095f0 con 0x7f7658100ea0 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f7656d9d700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f764800b680 tx=0x7f7648005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764801d070 con 0x7f7658100ea0 2023-10-06T21:08:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7648021410 con 0x7f7658100ea0 2023-10-06T21:08:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76581a0ed0 con 0x7f7658100ea0 2023-10-06T21:08:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.560+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648017650 con 0x7f7658100ea0 2023-10-06T21:08:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.561+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a1240 con 0x7f7658100ea0 2023-10-06T21:08:01.566 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 podman[171083]: 2023-10-06T21:08:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.562+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658193550 con 0x7f7658100ea0 2023-10-06T21:08:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.564+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7648021580 con 0x7f7658100ea0 2023-10-06T21:08:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.565+0000 7f763ffff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 0x7f76400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.565+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76480cdf50 con 0x7f7658100ea0 2023-10-06T21:08:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.566+0000 7f765659c700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 0x7f76400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.566+0000 7f765659c700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 0x7f76400a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7658100bc0 tx=0x7f764c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.567+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7648097a40 con 0x7f7658100ea0 2023-10-06T21:08:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.840+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765802d090 con 0x7f7658100ea0 2023-10-06T21:08:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.840+0000 7f763ffff700 1 -- 172.21.15.26:0/1984254444 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7f764809b060 con 0x7f7658100ea0 2023-10-06T21:08:01.843 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:01.843 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-06T21:08:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 msgr2=0x7f76400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 0x7f76400a3990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7658100bc0 tx=0x7f764c009260 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 msgr2=0x7f7658198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f764800b680 tx=0x7f7648005740 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 shutdown_connections 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76400a14d0 0x7f76400a3990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7658102a40 0x7f76581a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7658100ea0 0x7f7658198e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 --2- 172.21.15.26:0/1984254444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7658102080 0x7f7658199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 >> 172.21.15.26:0/1984254444 conn(0x7f76580fc420 msgr2=0x7f76581079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 shutdown_connections 2023-10-06T21:08:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:01.843+0000 7f765d1ff700 1 -- 172.21.15.26:0/1984254444 wait complete. 2023-10-06T21:08:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2023-10-06T21:08:02.032 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 ceph-mds[171123]: -- 172.21.15.167:0/2558996850 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559571b251e0 con 0x559571afa800 2023-10-06T21:08:02.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: starting mds.i at 2023-10-06T21:08:02.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 bash[171083]: fe178e39e8d69852a6abf878f9117cd7431b0736185b33144aaca5d4370bd8cc 2023-10-06T21:08:02.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:08:01 smithi167 systemd[1]: Started Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:08:02.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:02.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:02.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:02.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:02.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: Deploying daemon mds.g on smithi026 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6838/1710830195,v1:172.21.15.167:6839/1710830195] up:boot 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: fsmap 9 up:standby 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1984254444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:02.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:02 smithi026 ceph-mon[98490]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: Deploying daemon mds.g on smithi026 2023-10-06T21:08:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6838/1710830195,v1:172.21.15.167:6839/1710830195] up:boot 2023-10-06T21:08:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: fsmap 9 up:standby 2023-10-06T21:08:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:08:03.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1984254444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:03.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:02 smithi167 ceph-mon[134494]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: Deploying daemon mds.g on smithi026 2023-10-06T21:08:03.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6838/1710830195,v1:172.21.15.167:6839/1710830195] up:boot 2023-10-06T21:08:03.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: fsmap 9 up:standby 2023-10-06T21:08:03.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:08:03.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1984254444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:03.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:02 smithi078 ceph-mon[134896]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:03.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:03.919 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:03 smithi026 systemd[1]: Starting Ceph mds.g for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:08:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 -- 172.21.15.26:0/135043978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 msgr2=0x7f467c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 --2- 172.21.15.26:0/135043978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c073340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f466c009990 tx=0x7f466c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 -- 172.21.15.26:0/135043978 shutdown_connections 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 --2- 172.21.15.26:0/135043978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c108db0 0x7f467c10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 --2- 172.21.15.26:0/135043978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c073340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.093+0000 7f46809ff700 1 --2- 172.21.15.26:0/135043978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c073910 0x7f467c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.094+0000 7f46809ff700 1 -- 172.21.15.26:0/135043978 >> 172.21.15.26:0/135043978 conn(0x7f467c0fc420 msgr2=0x7f467c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.094+0000 7f46809ff700 1 -- 172.21.15.26:0/135043978 shutdown_connections 2023-10-06T21:08:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.094+0000 7f46809ff700 1 -- 172.21.15.26:0/135043978 wait complete. 2023-10-06T21:08:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 Processor -- start 2023-10-06T21:08:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 -- start start 2023-10-06T21:08:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 0x7f467c198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c108db0 0x7f467c1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1a0940 con 0x7f467c074ef0 2023-10-06T21:08:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1a0a80 con 0x7f467c073910 2023-10-06T21:08:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f46809ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1a0c10 con 0x7f467c108db0 2023-10-06T21:08:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f467a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:04.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f467259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 0x7f467c198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f467a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34526/0 (socket says 172.21.15.26:34526) 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.095+0000 7f467a59c700 1 -- 172.21.15.26:0/2138847188 learned_addr learned my addr 172.21.15.26:0/2138847188 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467ad9d700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c108db0 0x7f467c1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 -- 172.21.15.26:0/2138847188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c108db0 msgr2=0x7f467c1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c108db0 0x7f467c1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 -- 172.21.15.26:0/2138847188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 msgr2=0x7f467c198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 0x7f467c198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 -- 172.21.15.26:0/2138847188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0095f0 con 0x7f467c074ef0 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467259c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 0x7f467c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f467a59c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f466400d980 tx=0x7f466400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:04.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664004020 con 0x7f467c074ef0 2023-10-06T21:08:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f466400f690 con 0x7f467c074ef0 2023-10-06T21:08:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.096+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c1a0ef0 con 0x7f467c074ef0 2023-10-06T21:08:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.097+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664003a60 con 0x7f467c074ef0 2023-10-06T21:08:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.097+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c1a1370 con 0x7f467c074ef0 2023-10-06T21:08:04.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.098+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c107460 con 0x7f467c074ef0 2023-10-06T21:08:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.100+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f466400fb00 con 0x7f467c074ef0 2023-10-06T21:08:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.101+0000 7f4673fff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 0x7f46680a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.101+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f46640cc480 con 0x7f467c074ef0 2023-10-06T21:08:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.101+0000 7f467259c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 0x7f46680a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.102+0000 7f467259c700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 0x7f46680a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b5c0 tx=0x7f466c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.106+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4664095ee0 con 0x7f467c074ef0 2023-10-06T21:08:04.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.370+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f467c10e930 con 0x7f467c074ef0 2023-10-06T21:08:04.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.371+0000 7f4673fff700 1 -- 172.21.15.26:0/2138847188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7658 (secure 0 0 0) 0x7f4664099500 con 0x7f467c074ef0 2023-10-06T21:08:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:04.374 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-06T21:08:04.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.373+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 msgr2=0x7f46680a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:04.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.373+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 0x7f46680a3990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f466c00b5c0 tx=0x7f466c005fd0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 msgr2=0x7f467c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f466400d980 tx=0x7f466400dc90 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 shutdown_connections 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c108db0 0x7f467c1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467c073910 0x7f467c198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f46680a14d0 0x7f46680a3990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 --2- 172.21.15.26:0/2138847188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f467c074ef0 0x7f467c199300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 >> 172.21.15.26:0/2138847188 conn(0x7f467c0fc420 msgr2=0x7f467c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 shutdown_connections 2023-10-06T21:08:04.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:04.374+0000 7f46809ff700 1 -- 172.21.15.26:0/2138847188 wait complete. 2023-10-06T21:08:04.378 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2023-10-06T21:08:04.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:04 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2138847188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:04.701 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:04 smithi026 podman[149534]: 2023-10-06T21:08:04.702 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:04 smithi026 bash[149534]: be12dbaefb9697328238da4f214568cd99f983790ac1ce9f62bfa836e4378f28 2023-10-06T21:08:04.702 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:04 smithi026 ceph-mds[149576]: -- 172.21.15.26:0/1746305777 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556453ef71e0 con 0x556453eccc00 2023-10-06T21:08:04.702 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:04 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-g[149552]: starting mds.g at 2023-10-06T21:08:04.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:04 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2138847188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:04 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2138847188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:05.073 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:08:04 smithi026 systemd[1]: Started Ceph mds.g for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:08:05.761 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:05 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:05.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:05 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:05.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:05 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:06.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:06 smithi026 ceph-mon[98490]: Deploying daemon mds.a on smithi026 2023-10-06T21:08:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:06 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6838/4229413867,v1:172.21.15.26:6839/4229413867] up:boot 2023-10-06T21:08:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:06 smithi026 ceph-mon[98490]: fsmap 10 up:standby 2023-10-06T21:08:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:06 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:08:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 -- 172.21.15.26:0/3994391155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874102040 msgr2=0x7fe8741044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 --2- 172.21.15.26:0/3994391155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874102040 0x7fe8741044d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe868009990 tx=0x7fe868009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 -- 172.21.15.26:0/3994391155 shutdown_connections 2023-10-06T21:08:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 --2- 172.21.15.26:0/3994391155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe874104a10 0x7fe874104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 --2- 172.21.15.26:0/3994391155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8740ff6d0 0x7fe874101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 --2- 172.21.15.26:0/3994391155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874102040 0x7fe8741044d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 -- 172.21.15.26:0/3994391155 >> 172.21.15.26:0/3994391155 conn(0x7fe8740fb2b0 msgr2=0x7fe8740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 -- 172.21.15.26:0/3994391155 shutdown_connections 2023-10-06T21:08:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.625+0000 7fe878bff700 1 -- 172.21.15.26:0/3994391155 wait complete. 2023-10-06T21:08:06.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 Processor -- start 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 -- start start 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8740ff6d0 0x7fe87419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874104a10 0x7fe8741a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741a4cb0 con 0x7fe874102040 2023-10-06T21:08:06.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741a4df0 con 0x7fe874104a10 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.627+0000 7fe878bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741a4f80 con 0x7fe8740ff6d0 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe87259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8740ff6d0 0x7fe87419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874104a10 0x7fe8741a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34554/0 (socket says 172.21.15.26:34554) 2023-10-06T21:08:06.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 -- 172.21.15.26:0/3670767896 learned_addr learned my addr 172.21.15.26:0/3670767896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 -- 172.21.15.26:0/3670767896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8740ff6d0 msgr2=0x7fe87419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8740ff6d0 0x7fe87419d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 -- 172.21.15.26:0/3670767896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874104a10 msgr2=0x7fe8741a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874104a10 0x7fe8741a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 -- 172.21.15.26:0/3670767896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680095f0 con 0x7fe874102040 2023-10-06T21:08:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.628+0000 7fe871d9b700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe868004740 tx=0x7fe868009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.629+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86801d070 con 0x7fe874102040 2023-10-06T21:08:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.629+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe868004830 con 0x7fe874102040 2023-10-06T21:08:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.629+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741a5200 con 0x7fe874102040 2023-10-06T21:08:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.629+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680053c0 con 0x7fe874102040 2023-10-06T21:08:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.629+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741a55c0 con 0x7fe874102040 2023-10-06T21:08:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.630+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740fce30 con 0x7fe874102040 2023-10-06T21:08:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.632+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe868005580 con 0x7fe874102040 2023-10-06T21:08:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.633+0000 7fe8637fe700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 0x7fe85c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.633+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8680cda00 con 0x7fe874102040 2023-10-06T21:08:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.633+0000 7fe87259c700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 0x7fe85c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.634+0000 7fe87259c700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 0x7fe85c0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe864009c60 tx=0x7fe864009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.637+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe86805b100 con 0x7fe874102040 2023-10-06T21:08:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:06 smithi167 ceph-mon[134494]: Deploying daemon mds.a on smithi026 2023-10-06T21:08:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:06 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6838/4229413867,v1:172.21.15.26:6839/4229413867] up:boot 2023-10-06T21:08:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:06 smithi167 ceph-mon[134494]: fsmap 10 up:standby 2023-10-06T21:08:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:06 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:08:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:06 smithi078 ceph-mon[134896]: Deploying daemon mds.a on smithi026 2023-10-06T21:08:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:06 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6838/4229413867,v1:172.21.15.26:6839/4229413867] up:boot 2023-10-06T21:08:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:06 smithi078 ceph-mon[134896]: fsmap 10 up:standby 2023-10-06T21:08:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:06 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:08:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.931+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe874112ca0 con 0x7fe874102040 2023-10-06T21:08:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.932+0000 7fe8637fe700 1 -- 172.21.15.26:0/3670767896 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7fe8680974f0 con 0x7fe874102040 2023-10-06T21:08:06.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:06.935 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-06T21:08:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 msgr2=0x7fe85c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 0x7fe85c0a3990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe864009c60 tx=0x7fe864009260 comp rx=0 tx=0).stop 2023-10-06T21:08:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 msgr2=0x7fe87419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe868004740 tx=0x7fe868009580 comp rx=0 tx=0).stop 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 shutdown_connections 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe874104a10 0x7fe8741a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8740ff6d0 0x7fe87419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe85c0a14d0 0x7fe85c0a3990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 --2- 172.21.15.26:0/3670767896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe874102040 0x7fe87419d670 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 >> 172.21.15.26:0/3670767896 conn(0x7fe8740fb2b0 msgr2=0x7fe874100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 shutdown_connections 2023-10-06T21:08:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:06.934+0000 7fe878bff700 1 -- 172.21.15.26:0/3670767896 wait complete. 2023-10-06T21:08:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 11 2023-10-06T21:08:07.323 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 systemd[1]: Starting Ceph mds.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:08:07.780 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 podman[149921]: 2023-10-06T21:08:07.780 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:07 smithi026 ceph-mon[98490]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:07.781 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:07 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3670767896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:07.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:07 smithi167 ceph-mon[134494]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:07.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:07 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3670767896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:07 smithi078 ceph-mon[134896]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:07 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3670767896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:08.073 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 ceph-mds[149961]: -- 172.21.15.26:0/1308598448 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595e11ff1e0 con 0x5595e11d5000 2023-10-06T21:08:08.073 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: starting mds.a at 2023-10-06T21:08:08.074 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 bash[149921]: 579a0862020004244d4e33e2b6f161db1a909ba9e452125a3740ed4dae2d044c 2023-10-06T21:08:08.074 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:08:07 smithi026 systemd[1]: Started Ceph mds.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:08:08.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:09.150 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: Deploying daemon mds.b on smithi078 2023-10-06T21:08:09.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:09.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:boot 2023-10-06T21:08:09.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: fsmap 11 up:standby 2023-10-06T21:08:09.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:08 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:08:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 -- 172.21.15.26:0/1238749331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858102a40 msgr2=0x7f3858109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 --2- 172.21.15.26:0/1238749331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858102a40 0x7f3858109360 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3850009f90 tx=0x7f385000b330 comp rx=0 tx=0).stop 2023-10-06T21:08:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 -- 172.21.15.26:0/1238749331 shutdown_connections 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 --2- 172.21.15.26:0/1238749331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858102a40 0x7f3858109360 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 --2- 172.21.15.26:0/1238749331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858100ea0 0x7f38581012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 --2- 172.21.15.26:0/1238749331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102080 0x7f3858102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 -- 172.21.15.26:0/1238749331 >> 172.21.15.26:0/1238749331 conn(0x7f38580fc420 msgr2=0x7f38580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.175+0000 7f38601ff700 1 -- 172.21.15.26:0/1238749331 shutdown_connections 2023-10-06T21:08:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.176+0000 7f38601ff700 1 -- 172.21.15.26:0/1238749331 wait complete. 2023-10-06T21:08:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.176+0000 7f38601ff700 1 Processor -- start 2023-10-06T21:08:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 -- start start 2023-10-06T21:08:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858102080 0x7f3858199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:09.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 0x7f38581a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581a0a40 con 0x7f3858100ea0 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581a0b80 con 0x7f3858102080 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f38601ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581a0d10 con 0x7f3858102a40 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f385df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f385df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34596/0 (socket says 172.21.15.26:34596) 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f385df9b700 1 -- 172.21.15.26:0/3845418131 learned_addr learned my addr 172.21.15.26:0/3845418131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f385d79a700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858102080 0x7f3858199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.177+0000 7f385e79c700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 0x7f38581a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 -- 172.21.15.26:0/3845418131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 msgr2=0x7f38581a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 0x7f38581a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 -- 172.21.15.26:0/3845418131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858102080 msgr2=0x7f3858199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858102080 0x7f3858199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 -- 172.21.15.26:0/3845418131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480095f0 con 0x7f3858100ea0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385e79c700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 0x7f38581a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f385df9b700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f385400e970 tx=0x7f385400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854009790 con 0x7f3858100ea0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385400eec0 con 0x7f3858100ea0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854003cf0 con 0x7f3858100ea0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3850009c40 con 0x7f3858100ea0 2023-10-06T21:08:09.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.178+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38581a1280 con 0x7f3858100ea0 2023-10-06T21:08:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.179+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385810c710 con 0x7f3858100ea0 2023-10-06T21:08:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.181+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f385400ccd0 con 0x7f3858100ea0 2023-10-06T21:08:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.183+0000 7f384effd700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 0x7f38440a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.183+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3854014070 con 0x7f3858100ea0 2023-10-06T21:08:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.183+0000 7f385d79a700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 0x7f38440a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:09.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.183+0000 7f385d79a700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 0x7f38440a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3858100bc0 tx=0x7f3848009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.187+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3854096560 con 0x7f3858100ea0 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: Deploying daemon mds.b on smithi078 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:boot 2023-10-06T21:08:09.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: fsmap 11 up:standby 2023-10-06T21:08:09.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:08 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:08:09.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: Deploying daemon mds.b on smithi078 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:boot 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: fsmap 11 up:standby 2023-10-06T21:08:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:08 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:08:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.447+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38581a17b0 con 0x7f3858100ea0 2023-10-06T21:08:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.448+0000 7f384effd700 1 -- 172.21.15.26:0/3845418131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9242 (secure 0 0 0) 0x7f3854099b80 con 0x7f3858100ea0 2023-10-06T21:08:09.453 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:09.453 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-10-06T21:08:09.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 msgr2=0x7f38440a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:09.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 0x7f38440a3a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3858100bc0 tx=0x7f3848009310 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 msgr2=0x7f3858198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f385400e970 tx=0x7f385400ec80 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 shutdown_connections 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3858102a40 0x7f38581a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38440a1550 0x7f38440a3a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3858100ea0 0x7f3858198ec0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 --2- 172.21.15.26:0/3845418131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3858102080 0x7f3858199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 >> 172.21.15.26:0/3845418131 conn(0x7f38580fc420 msgr2=0x7f38581079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.450+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 shutdown_connections 2023-10-06T21:08:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:09.451+0000 7f38601ff700 1 -- 172.21.15.26:0/3845418131 wait complete. 2023-10-06T21:08:09.456 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2023-10-06T21:08:09.813 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:09 smithi078 systemd[1]: Starting Ceph mds.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:08:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3845418131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:10.121 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:09 smithi078 podman[171855]: 2023-10-06T21:08:10.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3845418131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:10.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3845418131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:10.620 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:10 smithi078 ceph-mds[171895]: -- 172.21.15.78:0/2821712086 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559954e3f1e0 con 0x559954e15000 2023-10-06T21:08:10.621 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: starting mds.b at 2023-10-06T21:08:10.621 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:10 smithi078 bash[171855]: f0f127b434a36013501143ccd7eeec5851bd1c67c37f0cdd2cd478ba5fd641fa 2023-10-06T21:08:10.621 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:10 smithi078 systemd[1]: Started Ceph mds.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:08:11.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:11.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.538 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:boot 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: fsmap 12 up:standby 2023-10-06T21:08:11.539 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:11 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:boot 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: fsmap 12 up:standby 2023-10-06T21:08:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:11 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:08:11.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 -- 172.21.15.26:0/2098437461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500100ea0 msgr2=0x7fc5001012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2098437461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500100ea0 0x7fc5001012c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009990 tx=0x7fc4f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 -- 172.21.15.26:0/2098437461 shutdown_connections 2023-10-06T21:08:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2098437461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102a40 0x7fc500109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2098437461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500100ea0 0x7fc5001012c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2098437461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500102080 0x7fc500102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 -- 172.21.15.26:0/2098437461 >> 172.21.15.26:0/2098437461 conn(0x7fc5000fc420 msgr2=0x7fc5000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 -- 172.21.15.26:0/2098437461 shutdown_connections 2023-10-06T21:08:11.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.708+0000 7fc5063ff700 1 -- 172.21.15.26:0/2098437461 wait complete. 2023-10-06T21:08:11.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.709+0000 7fc5063ff700 1 Processor -- start 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 -- start start 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102080 0x7fc5000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500102a40 0x7fc5000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500073f80 con 0x7fc500102a40 2023-10-06T21:08:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000740f0 con 0x7fc500100ea0 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc5063ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001a0c60 con 0x7fc500102080 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35776/0 (socket says 172.21.15.26:35776) 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 -- 172.21.15.26:0/2965406232 learned_addr learned my addr 172.21.15.26:0/2965406232 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc50499c700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500102a40 0x7fc5000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4ff7fe700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102080 0x7fc5000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 -- 172.21.15.26:0/2965406232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102080 msgr2=0x7fc5000732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102080 0x7fc5000732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 -- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500102a40 msgr2=0x7fc5000737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500102a40 0x7fc5000737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.710+0000 7fc4fffff700 1 -- 172.21.15.26:0/2965406232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f00095f0 con 0x7fc500100ea0 2023-10-06T21:08:11.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc4fffff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0005720 tx=0x7fc4f0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f001d070 con 0x7fc500100ea0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4f0021410 con 0x7fc500100ea0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0017650 con 0x7fc500100ea0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5001a0ee0 con 0x7fc500100ea0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.711+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5001a1250 con 0x7fc500100ea0 2023-10-06T21:08:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.712+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50010c760 con 0x7fc500100ea0 2023-10-06T21:08:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.714+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc4f0021580 con 0x7fc500100ea0 2023-10-06T21:08:11.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.715+0000 7fc4fd7fa700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 0x7fc4e80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.715+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4f00cd9d0 con 0x7fc500100ea0 2023-10-06T21:08:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.716+0000 7fc4ff7fe700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 0x7fc4e80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.716+0000 7fc4ff7fe700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 0x7fc4e80a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc500100bc0 tx=0x7fc4f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:11.717+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc4f00974c0 con 0x7fc500100ea0 2023-10-06T21:08:11.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:11.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:11.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:11.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:boot 2023-10-06T21:08:11.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: fsmap 12 up:standby 2023-10-06T21:08:11.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:11 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:08:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.013+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc500107e10 con 0x7fc500100ea0 2023-10-06T21:08:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.014+0000 7fc4fd7fa700 1 -- 172.21.15.26:0/2965406232 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10034 (secure 0 0 0) 0x7fc4f009aae0 con 0x7fc500100ea0 2023-10-06T21:08:12.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:12.017 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-06T21:08:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.016+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 msgr2=0x7fc4e80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.016+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 0x7fc4e80a3a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc500100bc0 tx=0x7fc4f4009260 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.016+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 msgr2=0x7fc500074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.016+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0005720 tx=0x7fc4f0009580 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 shutdown_connections 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc4e80a1550 0x7fc4e80a3a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc500102a40 0x7fc5000737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc500100ea0 0x7fc500074c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 --2- 172.21.15.26:0/2965406232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc500102080 0x7fc5000732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 >> 172.21.15.26:0/2965406232 conn(0x7fc5000fc420 msgr2=0x7fc5000fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 shutdown_connections 2023-10-06T21:08:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.017+0000 7fc5063ff700 1 -- 172.21.15.26:0/2965406232 wait complete. 2023-10-06T21:08:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2023-10-06T21:08:12.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:08:12.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 -- 172.21.15.26:0/1729877977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628073910 msgr2=0x7fe628073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/1729877977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628073910 0x7fe628073d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009990 tx=0x7fe61c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 -- 172.21.15.26:0/1729877977 shutdown_connections 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/1729877977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 0x7fe62810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/1729877977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628074ef0 0x7fe628073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/1729877977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628073910 0x7fe628073d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 -- 172.21.15.26:0/1729877977 >> 172.21.15.26:0/1729877977 conn(0x7fe6280fc420 msgr2=0x7fe6280fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:12.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 -- 172.21.15.26:0/1729877977 shutdown_connections 2023-10-06T21:08:12.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.249+0000 7fe62e9ff700 1 -- 172.21.15.26:0/1729877977 wait complete. 2023-10-06T21:08:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.250+0000 7fe62e9ff700 1 Processor -- start 2023-10-06T21:08:12.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 -- start start 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628073910 0x7fe628198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 0x7fe6281a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a0950 con 0x7fe628108db0 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a0a90 con 0x7fe628073910 2023-10-06T21:08:12.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62e9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a0c20 con 0x7fe628074ef0 2023-10-06T21:08:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe6277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe6277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42592/0 (socket says 172.21.15.26:42592) 2023-10-06T21:08:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe6277fe700 1 -- 172.21.15.26:0/4085289816 learned_addr learned my addr 172.21.15.26:0/4085289816 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:12.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe62cf9c700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 0x7fe6281a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.251+0000 7fe627fff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628073910 0x7fe628198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 -- 172.21.15.26:0/4085289816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628073910 msgr2=0x7fe628198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628073910 0x7fe628198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 -- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 msgr2=0x7fe6281a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 0x7fe6281a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 -- 172.21.15.26:0/4085289816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0095f0 con 0x7fe628074ef0 2023-10-06T21:08:12.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6277fe700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005720 tx=0x7fe61c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c01d070 con 0x7fe628074ef0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61c021410 con 0x7fe628074ef0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c017650 con 0x7fe628074ef0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6281a0ea0 con 0x7fe628074ef0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.252+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6281a1210 con 0x7fe628074ef0 2023-10-06T21:08:12.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.254+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe628107460 con 0x7fe628074ef0 2023-10-06T21:08:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.256+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe61c021580 con 0x7fe628074ef0 2023-10-06T21:08:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.257+0000 7fe6257fa700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 0x7fe6100a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.257+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe61c0cda00 con 0x7fe628074ef0 2023-10-06T21:08:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.257+0000 7fe627fff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 0x7fe6100a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.259+0000 7fe627fff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 0x7fe6100a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe618009c60 tx=0x7fe618009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:12.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.261+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe61c0974f0 con 0x7fe628074ef0 2023-10-06T21:08:12.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.486+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe62810e930 con 0x7fe628074ef0 2023-10-06T21:08:12.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.488+0000 7fe6257fa700 1 -- 172.21.15.26:0/4085289816 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18371 (secure 0 0 0) 0x7fe61c09ab10 con 0x7fe628074ef0 2023-10-06T21:08:12.492 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:12.492 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:07:29.352305+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 msgr2=0x7fe6100a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 0x7fe6100a3a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe618009c60 tx=0x7fe618009260 comp rx=0 tx=0).stop 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 msgr2=0x7fe628199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005720 tx=0x7fe61c009580 comp rx=0 tx=0).stop 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 shutdown_connections 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe6100a1550 0x7fe6100a3a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe628108db0 0x7fe6281a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe628073910 0x7fe628198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 --2- 172.21.15.26:0/4085289816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe628074ef0 0x7fe628199310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 >> 172.21.15.26:0/4085289816 conn(0x7fe6280fc420 msgr2=0x7fe6280fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 shutdown_connections 2023-10-06T21:08:12.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.491+0000 7fe62e9ff700 1 -- 172.21.15.26:0/4085289816 wait complete. 2023-10-06T21:08:12.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:08:12.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:08:12.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:12 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2965406232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:12.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:12 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2965406232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:12.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 -- 172.21.15.26:0/1836341195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106be0 msgr2=0x7f919c109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 --2- 172.21.15.26:0/1836341195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106be0 0x7f919c109050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9194009f90 tx=0x7f919400b330 comp rx=0 tx=0).stop 2023-10-06T21:08:12.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 -- 172.21.15.26:0/1836341195 shutdown_connections 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 --2- 172.21.15.26:0/1836341195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c106be0 0x7f919c109050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 --2- 172.21.15.26:0/1836341195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c101980 0x7f919c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 --2- 172.21.15.26:0/1836341195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c1042b0 0x7f919c1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 -- 172.21.15.26:0/1836341195 >> 172.21.15.26:0/1836341195 conn(0x7f919c0fb370 msgr2=0x7f919c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 -- 172.21.15.26:0/1836341195 shutdown_connections 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.735+0000 7f91a37ff700 1 -- 172.21.15.26:0/1836341195 wait complete. 2023-10-06T21:08:12.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.736+0000 7f91a37ff700 1 Processor -- start 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.736+0000 7f91a37ff700 1 -- start start 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.736+0000 7f91a37ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c101980 0x7f919c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.736+0000 7f91a37ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.736+0000 7f91a37ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 0x7f919c1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a37ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0a70 con 0x7f919c101980 2023-10-06T21:08:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a37ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0bb0 con 0x7f919c1042b0 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a37ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0d40 con 0x7f919c106be0 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a159b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c101980 0x7f919c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35818/0 (socket says 172.21.15.26:35818) 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 -- 172.21.15.26:0/450344751 learned_addr learned my addr 172.21.15.26:0/450344751 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a1d9c700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 0x7f919c1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 -- 172.21.15.26:0/450344751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 msgr2=0x7f919c1a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 0x7f919c1a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 -- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c101980 msgr2=0x7f919c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c101980 0x7f919c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 -- 172.21.15.26:0/450344751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c0095f0 con 0x7f919c1042b0 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a1d9c700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 0x7f919c1a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:08:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.737+0000 7f91a0d9a700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f918c00ec10 tx=0x7f918c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:12.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.738+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cd40 con 0x7f919c1042b0 2023-10-06T21:08:12.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.738+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f918c0044e0 con 0x7f919c1042b0 2023-10-06T21:08:12.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.738+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194009c40 con 0x7f919c1042b0 2023-10-06T21:08:12.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.738+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c010690 con 0x7f919c1042b0 2023-10-06T21:08:12.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.738+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1a1360 con 0x7f919c1042b0 2023-10-06T21:08:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.739+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c108760 con 0x7f919c1042b0 2023-10-06T21:08:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.741+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f918c003710 con 0x7f919c1042b0 2023-10-06T21:08:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.742+0000 7f91927fc700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 0x7f91880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.742+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f918c014070 con 0x7f919c1042b0 2023-10-06T21:08:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.742+0000 7f91a159b700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 0x7f91880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.743+0000 7f91a159b700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 0x7f91880a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9198009c60 tx=0x7f9198009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.744+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f918c096c20 con 0x7f919c1042b0 2023-10-06T21:08:12.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:12 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2965406232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:12.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.962+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f919c10c8c0 con 0x7f919c1042b0 2023-10-06T21:08:12.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.963+0000 7f91927fc700 1 -- 172.21.15.26:0/450344751 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f918c09a240 con 0x7f919c1042b0 2023-10-06T21:08:12.966 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:08:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.965+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 msgr2=0x7f91880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 0x7f91880a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9198009c60 tx=0x7f9198009260 comp rx=0 tx=0).stop 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 msgr2=0x7f919c199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f918c00ec10 tx=0x7f918c00ef20 comp rx=0 tx=0).stop 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 shutdown_connections 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f919c106be0 0x7f919c1a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f91880a1500 0x7f91880a39c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f919c101980 0x7f919c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 --2- 172.21.15.26:0/450344751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f919c1042b0 0x7f919c199430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 >> 172.21.15.26:0/450344751 conn(0x7f919c0fb370 msgr2=0x7f919c100000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 shutdown_connections 2023-10-06T21:08:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:12.966+0000 7f91a37ff700 1 -- 172.21.15.26:0/450344751 wait complete. 2023-10-06T21:08:12.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 -- 172.21.15.26:0/355538253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c100f10 msgr2=0x7f123c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 --2- 172.21.15.26:0/355538253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c100f10 0x7f123c101330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1238009990 tx=0x7f1238009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 -- 172.21.15.26:0/355538253 shutdown_connections 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 --2- 172.21.15.26:0/355538253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 0x7f123c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 --2- 172.21.15.26:0/355538253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c100f10 0x7f123c101330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 --2- 172.21.15.26:0/355538253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c102060 0x7f123c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 -- 172.21.15.26:0/355538253 >> 172.21.15.26:0/355538253 conn(0x7f123c0fc460 msgr2=0x7f123c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 -- 172.21.15.26:0/355538253 shutdown_connections 2023-10-06T21:08:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.199+0000 7f12441ff700 1 -- 172.21.15.26:0/355538253 wait complete. 2023-10-06T21:08:13.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.200+0000 7f12441ff700 1 Processor -- start 2023-10-06T21:08:13.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.200+0000 7f12441ff700 1 -- start start 2023-10-06T21:08:13.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 0x7f123c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 0x7f123c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 0x7f123c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a0890 con 0x7f123c100f10 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a0a00 con 0x7f123c102a20 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f12441ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1a0bc0 con 0x7f123c102060 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f1241f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 0x7f123c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 0x7f123c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f1241f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 0x7f123c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60782/0 (socket says 172.21.15.26:60782) 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f1241f9b700 1 -- 172.21.15.26:0/881061735 learned_addr learned my addr 172.21.15.26:0/881061735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124279c700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 0x7f123c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 -- 172.21.15.26:0/881061735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 msgr2=0x7f123c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 0x7f123c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 -- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 msgr2=0x7f123c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 0x7f123c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.201+0000 7f124179a700 1 -- 172.21.15.26:0/881061735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380095f0 con 0x7f123c102060 2023-10-06T21:08:13.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f124179a700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 0x7f123c199290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f122c00e9f0 tx=0x7f122c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c00c9c0 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c010430 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c018680 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c1a0ea0 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.202+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c1a1270 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.204+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c10c700 con 0x7f123c102060 2023-10-06T21:08:13.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.205+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f122c010860 con 0x7f123c102060 2023-10-06T21:08:13.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.206+0000 7f1232ffd700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 0x7f12280a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.206+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f122c014070 con 0x7f123c102060 2023-10-06T21:08:13.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.207+0000 7f1241f9b700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 0x7f12280a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.207+0000 7f1241f9b700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 0x7f12280a3a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f123800b2e0 tx=0x7f1238005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.210+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f122c052100 con 0x7f123c102060 2023-10-06T21:08:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.435+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f123c1a1490 con 0x7f123c102060 2023-10-06T21:08:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.436+0000 7f1232ffd700 1 -- 172.21.15.26:0/881061735 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f122c096400 con 0x7f123c102060 2023-10-06T21:08:13.439 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:13.439 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-10-06T21:08:13.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:08:13.439 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-10-06T21:08:13.440 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-10-06T21:08:13.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:08:13.440 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-10-06T21:08:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.438+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 msgr2=0x7f12280a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.438+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 0x7f12280a3a10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f123800b2e0 tx=0x7f1238005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.438+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 msgr2=0x7f123c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.438+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 0x7f123c199290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f122c00e9f0 tx=0x7f122c00ed00 comp rx=0 tx=0).stop 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 shutdown_connections 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f123c102a20 0x7f123c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f12280a1550 0x7f12280a3a10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f123c100f10 0x7f123c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 --2- 172.21.15.26:0/881061735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f123c102060 0x7f123c199290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 >> 172.21.15.26:0/881061735 conn(0x7f123c0fc460 msgr2=0x7f123c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 shutdown_connections 2023-10-06T21:08:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.439+0000 7f12441ff700 1 -- 172.21.15.26:0/881061735 wait complete. 2023-10-06T21:08:13.460 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T21:08:13.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T21:08:13.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4085289816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/450344751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:13 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 -- 172.21.15.26:0/1211633808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 msgr2=0x7f87f0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1211633808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0102510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009990 tx=0x7f87e4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 -- 172.21.15.26:0/1211633808 shutdown_connections 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1211633808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0102a50 0x7f87f0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1211633808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0100f40 0x7f87f0101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1211633808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0102510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 -- 172.21.15.26:0/1211633808 >> 172.21.15.26:0/1211633808 conn(0x7f87f00fc4d0 msgr2=0x7f87f00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 -- 172.21.15.26:0/1211633808 shutdown_connections 2023-10-06T21:08:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.672+0000 7f87f61ff700 1 -- 172.21.15.26:0/1211633808 wait complete. 2023-10-06T21:08:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.673+0000 7f87f61ff700 1 Processor -- start 2023-10-06T21:08:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 -- start start 2023-10-06T21:08:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0102a50 0x7f87f019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019e790 con 0x7f87f0102090 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019e900 con 0x7f87f0102a50 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87f61ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f019eac0 con 0x7f87f0100f40 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87eeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87ef7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87effff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0102a50 0x7f87f019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.674+0000 7f87ef7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42656/0 (socket says 172.21.15.26:42656) 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 -- 172.21.15.26:0/1212418771 learned_addr learned my addr 172.21.15.26:0/1212418771 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 -- 172.21.15.26:0/1212418771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0102a50 msgr2=0x7f87f019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0102a50 0x7f87f019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 -- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 msgr2=0x7f87f0197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 -- 172.21.15.26:0/1212418771 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40095f0 con 0x7f87f0100f40 2023-10-06T21:08:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ef7fe700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9a0 tx=0x7f87e000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009790 con 0x7f87f0100f40 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e0010430 con 0x7f87f0100f40 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f019eda0 con 0x7f87f0100f40 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.675+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0018680 con 0x7f87f0100f40 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.676+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f019f210 con 0x7f87f0100f40 2023-10-06T21:08:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.676+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f010c720 con 0x7f87f0100f40 2023-10-06T21:08:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.678+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f87e00105a0 con 0x7f87f0100f40 2023-10-06T21:08:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.680+0000 7f87ecff9700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 0x7f87d80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.680+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87e0014070 con 0x7f87f0100f40 2023-10-06T21:08:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.680+0000 7f87eeffd700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 0x7f87d80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.681+0000 7f87eeffd700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 0x7f87d80a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b2e0 tx=0x7f87e4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.682+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87e0096490 con 0x7f87f0100f40 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4085289816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/450344751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:13 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4085289816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/450344751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:08:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:13 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:13.923+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87f0107e80 con 0x7f87f0100f40 2023-10-06T21:08:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.498+0000 7f87ecff9700 1 -- 172.21.15.26:0/1212418771 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87e0099ab0 con 0x7f87f0100f40 2023-10-06T21:08:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.504+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 msgr2=0x7f87d80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.504+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 0x7f87d80a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b2e0 tx=0x7f87e4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.504+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 msgr2=0x7f87f0196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.504+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9a0 tx=0x7f87e000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.504+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 shutdown_connections 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0102a50 0x7f87f019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f0100f40 0x7f87f0196c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f87d80a1500 0x7f87d80a39c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 --2- 172.21.15.26:0/1212418771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87f0102090 0x7f87f0197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 >> 172.21.15.26:0/1212418771 conn(0x7f87f00fc4d0 msgr2=0x7f87f00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 shutdown_connections 2023-10-06T21:08:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.505+0000 7f87f61ff700 1 -- 172.21.15.26:0/1212418771 wait complete. 2023-10-06T21:08:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-10-06T21:08:14.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T21:08:14.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 -- 172.21.15.26:0/3060489320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078108e00 msgr2=0x7f307810b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:14.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 --2- 172.21.15.26:0/3060489320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078108e00 0x7f307810b290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f307000b0a0 tx=0x7f307000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 -- 172.21.15.26:0/3060489320 shutdown_connections 2023-10-06T21:08:14.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 --2- 172.21.15.26:0/3060489320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078108e00 0x7f307810b290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 --2- 172.21.15.26:0/3060489320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078074ef0 0x7f3078073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 --2- 172.21.15.26:0/3060489320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078073910 0x7f3078073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 -- 172.21.15.26:0/3060489320 >> 172.21.15.26:0/3060489320 conn(0x7f30780fc4d0 msgr2=0x7f30780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:14.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.738+0000 7f307dfff700 1 -- 172.21.15.26:0/3060489320 shutdown_connections 2023-10-06T21:08:14.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.739+0000 7f307dfff700 1 -- 172.21.15.26:0/3060489320 wait complete. 2023-10-06T21:08:14.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 Processor -- start 2023-10-06T21:08:14.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 -- start start 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078073910 0x7f3078198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078108e00 0x7f30781a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30781a0a80 con 0x7f3078108e00 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30781a0bc0 con 0x7f3078074ef0 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f307dfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30781a0d50 con 0x7f3078073910 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.740+0000 7f3076ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3077fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078108e00 0x7f30781a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35860/0 (socket says 172.21.15.26:35860) 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 -- 172.21.15.26:0/3105781451 learned_addr learned my addr 172.21.15.26:0/3105781451 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:14.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f30777fe700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078073910 0x7f3078198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:14.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 -- 172.21.15.26:0/3105781451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078073910 msgr2=0x7f3078198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:14.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078073910 0x7f3078198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 -- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078108e00 msgr2=0x7f30781a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:14.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078108e00 0x7f30781a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:14.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 -- 172.21.15.26:0/3105781451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070009d00 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3076ffd700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e970 tx=0x7f306c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.741+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c009790 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.742+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f306c00c7e0 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.742+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30781a0fd0 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.742+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003c70 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.742+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30781a14b0 con 0x7f3078074ef0 2023-10-06T21:08:14.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.743+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3078107bc0 con 0x7f3078074ef0 2023-10-06T21:08:14.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.745+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f306c00cc50 con 0x7f3078074ef0 2023-10-06T21:08:14.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.746+0000 7f3074ff9700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 0x7f30600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:14.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.747+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f306c014070 con 0x7f3078074ef0 2023-10-06T21:08:14.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.747+0000 7f30777fe700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 0x7f30600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:14.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.747+0000 7f30777fe700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 0x7f30600a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3068009c60 tx=0x7f3068009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:14.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.749+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f306c0968a0 con 0x7f3078074ef0 2023-10-06T21:08:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:14 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/881061735' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:08:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:14 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/881061735' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:08:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:14 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/881061735' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-06T21:08:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:14.966+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f30780008d0 con 0x7f3078074ef0 2023-10-06T21:08:15.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.506+0000 7f3074ff9700 1 -- 172.21.15.26:0/3105781451 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f306c096030 con 0x7f3078074ef0 2023-10-06T21:08:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.515+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 msgr2=0x7f30600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.515+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 0x7f30600a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3068009c60 tx=0x7f3068009260 comp rx=0 tx=0).stop 2023-10-06T21:08:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.515+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 msgr2=0x7f3078199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.515+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e970 tx=0x7f306c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 shutdown_connections 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f30600a1480 0x7f30600a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3078108e00 0x7f30781a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3078073910 0x7f3078198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 --2- 172.21.15.26:0/3105781451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3078074ef0 0x7f3078199440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.521+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 >> 172.21.15.26:0/3105781451 conn(0x7f30780fc4d0 msgr2=0x7f3078104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.526+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 shutdown_connections 2023-10-06T21:08:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.527+0000 7f307dfff700 1 -- 172.21.15.26:0/3105781451 wait complete. 2023-10-06T21:08:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-10-06T21:08:15.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T21:08:15.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/698633443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24074ef0 msgr2=0x7f7b24073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:15.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/698633443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24074ef0 0x7f7b24073340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/698633443 shutdown_connections 2023-10-06T21:08:15.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/698633443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 0x7f7b2410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/698633443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24074ef0 0x7f7b24073340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/698633443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24073910 0x7f7b24073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/698633443 >> 172.21.15.26:0/698633443 conn(0x7f7b240fc460 msgr2=0x7f7b240fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/698633443 shutdown_connections 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.766+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/698633443 wait complete. 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.768+0000 7f7b2c5ff700 1 Processor -- start 2023-10-06T21:08:15.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.768+0000 7f7b2c5ff700 1 -- start start 2023-10-06T21:08:15.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.768+0000 7f7b2c5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:15.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2c5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24074ef0 0x7f7b2419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2c5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 0x7f7b241a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2c5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b241a4d40 con 0x7f7b2410d4c0 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2c5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b241a4e80 con 0x7f7b24074ef0 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2c5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b241a5010 con 0x7f7b24073910 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2ab9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 0x7f7b241a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42702/0 (socket says 172.21.15.26:42702) 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 -- 172.21.15.26:0/34818820 learned_addr learned my addr 172.21.15.26:0/34818820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b29b9a700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24074ef0 0x7f7b2419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 -- 172.21.15.26:0/34818820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24074ef0 msgr2=0x7f7b2419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:15.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24074ef0 0x7f7b2419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 -- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 msgr2=0x7f7b241a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 0x7f7b241a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.769+0000 7f7b2a39b700 1 -- 172.21.15.26:0/34818820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200095f0 con 0x7f7b24073910 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b2a39b700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b200059d0 tx=0x7f7b20004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2001d070 con 0x7f7b24073910 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2000fb90 con 0x7f7b24073910 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b241a5290 con 0x7f7b24073910 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b200173f0 con 0x7f7b24073910 2023-10-06T21:08:15.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.770+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b241a5710 con 0x7f7b24073910 2023-10-06T21:08:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.771+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b24103010 con 0x7f7b24073910 2023-10-06T21:08:15.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.773+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7b2000fd00 con 0x7f7b24073910 2023-10-06T21:08:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.774+0000 7f7b1b7fe700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 0x7f7b100a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.775+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f7b200cdf00 con 0x7f7b24073910 2023-10-06T21:08:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.775+0000 7f7b29b9a700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 0x7f7b100a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:15.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.775+0000 7f7b29b9a700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 0x7f7b100a39c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b24074c10 tx=0x7f7b14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:15.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:15.776+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7b20097700 con 0x7f7b24073910 2023-10-06T21:08:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1212418771' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:08:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:15 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1212418771' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:08:15.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:15 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:15.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1212418771' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: osdmap e82: 12 total, 12 up, 12 in 2023-10-06T21:08:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:15 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:08:16.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.062+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7b24112dd0 con 0x7f7b24073910 2023-10-06T21:08:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.532+0000 7f7b1b7fe700 1 -- 172.21.15.26:0/34818820 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7b2009ad20 con 0x7f7b24073910 2023-10-06T21:08:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 msgr2=0x7f7b100a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 0x7f7b100a39c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b24074c10 tx=0x7f7b14009260 comp rx=0 tx=0).stop 2023-10-06T21:08:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 msgr2=0x7f7b2419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b200059d0 tx=0x7f7b20004730 comp rx=0 tx=0).stop 2023-10-06T21:08:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 shutdown_connections 2023-10-06T21:08:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7b100a1500 0x7f7b100a39c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b2410d4c0 0x7f7b241a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24073910 0x7f7b2419d1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 --2- 172.21.15.26:0/34818820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b24074ef0 0x7f7b2419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.536+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 >> 172.21.15.26:0/34818820 conn(0x7f7b240fc460 msgr2=0x7f7b240fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:16.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.537+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 shutdown_connections 2023-10-06T21:08:16.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.537+0000 7f7b2c5ff700 1 -- 172.21.15.26:0/34818820 wait complete. 2023-10-06T21:08:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2023-10-06T21:08:16.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 -- 172.21.15.26:0/1573541693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 msgr2=0x7fec641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 --2- 172.21.15.26:0/1573541693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec641012c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec54009990 tx=0x7fec54009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 -- 172.21.15.26:0/1573541693 shutdown_connections 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 --2- 172.21.15.26:0/1573541693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102a40 0x7fec64109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 --2- 172.21.15.26:0/1573541693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec641012c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 --2- 172.21.15.26:0/1573541693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102080 0x7fec64102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 -- 172.21.15.26:0/1573541693 >> 172.21.15.26:0/1573541693 conn(0x7fec640fc420 msgr2=0x7fec640fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:16.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 -- 172.21.15.26:0/1573541693 shutdown_connections 2023-10-06T21:08:16.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.773+0000 7fec69dff700 1 -- 172.21.15.26:0/1573541693 wait complete. 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.774+0000 7fec69dff700 1 Processor -- start 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 -- start start 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec64196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419e7b0 con 0x7fec64102a40 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419e920 con 0x7fec64100ea0 2023-10-06T21:08:16.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec69dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419eae0 con 0x7fec64102080 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec63fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec62ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec63fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60820/0 (socket says 172.21.15.26:60820) 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec63fff700 1 -- 172.21.15.26:0/747557432 learned_addr learned my addr 172.21.15.26:0/747557432 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec62ffd700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42716/0 (socket says 172.21.15.26:42716) 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.775+0000 7fec637fe700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec64196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 -- 172.21.15.26:0/747557432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 msgr2=0x7fec641971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 -- 172.21.15.26:0/747557432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 msgr2=0x7fec64196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec64196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 -- 172.21.15.26:0/747557432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec540095f0 con 0x7fec64102a40 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec637fe700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec64196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec62ffd700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec63fff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007ee0 tx=0x7fec5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00dd20 con 0x7fec64102a40 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5c00f040 con 0x7fec64102a40 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6419edc0 con 0x7fec64102a40 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0048a0 con 0x7fec64102a40 2023-10-06T21:08:16.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.776+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6419f1c0 con 0x7fec64102a40 2023-10-06T21:08:16.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.777+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64108530 con 0x7fec64102a40 2023-10-06T21:08:16.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.780+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fec5c003a00 con 0x7fec64102a40 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3105781451' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/34818820' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:16 smithi167 ceph-mon[134494]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:08:16.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.782+0000 7fec60ff9700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 0x7fec4c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:16.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.782+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec5c0ccd00 con 0x7fec64102a40 2023-10-06T21:08:16.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.782+0000 7fec637fe700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 0x7fec4c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:16.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.783+0000 7fec637fe700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 0x7fec4c0a3990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec54005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:16.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:16.785+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec5c096410 con 0x7fec64102a40 2023-10-06T21:08:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3105781451' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:08:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:08:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/34818820' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:16 smithi026 ceph-mon[98490]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3105781451' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: osdmap e83: 12 total, 12 up, 12 in 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/34818820' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-06T21:08:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:16 smithi078 ceph-mon[134896]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-06T21:08:17.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.015+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec6410c930 con 0x7fec64102a40 2023-10-06T21:08:17.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.017+0000 7fec60ff9700 1 -- 172.21.15.26:0/747557432 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22082 (secure 0 0 0) 0x7fec5c023090 con 0x7fec64102a40 2023-10-06T21:08:17.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:17.020 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:16.065241+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 msgr2=0x7fec4c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 0x7fec4c0a3990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec54005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 msgr2=0x7fec6419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007ee0 tx=0x7fec5c00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 shutdown_connections 2023-10-06T21:08:17.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fec4c0a14d0 0x7fec4c0a3990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec64102a40 0x7fec6419e1e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec64100ea0 0x7fec64196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 --2- 172.21.15.26:0/747557432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec64102080 0x7fec641971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 >> 172.21.15.26:0/747557432 conn(0x7fec640fc420 msgr2=0x7fec64105670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.019+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 shutdown_connections 2023-10-06T21:08:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.020+0000 7fec69dff700 1 -- 172.21.15.26:0/747557432 wait complete. 2023-10-06T21:08:17.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:08:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 -- 172.21.15.26:0/783951939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102080 msgr2=0x7f5798102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 --2- 172.21.15.26:0/783951939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102080 0x7f5798102500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f578c009990 tx=0x7f578c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 -- 172.21.15.26:0/783951939 shutdown_connections 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 --2- 172.21.15.26:0/783951939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798102a40 0x7f5798109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 --2- 172.21.15.26:0/783951939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798100ea0 0x7f57981012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 --2- 172.21.15.26:0/783951939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102080 0x7f5798102500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.268+0000 7f579d5ff700 1 -- 172.21.15.26:0/783951939 >> 172.21.15.26:0/783951939 conn(0x7f57980fc420 msgr2=0x7f57980fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.269+0000 7f579d5ff700 1 -- 172.21.15.26:0/783951939 shutdown_connections 2023-10-06T21:08:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.269+0000 7f579d5ff700 1 -- 172.21.15.26:0/783951939 wait complete. 2023-10-06T21:08:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.269+0000 7f579d5ff700 1 Processor -- start 2023-10-06T21:08:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 -- start start 2023-10-06T21:08:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 0x7f5798198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 0x7f5798199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 0x7f57981a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57981a0970 con 0x7f5798102080 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57981a0ab0 con 0x7f5798102a40 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f579d5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57981a0c40 con 0x7f5798100ea0 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f57967fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 0x7f5798199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f57977fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 0x7f57981a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f5796ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 0x7f5798198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f57967fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 0x7f5798199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60848/0 (socket says 172.21.15.26:60848) 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.270+0000 7f57967fc700 1 -- 172.21.15.26:0/4026621013 learned_addr learned my addr 172.21.15.26:0/4026621013 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 -- 172.21.15.26:0/4026621013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 msgr2=0x7f5798198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 0x7f5798198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 -- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 msgr2=0x7f5798199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 0x7f5798199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 -- 172.21.15.26:0/4026621013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009520 con 0x7f5798102a40 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f5796ffd700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 0x7f5798198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f57977fe700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 0x7f57981a03a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f579000efc0 tx=0x7f579000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790018070 con 0x7f5798102a40 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579000f040 con 0x7f5798102a40 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c0095f0 con 0x7f5798102a40 2023-10-06T21:08:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000cd40 con 0x7f5798102a40 2023-10-06T21:08:17.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.271+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57981a11e0 con 0x7f5798102a40 2023-10-06T21:08:17.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.272+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579810c760 con 0x7f5798102a40 2023-10-06T21:08:17.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.274+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5790003a00 con 0x7f5798102a40 2023-10-06T21:08:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.275+0000 7f577ffff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 0x7f57800a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.276+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57900cc8d0 con 0x7f5798102a40 2023-10-06T21:08:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.276+0000 7f5796ffd700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 0x7f57800a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.277+0000 7f5796ffd700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 0x7f57800a3a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5788009fd0 tx=0x7f5788009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.278+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5790096070 con 0x7f5798102a40 2023-10-06T21:08:17.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.504+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57981a13d0 con 0x7f5798102a40 2023-10-06T21:08:17.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.505+0000 7f577ffff700 1 -- 172.21.15.26:0/4026621013 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22082 (secure 0 0 0) 0x7f579004d660 con 0x7f5798102a40 2023-10-06T21:08:17.508 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:17.508 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":84,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:16.065241+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:17.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 msgr2=0x7f57800a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 0x7f57800a3a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5788009fd0 tx=0x7f5788009260 comp rx=0 tx=0).stop 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 msgr2=0x7f57981a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 0x7f57981a03a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f579000efc0 tx=0x7f579000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 shutdown_connections 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5798102a40 0x7f57981a03a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5798100ea0 0x7f5798198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f57800a1550 0x7f57800a3a10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 --2- 172.21.15.26:0/4026621013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5798102080 0x7f5798199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 >> 172.21.15.26:0/4026621013 conn(0x7f57980fc420 msgr2=0x7f57980fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 shutdown_connections 2023-10-06T21:08:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.509+0000 7f579d5ff700 1 -- 172.21.15.26:0/4026621013 wait complete. 2023-10-06T21:08:17.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T21:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.742+0000 7f96b1dff700 1 -- 172.21.15.26:0/2854756446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 msgr2=0x7f96ac103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.742+0000 7f96b1dff700 1 --2- 172.21.15.26:0/2854756446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac103d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f969c009990 tx=0x7f969c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 -- 172.21.15.26:0/2854756446 shutdown_connections 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 --2- 172.21.15.26:0/2854756446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 0x7f96ac109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 --2- 172.21.15.26:0/2854756446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac103d70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 --2- 172.21.15.26:0/2854756446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 0x7f96ac1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 -- 172.21.15.26:0/2854756446 >> 172.21.15.26:0/2854756446 conn(0x7f96ac0fb370 msgr2=0x7f96ac0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 -- 172.21.15.26:0/2854756446 shutdown_connections 2023-10-06T21:08:17.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.743+0000 7f96b1dff700 1 -- 172.21.15.26:0/2854756446 wait complete. 2023-10-06T21:08:17.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.744+0000 7f96b1dff700 1 Processor -- start 2023-10-06T21:08:17.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.744+0000 7f96b1dff700 1 -- start start 2023-10-06T21:08:17.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 0x7f96ac0ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 0x7f96ac0ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac100610 con 0x7f96ac1042b0 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac100780 con 0x7f96ac106be0 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96b1dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac1a0c20 con 0x7f96ac101980 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96ab7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96aaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 0x7f96ac0ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96ab7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42740/0 (socket says 172.21.15.26:42740) 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96ab7fe700 1 -- 172.21.15.26:0/1887510836 learned_addr learned my addr 172.21.15.26:0/1887510836 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.745+0000 7f96abfff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 0x7f96ac0ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 -- 172.21.15.26:0/1887510836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 msgr2=0x7f96ac0ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 0x7f96ac0ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 -- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 msgr2=0x7f96ac0ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 0x7f96ac0ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:17.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 -- 172.21.15.26:0/1887510836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a0009520 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96ab7fe700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f969c004740 tx=0x7f969c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c01d070 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c004830 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a4009c40 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c0053c0 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.746+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0095f0 con 0x7f96ac101980 2023-10-06T21:08:17.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.748+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ac0fcef0 con 0x7f96ac101980 2023-10-06T21:08:17.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.749+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f969c005580 con 0x7f96ac101980 2023-10-06T21:08:17.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.751+0000 7f96a8ff9700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 0x7f96940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:17.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.751+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f969c0cdd20 con 0x7f96ac101980 2023-10-06T21:08:17.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.751+0000 7f96aaffd700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 0x7f96940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:17.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.752+0000 7f96aaffd700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 0x7f96940a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009df0 tx=0x7f96a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:17.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:17.755+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f969c05b100 con 0x7f96ac101980 2023-10-06T21:08:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: pgmap v269: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:08:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/747557432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:17 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: pgmap v269: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:08:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/747557432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:17 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: pgmap v269: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: osdmap e84: 12 total, 12 up, 12 in 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/747557432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:17 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.043+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f96ac10c730 con 0x7f96ac101980 2023-10-06T21:08:18.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4026621013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:active 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1887510836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:18.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4026621013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:active 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1887510836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:18.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:18.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.828+0000 7f96a8ff9700 1 -- 172.21.15.26:0/1887510836 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f969c0974c0 con 0x7f96ac101980 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 msgr2=0x7f96940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 0x7f96940a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009df0 tx=0x7f96a0009260 comp rx=0 tx=0).stop 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 msgr2=0x7f96ac1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f969c004740 tx=0x7f969c009580 comp rx=0 tx=0).stop 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 shutdown_connections 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96ac106be0 0x7f96ac0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101980 0x7f96ac1012f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96940a1500 0x7f96940a39c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 --2- 172.21.15.26:0/1887510836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96ac1042b0 0x7f96ac0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 >> 172.21.15.26:0/1887510836 conn(0x7f96ac0fb370 msgr2=0x7f96ac1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 shutdown_connections 2023-10-06T21:08:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:18.831+0000 7f96b1dff700 1 -- 172.21.15.26:0/1887510836 wait complete. 2023-10-06T21:08:18.853 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-06T21:08:18.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4026621013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: osdmap e85: 12 total, 12 up, 12 in 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] up:active 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1887510836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:08:18.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:08:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 -- 172.21.15.26:0/890419794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102a40 msgr2=0x7fd22c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 --2- 172.21.15.26:0/890419794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102a40 0x7fd22c109360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd224009f90 tx=0x7fd22400b330 comp rx=0 tx=0).stop 2023-10-06T21:08:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 -- 172.21.15.26:0/890419794 shutdown_connections 2023-10-06T21:08:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 --2- 172.21.15.26:0/890419794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102a40 0x7fd22c109360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 --2- 172.21.15.26:0/890419794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c100ea0 0x7fd22c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 --2- 172.21.15.26:0/890419794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c102080 0x7fd22c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 -- 172.21.15.26:0/890419794 >> 172.21.15.26:0/890419794 conn(0x7fd22c0fc460 msgr2=0x7fd22c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 -- 172.21.15.26:0/890419794 shutdown_connections 2023-10-06T21:08:19.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.057+0000 7fd2329ff700 1 -- 172.21.15.26:0/890419794 wait complete. 2023-10-06T21:08:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.058+0000 7fd2329ff700 1 Processor -- start 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.058+0000 7fd2329ff700 1 -- start start 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c100ea0 0x7fd22c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102080 0x7fd22c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c073f80 con 0x7fd22c102a40 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0740f0 con 0x7fd22c100ea0 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd2329ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c1a0d40 con 0x7fd22c102080 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd22b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102080 0x7fd22c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd22bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c100ea0 0x7fd22c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60904/0 (socket says 172.21.15.26:60904) 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 -- 172.21.15.26:0/1707215682 learned_addr learned my addr 172.21.15.26:0/1707215682 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 -- 172.21.15.26:0/1707215682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102080 msgr2=0x7fd22c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102080 0x7fd22c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 -- 172.21.15.26:0/1707215682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c100ea0 msgr2=0x7fd22c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c100ea0 0x7fd22c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.059+0000 7fd230f9c700 1 -- 172.21.15.26:0/1707215682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224009c40 con 0x7fd22c102a40 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd230f9c700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd224009f60 tx=0x7fd224004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22400e040 con 0x7fd22c102a40 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c1a0fc0 con 0x7fd22c102a40 2023-10-06T21:08:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd22400bbf0 con 0x7fd22c102a40 2023-10-06T21:08:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2240052a0 con 0x7fd22c102a40 2023-10-06T21:08:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.060+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c1a13f0 con 0x7fd22c102a40 2023-10-06T21:08:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.061+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c108410 con 0x7fd22c102a40 2023-10-06T21:08:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.063+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd22401d470 con 0x7fd22c102a40 2023-10-06T21:08:19.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.065+0000 7fd2297fa700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.066+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2240cdfe0 con 0x7fd22c102a40 2023-10-06T21:08:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.066+0000 7fd22bfff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:19.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.066+0000 7fd22bfff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 0x7fd2140a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd22c101bf0 tx=0x7fd21c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:19.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.070+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd2240977e0 con 0x7fd22c102a40 2023-10-06T21:08:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.329+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fd22c0008d0 con 0x7fd22c102a40 2023-10-06T21:08:19.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: pgmap v272: 97 pgs: 48 active+clean, 19 creating+peering, 30 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-10-06T21:08:19.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:08:19.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:19.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-06T21:08:19.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T21:08:19.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:19 smithi167 ceph-mon[134494]: Cluster is now healthy 2023-10-06T21:08:19.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: pgmap v272: 97 pgs: 48 active+clean, 19 creating+peering, 30 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-10-06T21:08:19.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:08:19.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:19.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-06T21:08:19.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T21:08:19.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:19 smithi026 ceph-mon[98490]: Cluster is now healthy 2023-10-06T21:08:19.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.827+0000 7fd2297fa700 1 -- 172.21.15.26:0/1707215682 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd22409ae00 con 0x7fd22c102a40 2023-10-06T21:08:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 msgr2=0x7fd2140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 0x7fd2140a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd22c101bf0 tx=0x7fd21c009310 comp rx=0 tx=0).stop 2023-10-06T21:08:19.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 msgr2=0x7fd22c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd224009f60 tx=0x7fd224004c60 comp rx=0 tx=0).stop 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 shutdown_connections 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd22c102a40 0x7fd22c0737e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd22c100ea0 0x7fd22c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 --2- 172.21.15.26:0/1707215682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd22c102080 0x7fd22c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 >> 172.21.15.26:0/1707215682 conn(0x7fd22c0fc460 msgr2=0x7fd22c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 shutdown_connections 2023-10-06T21:08:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:19.830+0000 7fd2329ff700 1 -- 172.21.15.26:0/1707215682 wait complete. 2023-10-06T21:08:19.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-06T21:08:19.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: pgmap v272: 97 pgs: 48 active+clean, 19 creating+peering, 30 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2023-10-06T21:08:19.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-06T21:08:19.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:19.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-06T21:08:19.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-06T21:08:19.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:19 smithi078 ceph-mon[134896]: Cluster is now healthy 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 -- 172.21.15.26:0/3208775945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 msgr2=0x7f49041024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 --2- 172.21.15.26:0/3208775945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f49041024f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009990 tx=0x7f48ec009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 -- 172.21.15.26:0/3208775945 shutdown_connections 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 --2- 172.21.15.26:0/3208775945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904102a30 0x7f4904109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 --2- 172.21.15.26:0/3208775945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904100e90 0x7f49041012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 --2- 172.21.15.26:0/3208775945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f49041024f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 -- 172.21.15.26:0/3208775945 >> 172.21.15.26:0/3208775945 conn(0x7f49040fc430 msgr2=0x7f49040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 -- 172.21.15.26:0/3208775945 shutdown_connections 2023-10-06T21:08:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.069+0000 7f490a7ff700 1 -- 172.21.15.26:0/3208775945 wait complete. 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 Processor -- start 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 -- start start 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 0x7f4904196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f4904197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904102a30 0x7f490419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419e770 con 0x7f4904102070 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419e8e0 con 0x7f4904102a30 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.070+0000 7f490a7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419eaa0 con 0x7f4904100e90 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f4904197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 0x7f4904196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4908d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904102a30 0x7f490419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 0x7f4904196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42780/0 (socket says 172.21.15.26:42780) 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f48fbfff700 1 -- 172.21.15.26:0/2662375679 learned_addr learned my addr 172.21.15.26:0/2662375679 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 -- 172.21.15.26:0/2662375679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 msgr2=0x7f4904196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 0x7f4904196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 -- 172.21.15.26:0/2662375679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904102a30 msgr2=0x7f490419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904102a30 0x7f490419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 -- 172.21.15.26:0/2662375679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0095f0 con 0x7f4904102070 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.071+0000 7f4903fff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f4904197170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0098e0 tx=0x7f48ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.072+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec01d070 con 0x7f4904102070 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.072+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48ec021410 con 0x7f4904102070 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.072+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec005720 con 0x7f4904102070 2023-10-06T21:08:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.072+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490419ed20 con 0x7f4904102070 2023-10-06T21:08:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.072+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490419f0f0 con 0x7f4904102070 2023-10-06T21:08:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.073+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4904107e90 con 0x7f4904102070 2023-10-06T21:08:20.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.075+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f48ec021580 con 0x7f4904102070 2023-10-06T21:08:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.076+0000 7f4901ffb700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 0x7f48f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.077+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48ec0ce590 con 0x7f4904102070 2023-10-06T21:08:20.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.077+0000 7f48fbfff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 0x7f48f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:20.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.077+0000 7f48fbfff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 0x7f48f00a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4904198100 tx=0x7f48f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:20.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.078+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f48ec097d30 con 0x7f4904102070 2023-10-06T21:08:20.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.338+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f490402d190 con 0x7f4904102070 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-10-06T21:08:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:20 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:08:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:08:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-06T21:08:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:08:20.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:20.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-10-06T21:08:20.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-10-06T21:08:20.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:20 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:08:20.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.839+0000 7f4901ffb700 1 -- 172.21.15.26:0/2662375679 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f48ec0b9e60 con 0x7f4904102070 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 msgr2=0x7f48f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 0x7f48f00a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4904198100 tx=0x7f48f4009260 comp rx=0 tx=0).stop 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 msgr2=0x7f4904197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f4904197170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0098e0 tx=0x7f48ec009580 comp rx=0 tx=0).stop 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 shutdown_connections 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4904102a30 0x7f490419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4904100e90 0x7f4904196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f48f00a1480 0x7f48f00a3940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 --2- 172.21.15.26:0/2662375679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4904102070 0x7f4904197170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 >> 172.21.15.26:0/2662375679 conn(0x7f49040fc430 msgr2=0x7f49040fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 shutdown_connections 2023-10-06T21:08:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:20.843+0000 7f490a7ff700 1 -- 172.21.15.26:0/2662375679 wait complete. 2023-10-06T21:08:20.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-06T21:08:20.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1707215682' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-06T21:08:20.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-06T21:08:20.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-06T21:08:20.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-10-06T21:08:20.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-10-06T21:08:20.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-06T21:08:20.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-06T21:08:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 -- 172.21.15.26:0/1607218227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 msgr2=0x7f38f01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 --2- 172.21.15.26:0/1607218227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f01024f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009990 tx=0x7f38e4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 -- 172.21.15.26:0/1607218227 shutdown_connections 2023-10-06T21:08:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 --2- 172.21.15.26:0/1607218227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0102a30 0x7f38f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 --2- 172.21.15.26:0/1607218227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0100e90 0x7f38f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 --2- 172.21.15.26:0/1607218227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f01024f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 -- 172.21.15.26:0/1607218227 >> 172.21.15.26:0/1607218227 conn(0x7f38f00fc430 msgr2=0x7f38f00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 -- 172.21.15.26:0/1607218227 shutdown_connections 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.080+0000 7f38f63ff700 1 -- 172.21.15.26:0/1607218227 wait complete. 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.081+0000 7f38f63ff700 1 Processor -- start 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.081+0000 7f38f63ff700 1 -- start start 2023-10-06T21:08:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0100e90 0x7f38f0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0102a30 0x7f38f01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a09c0 con 0x7f38f0102070 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a0b00 con 0x7f38f0100e90 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f63ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01a0c90 con 0x7f38f0102a30 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38ef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43638/0 (socket says 172.21.15.26:43638) 2023-10-06T21:08:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38ef7fe700 1 -- 172.21.15.26:0/4210845642 learned_addr learned my addr 172.21.15.26:0/4210845642 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38effff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0100e90 0x7f38f0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38f499c700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0102a30 0x7f38f01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.082+0000 7f38ef7fe700 1 -- 172.21.15.26:0/4210845642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0102a30 msgr2=0x7f38f01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ef7fe700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0102a30 0x7f38f01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ef7fe700 1 -- 172.21.15.26:0/4210845642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0100e90 msgr2=0x7f38f0198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ef7fe700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0100e90 0x7f38f0198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ef7fe700 1 -- 172.21.15.26:0/4210845642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40095f0 con 0x7f38f0102070 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ef7fe700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f38e4004710 tx=0x7f38e4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e401d070 con 0x7f38f0102070 2023-10-06T21:08:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e4021410 con 0x7f38f0102070 2023-10-06T21:08:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4017650 con 0x7f38f0102070 2023-10-06T21:08:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01a0f10 con 0x7f38f0102070 2023-10-06T21:08:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.083+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f01a1280 con 0x7f38f0102070 2023-10-06T21:08:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.084+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f010c7c0 con 0x7f38f0102070 2023-10-06T21:08:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.086+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f38e4021580 con 0x7f38f0102070 2023-10-06T21:08:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.087+0000 7f38ed7fa700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 0x7f38d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.088+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f38e40ce6e0 con 0x7f38f0102070 2023-10-06T21:08:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.088+0000 7f38effff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 0x7f38d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.089+0000 7f38effff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 0x7f38d80a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38f0101be0 tx=0x7f38e0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.090+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f38e4097e80 con 0x7f38f0102070 2023-10-06T21:08:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.349+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f38f01a1410 con 0x7f38f0102070 2023-10-06T21:08:21.370 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: 2023-10-06T21:08:20.951+0000 7fe7e357b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T21:08:21.371 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:08:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: 2023-10-06T21:08:20.951+0000 7fe7e357b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: pgmap v273: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:active 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: Cluster is now healthy 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-10-06T21:08:21.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-06T21:08:21.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:21 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:08:21.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: pgmap v273: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-06T21:08:21.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:active 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: Cluster is now healthy 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-06T21:08:21.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:21 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:08:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.845+0000 7f38ed7fa700 1 -- 172.21.15.26:0/4210845642 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f38e409b4a0 con 0x7f38f0102070 2023-10-06T21:08:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.848+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 msgr2=0x7f38d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.848+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 0x7f38d80a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38f0101be0 tx=0x7f38e0009260 comp rx=0 tx=0).stop 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.848+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 msgr2=0x7f38f0199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.848+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f38e4004710 tx=0x7f38e4009580 comp rx=0 tx=0).stop 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 shutdown_connections 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38f0102a30 0x7f38f01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38f0100e90 0x7f38f0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f38d80a1480 0x7f38d80a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 --2- 172.21.15.26:0/4210845642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f0102070 0x7f38f0199380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 >> 172.21.15.26:0/4210845642 conn(0x7f38f00fc430 msgr2=0x7f38f00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 shutdown_connections 2023-10-06T21:08:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:21.849+0000 7f38f63ff700 1 -- 172.21.15.26:0/4210845642 wait complete. 2023-10-06T21:08:21.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: pgmap v273: 97 pgs: 89 active+clean, 8 creating+peering; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2662375679' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] up:active 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: Cluster is now healthy 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-06T21:08:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:21 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-06T21:08:21.873 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-06T21:08:21.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-06T21:08:22.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.079+0000 7fe0729ff700 1 -- 172.21.15.26:0/2039297181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 msgr2=0x7fe06c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.079+0000 7fe0729ff700 1 --2- 172.21.15.26:0/2039297181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1024f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe054009990 tx=0x7fe054009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 -- 172.21.15.26:0/2039297181 shutdown_connections 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 --2- 172.21.15.26:0/2039297181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c102a30 0x7fe06c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 --2- 172.21.15.26:0/2039297181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c100e90 0x7fe06c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 --2- 172.21.15.26:0/2039297181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1024f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 -- 172.21.15.26:0/2039297181 >> 172.21.15.26:0/2039297181 conn(0x7fe06c0fc430 msgr2=0x7fe06c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 -- 172.21.15.26:0/2039297181 shutdown_connections 2023-10-06T21:08:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.080+0000 7fe0729ff700 1 -- 172.21.15.26:0/2039297181 wait complete. 2023-10-06T21:08:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 Processor -- start 2023-10-06T21:08:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 -- start start 2023-10-06T21:08:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c100e90 0x7fe06c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c102a30 0x7fe06c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1a0a20 con 0x7fe06c102070 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1a0b60 con 0x7fe06c100e90 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe0729ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1a0cf0 con 0x7fe06c102a30 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe06bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe06bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43654/0 (socket says 172.21.15.26:43654) 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe06bfff700 1 -- 172.21.15.26:0/502751749 learned_addr learned my addr 172.21.15.26:0/502751749 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.081+0000 7fe070f9c700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c102a30 0x7fe06c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe063fff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c100e90 0x7fe06c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 -- 172.21.15.26:0/502751749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c102a30 msgr2=0x7fe06c1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c102a30 0x7fe06c1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 -- 172.21.15.26:0/502751749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c100e90 msgr2=0x7fe06c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c100e90 0x7fe06c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 -- 172.21.15.26:0/502751749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540095f0 con 0x7fe06c102070 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe06bfff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe054004710 tx=0x7fe054009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05401d070 con 0x7fe06c102070 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe054021410 con 0x7fe06c102070 2023-10-06T21:08:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054017650 con 0x7fe06c102070 2023-10-06T21:08:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c1a0f70 con 0x7fe06c102070 2023-10-06T21:08:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.082+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c1a12e0 con 0x7fe06c102070 2023-10-06T21:08:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.084+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c193540 con 0x7fe06c102070 2023-10-06T21:08:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.086+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe054021580 con 0x7fe06c102070 2023-10-06T21:08:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.087+0000 7fe069ffb700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 0x7fe0580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.087+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0540cdfa0 con 0x7fe06c102070 2023-10-06T21:08:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.087+0000 7fe063fff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 0x7fe0580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.088+0000 7fe063fff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 0x7fe0580a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe06c101be0 tx=0x7fe05c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:22.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.091+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe054097740 con 0x7fe06c102070 2023-10-06T21:08:22.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.333+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fe06c02d320 con 0x7fe0580a1500 2023-10-06T21:08:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.382+0000 7fe069ffb700 1 -- 172.21.15.26:0/502751749 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe06c02d320 con 0x7fe0580a1500 2023-10-06T21:08:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.384+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 msgr2=0x7fe0580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.384+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 0x7fe0580a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe06c101be0 tx=0x7fe05c009260 comp rx=0 tx=0).stop 2023-10-06T21:08:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 msgr2=0x7fe06c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe054004710 tx=0x7fe054009580 comp rx=0 tx=0).stop 2023-10-06T21:08:22.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 shutdown_connections 2023-10-06T21:08:22.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe06c102a30 0x7fe06c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe06c100e90 0x7fe06c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe0580a1500 0x7fe0580a39c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 --2- 172.21.15.26:0/502751749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe06c102070 0x7fe06c1993e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 >> 172.21.15.26:0/502751749 conn(0x7fe06c0fc430 msgr2=0x7fe06c107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 shutdown_connections 2023-10-06T21:08:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.385+0000 7fe0729ff700 1 -- 172.21.15.26:0/502751749 wait complete. 2023-10-06T21:08:22.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-06T21:08:22.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 -- 172.21.15.26:0/1098529841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74100ea0 msgr2=0x7fce741012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 --2- 172.21.15.26:0/1098529841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74100ea0 0x7fce741012c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce64009990 tx=0x7fce64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 -- 172.21.15.26:0/1098529841 shutdown_connections 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 --2- 172.21.15.26:0/1098529841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 0x7fce74109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 --2- 172.21.15.26:0/1098529841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74100ea0 0x7fce741012c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 --2- 172.21.15.26:0/1098529841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74102080 0x7fce74102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 -- 172.21.15.26:0/1098529841 >> 172.21.15.26:0/1098529841 conn(0x7fce740fc460 msgr2=0x7fce740fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 -- 172.21.15.26:0/1098529841 shutdown_connections 2023-10-06T21:08:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.611+0000 7fce78dff700 1 -- 172.21.15.26:0/1098529841 wait complete. 2023-10-06T21:08:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.612+0000 7fce78dff700 1 Processor -- start 2023-10-06T21:08:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.612+0000 7fce78dff700 1 -- start start 2023-10-06T21:08:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74102080 0x7fce740732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 0x7fce740737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7419aaf0 con 0x7fce74100ea0 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74073d20 con 0x7fce74102a40 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce78dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74073ee0 con 0x7fce74102080 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43686/0 (socket says 172.21.15.26:43686) 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 -- 172.21.15.26:0/1870864384 learned_addr learned my addr 172.21.15.26:0/1870864384 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce72d9d700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 0x7fce740737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce71d9b700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74102080 0x7fce740732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 -- 172.21.15.26:0/1870864384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74102080 msgr2=0x7fce740732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74102080 0x7fce740732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 -- 172.21.15.26:0/1870864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 msgr2=0x7fce740737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 0x7fce740737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 -- 172.21.15.26:0/1870864384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640095f0 con 0x7fce74100ea0 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.613+0000 7fce7259c700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce64004710 tx=0x7fce64009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.614+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6401d070 con 0x7fce74100ea0 2023-10-06T21:08:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.614+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce741a54a0 con 0x7fce74100ea0 2023-10-06T21:08:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.614+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce64021410 con 0x7fce74100ea0 2023-10-06T21:08:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.614+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64017650 con 0x7fce74100ea0 2023-10-06T21:08:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.614+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce741a5810 con 0x7fce74100ea0 2023-10-06T21:08:22.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.615+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce74107ed0 con 0x7fce74100ea0 2023-10-06T21:08:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.617+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fce64021580 con 0x7fce74100ea0 2023-10-06T21:08:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.618+0000 7fce637fe700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 0x7fce5c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.618+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fce640ce750 con 0x7fce74100ea0 2023-10-06T21:08:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.618+0000 7fce71d9b700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 0x7fce5c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.619+0000 7fce71d9b700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 0x7fce5c0a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce74100bc0 tx=0x7fce68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:22.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.620+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fce64097ef0 con 0x7fce74100ea0 2023-10-06T21:08:22.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:08:22.339+0000 7fc585e75700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:08:22.824 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:08:22.339+0000 7fc585e75700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:08:22.824 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:08:22.339+0000 7fc585e75700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:08:22.824 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:08:22.339+0000 7fc585e75700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:08:22.824 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:08:22.339+0000 7fc585e75700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:08:22.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.865+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fce640982c0 con 0x7fce74100ea0 2023-10-06T21:08:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.877+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fce7402d160 con 0x7fce5c0a1480 2023-10-06T21:08:22.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.881+0000 7fce637fe700 1 -- 172.21.15.26:0/1870864384 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fce7402d160 con 0x7fce5c0a1480 2023-10-06T21:08:22.884 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv_0/fee3bfe4-641d-4e4e-b6e7-d7aeebde4585 2023-10-06T21:08:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 msgr2=0x7fce5c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 0x7fce5c0a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce74100bc0 tx=0x7fce68009260 comp rx=0 tx=0).stop 2023-10-06T21:08:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 msgr2=0x7fce74074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce64004710 tx=0x7fce64009580 comp rx=0 tx=0).stop 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 shutdown_connections 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce74102a40 0x7fce740737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fce5c0a1480 0x7fce5c0a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce74100ea0 0x7fce74074c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 --2- 172.21.15.26:0/1870864384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce74102080 0x7fce740732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.883+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 >> 172.21.15.26:0/1870864384 conn(0x7fce740fc460 msgr2=0x7fce740fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.884+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 shutdown_connections 2023-10-06T21:08:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:22.884+0000 7fce78dff700 1 -- 172.21.15.26:0/1870864384 wait complete. 2023-10-06T21:08:22.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-06T21:08:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.110+0000 7f56cfdff700 1 -- 172.21.15.26:0/1331835311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 msgr2=0x7f56c80732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.110+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1331835311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c80732e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009990 tx=0x7f56c4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 -- 172.21.15.26:0/1331835311 shutdown_connections 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1331835311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c8108de0 0x7f56c810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1331835311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c80732e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1331835311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c80738b0 0x7f56c8073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 -- 172.21.15.26:0/1331835311 >> 172.21.15.26:0/1331835311 conn(0x7f56c80fc460 msgr2=0x7f56c80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 -- 172.21.15.26:0/1331835311 shutdown_connections 2023-10-06T21:08:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.111+0000 7f56cfdff700 1 -- 172.21.15.26:0/1331835311 wait complete. 2023-10-06T21:08:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 Processor -- start 2023-10-06T21:08:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 -- start start 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 0x7f56c8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c8108de0 0x7f56c81a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c81a0950 con 0x7f56c8074e90 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c81a0a90 con 0x7f56c8108de0 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cfdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c81a0c20 con 0x7f56c80738b0 2023-10-06T21:08:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cdb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 0x7f56c8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cdb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 0x7f56c8198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33096/0 (socket says 172.21.15.26:33096) 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cdb9b700 1 -- 172.21.15.26:0/1075568638 learned_addr learned my addr 172.21.15.26:0/1075568638 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.112+0000 7f56cd39a700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56ce39c700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c8108de0 0x7f56c81a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 -- 172.21.15.26:0/1075568638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 msgr2=0x7f56c8198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 0x7f56c8198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 -- 172.21.15.26:0/1075568638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c8108de0 msgr2=0x7f56c81a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c8108de0 0x7f56c81a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 -- 172.21.15.26:0/1075568638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40095f0 con 0x7f56c8074e90 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cd39a700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c8199310 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800f890 con 0x7f56c8074e90 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b8010460 con 0x7f56c8074e90 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800e610 con 0x7f56c8074e90 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.113+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c81a0f00 con 0x7f56c8074e90 2023-10-06T21:08:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.114+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c81a1380 con 0x7f56c8074e90 2023-10-06T21:08:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.115+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c81074e0 con 0x7f56c8074e90 2023-10-06T21:08:23.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:08:23.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:active 2023-10-06T21:08:23.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2023-10-06T21:08:23.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:08:23.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:23.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:22 smithi078 ceph-mon[134896]: pgmap v274: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-06T21:08:23.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.116+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f56b80105d0 con 0x7f56c8074e90 2023-10-06T21:08:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.118+0000 7f56beffd700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 0x7f56b40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.118+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56b80cc9a0 con 0x7f56c8074e90 2023-10-06T21:08:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.118+0000 7f56cdb9b700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 0x7f56b40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.119+0000 7f56cdb9b700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 0x7f56b40a3a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56c400b5c0 tx=0x7f56c4005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:23.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.122+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f56b8095f50 con 0x7f56c8074e90 2023-10-06T21:08:23.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:08:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:active 2023-10-06T21:08:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2023-10-06T21:08:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:08:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:22 smithi167 ceph-mon[134494]: pgmap v274: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-06T21:08:23.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4210845642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-06T21:08:23.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] up:active 2023-10-06T21:08:23.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2023-10-06T21:08:23.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:08:23.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:23.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:22 smithi026 ceph-mon[98490]: pgmap v274: 97 pgs: 97 active+clean; 580 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 2.8 KiB/s wr, 10 op/s 2023-10-06T21:08:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.367+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c802d350 con 0x7f56b40a1590 2023-10-06T21:08:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.394+0000 7f56beffd700 1 -- 172.21.15.26:0/1075568638 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f56c802d350 con 0x7f56b40a1590 2023-10-06T21:08:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.396+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 msgr2=0x7f56b40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.396+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 0x7f56b40a3a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56c400b5c0 tx=0x7f56c4005bc0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.396+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 msgr2=0x7f56c8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.396+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c8199310 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800dc60 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.396+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 shutdown_connections 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56c8108de0 0x7f56c81a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56c80738b0 0x7f56c8198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56b40a1590 0x7f56b40a3a50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 --2- 172.21.15.26:0/1075568638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c8074e90 0x7f56c8199310 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 >> 172.21.15.26:0/1075568638 conn(0x7f56c80fc460 msgr2=0x7f56c80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 shutdown_connections 2023-10-06T21:08:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.397+0000 7f56cfdff700 1 -- 172.21.15.26:0/1075568638 wait complete. 2023-10-06T21:08:23.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-06T21:08:23.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/2807597483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54102080 msgr2=0x7fbb54102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/2807597483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54102080 0x7fbb54102500 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009990 tx=0x7fbb44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/2807597483 shutdown_connections 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/2807597483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb54109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/2807597483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54100ea0 0x7fbb541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/2807597483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54102080 0x7fbb54102500 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/2807597483 >> 172.21.15.26:0/2807597483 conn(0x7fbb540fc460 msgr2=0x7fbb540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/2807597483 shutdown_connections 2023-10-06T21:08:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.624+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/2807597483 wait complete. 2023-10-06T21:08:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.625+0000 7fbb5c1ff700 1 Processor -- start 2023-10-06T21:08:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 -- start start 2023-10-06T21:08:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54102080 0x7fbb54199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb541a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a09b0 con 0x7fbb54100ea0 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a0af0 con 0x7fbb54102a40 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5c1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a0c80 con 0x7fbb54102080 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb59f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb59f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43720/0 (socket says 172.21.15.26:43720) 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb59f9b700 1 -- 172.21.15.26:0/1532783714 learned_addr learned my addr 172.21.15.26:0/1532783714 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.626+0000 7fbb5979a700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54102080 0x7fbb54199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb5a79c700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb541a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 -- 172.21.15.26:0/1532783714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54102080 msgr2=0x7fbb54199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54102080 0x7fbb54199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 -- 172.21.15.26:0/1532783714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 msgr2=0x7fbb541a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb541a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 -- 172.21.15.26:0/1532783714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440095f0 con 0x7fbb54100ea0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb5a79c700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb541a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb59f9b700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e970 tx=0x7fbb5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50009790 con 0x7fbb54100ea0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb500044e0 con 0x7fbb54100ea0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541a0f60 con 0x7fbb54100ea0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50003e30 con 0x7fbb54100ea0 2023-10-06T21:08:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.627+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a1380 con 0x7fbb54100ea0 2023-10-06T21:08:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.628+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5410c760 con 0x7fbb54100ea0 2023-10-06T21:08:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.630+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbb50004000 con 0x7fbb54100ea0 2023-10-06T21:08:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.632+0000 7fbb4affd700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 0x7fbb400a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.632+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb50014070 con 0x7fbb54100ea0 2023-10-06T21:08:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.632+0000 7fbb5979a700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 0x7fbb400a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.633+0000 7fbb5979a700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 0x7fbb400a39d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b2e0 tx=0x7fbb44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.636+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb500966a0 con 0x7fbb54100ea0 2023-10-06T21:08:23.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.888+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb54108b00 con 0x7fbb400a1510 2023-10-06T21:08:23.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.892+0000 7fbb4affd700 1 -- 172.21.15.26:0/1532783714 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fbb54108b00 con 0x7fbb400a1510 2023-10-06T21:08:23.895 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv_1/68c2ada3-e9f1-4f5f-952a-1c21257dff12 2023-10-06T21:08:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 msgr2=0x7fbb400a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 0x7fbb400a39d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b2e0 tx=0x7fbb44005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 msgr2=0x7fbb54198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:23.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e970 tx=0x7fbb5000ed30 comp rx=0 tx=0).stop 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 shutdown_connections 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54102a40 0x7fbb541a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbb400a1510 0x7fbb400a39d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54100ea0 0x7fbb54198e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 --2- 172.21.15.26:0/1532783714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb54102080 0x7fbb54199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 >> 172.21.15.26:0/1532783714 conn(0x7fbb540fc460 msgr2=0x7fbb540fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 shutdown_connections 2023-10-06T21:08:23.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:23.895+0000 7fbb5c1ff700 1 -- 172.21.15.26:0/1532783714 wait complete. 2023-10-06T21:08:23.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:08:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:23 smithi078 ceph-mon[134896]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T21:08:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:23 smithi078 ceph-mon[134896]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:23 smithi078 ceph-mon[134896]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 -- 172.21.15.26:0/1587622895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84102080 msgr2=0x7fee84102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/1587622895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84102080 0x7fee84102500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee74009990 tx=0x7fee74009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 -- 172.21.15.26:0/1587622895 shutdown_connections 2023-10-06T21:08:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/1587622895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102a40 0x7fee84109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/1587622895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84100ea0 0x7fee841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/1587622895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84102080 0x7fee84102500 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 -- 172.21.15.26:0/1587622895 >> 172.21.15.26:0/1587622895 conn(0x7fee840fc420 msgr2=0x7fee840fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:24.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 -- 172.21.15.26:0/1587622895 shutdown_connections 2023-10-06T21:08:24.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.118+0000 7fee8c7ff700 1 -- 172.21.15.26:0/1587622895 wait complete. 2023-10-06T21:08:24.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.119+0000 7fee8c7ff700 1 Processor -- start 2023-10-06T21:08:24.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.119+0000 7fee8c7ff700 1 -- start start 2023-10-06T21:08:24.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102080 0x7fee84199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841a0900 con 0x7fee84100ea0 2023-10-06T21:08:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841a0a70 con 0x7fee84102080 2023-10-06T21:08:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8c7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee841a0c30 con 0x7fee84102a40 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8a59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33120/0 (socket says 172.21.15.26:33120) 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8a59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43736/0 (socket says 172.21.15.26:43736) 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 -- 172.21.15.26:0/3890057437 learned_addr learned my addr 172.21.15.26:0/3890057437 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee89d9a700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102080 0x7fee84199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 -- 172.21.15.26:0/3890057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102080 msgr2=0x7fee84199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102080 0x7fee84199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 -- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 msgr2=0x7fee84198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8ad9c700 1 -- 172.21.15.26:0/3890057437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740095f0 con 0x7fee84102a40 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.120+0000 7fee8a59b700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee8ad9c700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee7c007ee0 tx=0x7fee7c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c017070 con 0x7fee84102a40 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee7c00f040 con 0x7fee84102a40 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee841a0f10 con 0x7fee84102a40 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c005580 con 0x7fee84102a40 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.121+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee841a12e0 con 0x7fee84102a40 2023-10-06T21:08:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.122+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee84193500 con 0x7fee84102a40 2023-10-06T21:08:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.124+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fee7c00db20 con 0x7fee84102a40 2023-10-06T21:08:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.125+0000 7fee7b7fe700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 0x7fee700a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.126+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee7c0cc4e0 con 0x7fee84102a40 2023-10-06T21:08:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.126+0000 7fee8a59b700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 0x7fee700a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.126+0000 7fee8a59b700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 0x7fee700a3a10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee8419a290 tx=0x7fee80009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.127+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fee7c095a90 con 0x7fee84102a40 2023-10-06T21:08:24.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:23 smithi167 ceph-mon[134494]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T21:08:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:23 smithi167 ceph-mon[134494]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:23 smithi167 ceph-mon[134494]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:23 smithi026 ceph-mon[98490]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-06T21:08:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:23 smithi026 ceph-mon[98490]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:23 smithi026 ceph-mon[98490]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:24.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.385+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee8402d2b0 con 0x7fee84102a40 2023-10-06T21:08:24.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.387+0000 7fee7b7fe700 1 -- 172.21.15.26:0/3890057437 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7fee7c0990b0 con 0x7fee84102a40 2023-10-06T21:08:24.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:24.390 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 msgr2=0x7fee700a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 0x7fee700a3a10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee8419a290 tx=0x7fee80009260 comp rx=0 tx=0).stop 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 msgr2=0x7fee841a0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee7c007ee0 tx=0x7fee7c00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 shutdown_connections 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee84102a40 0x7fee841a0330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee700a1550 0x7fee700a3a10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee84100ea0 0x7fee84198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 --2- 172.21.15.26:0/3890057437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee84102080 0x7fee84199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 >> 172.21.15.26:0/3890057437 conn(0x7fee840fc420 msgr2=0x7fee84107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 shutdown_connections 2023-10-06T21:08:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.389+0000 7fee8c7ff700 1 -- 172.21.15.26:0/3890057437 wait complete. 2023-10-06T21:08:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:08:24.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:08:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 -- 172.21.15.26:0/3205970197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000100ea0 msgr2=0x7f20001012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 --2- 172.21.15.26:0/3205970197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000100ea0 0x7f20001012c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009990 tx=0x7f1ffc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 -- 172.21.15.26:0/3205970197 shutdown_connections 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 --2- 172.21.15.26:0/3205970197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000102a40 0x7f2000109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 --2- 172.21.15.26:0/3205970197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000100ea0 0x7f20001012c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 --2- 172.21.15.26:0/3205970197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102080 0x7f2000102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 -- 172.21.15.26:0/3205970197 >> 172.21.15.26:0/3205970197 conn(0x7f20000fc420 msgr2=0x7f20000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.618+0000 7f20087ff700 1 -- 172.21.15.26:0/3205970197 shutdown_connections 2023-10-06T21:08:24.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.619+0000 7f20087ff700 1 -- 172.21.15.26:0/3205970197 wait complete. 2023-10-06T21:08:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.619+0000 7f20087ff700 1 Processor -- start 2023-10-06T21:08:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 -- start start 2023-10-06T21:08:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 0x7f2000196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 0x7f200019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019e770 con 0x7f2000102080 2023-10-06T21:08:24.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019e8e0 con 0x7f2000100ea0 2023-10-06T21:08:24.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f20087ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019eaa0 con 0x7f2000102a40 2023-10-06T21:08:24.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f2005d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43754/0 (socket says 172.21.15.26:43754) 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f2005d9a700 1 -- 172.21.15.26:0/2878058890 learned_addr learned my addr 172.21.15.26:0/2878058890 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.620+0000 7f200659b700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 0x7f2000196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2006d9c700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 0x7f200019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 -- 172.21.15.26:0/2878058890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 msgr2=0x7f200019e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 0x7f200019e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 -- 172.21.15.26:0/2878058890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 msgr2=0x7f2000196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 0x7f2000196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 -- 172.21.15.26:0/2878058890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0095f0 con 0x7f2000102080 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2006d9c700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 0x7f200019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f200659b700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 0x7f2000196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f2005d9a700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e970 tx=0x7f1ff000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0009790 con 0x7f2000102080 2023-10-06T21:08:24.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff000eec0 con 0x7f2000102080 2023-10-06T21:08:24.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200019ed80 con 0x7f2000102080 2023-10-06T21:08:24.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003cf0 con 0x7f2000102080 2023-10-06T21:08:24.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.621+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200019f220 con 0x7f2000102080 2023-10-06T21:08:24.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.622+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200010c760 con 0x7f2000102080 2023-10-06T21:08:24.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.625+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1ff000ccd0 con 0x7f2000102080 2023-10-06T21:08:24.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.626+0000 7f1ff77fe700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 0x7f1fec0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:24.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.626+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ff0014070 con 0x7f2000102080 2023-10-06T21:08:24.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.626+0000 7f200659b700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 0x7f1fec0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:24.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.627+0000 7f200659b700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 0x7f1fec0a3a90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:24.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.628+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ff00966f0 con 0x7f2000102080 2023-10-06T21:08:24.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.839+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f200010c970 con 0x7f2000102080 2023-10-06T21:08:24.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.840+0000 7f1ff77fe700 1 -- 172.21.15.26:0/2878058890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22069 (secure 0 0 0) 0x7f1ff0099d10 con 0x7f2000102080 2023-10-06T21:08:24.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:24.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:17.510362+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 msgr2=0x7f1fec0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 0x7f1fec0a3a90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 msgr2=0x7f2000197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e970 tx=0x7f1ff000ec80 comp rx=0 tx=0).stop 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 shutdown_connections 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2000102a40 0x7f200019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2000100ea0 0x7f2000196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1fec0a15d0 0x7f1fec0a3a90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 --2- 172.21.15.26:0/2878058890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2000102080 0x7f2000197170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 >> 172.21.15.26:0/2878058890 conn(0x7f20000fc420 msgr2=0x7f20000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 shutdown_connections 2023-10-06T21:08:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:24.843+0000 7f20087ff700 1 -- 172.21.15.26:0/2878058890 wait complete. 2023-10-06T21:08:24.864 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T21:08:24.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:08:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.065+0000 7f4288fff700 1 -- 172.21.15.26:0/2177704149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102a40 msgr2=0x7f4284109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 --2- 172.21.15.26:0/2177704149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102a40 0x7f4284109360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b0a0 tx=0x7f427c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 -- 172.21.15.26:0/2177704149 shutdown_connections 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 --2- 172.21.15.26:0/2177704149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102a40 0x7f4284109360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 --2- 172.21.15.26:0/2177704149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284100ea0 0x7f42841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 --2- 172.21.15.26:0/2177704149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284102080 0x7f4284102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 -- 172.21.15.26:0/2177704149 >> 172.21.15.26:0/2177704149 conn(0x7f42840fc420 msgr2=0x7f42840fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 -- 172.21.15.26:0/2177704149 shutdown_connections 2023-10-06T21:08:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.066+0000 7f4288fff700 1 -- 172.21.15.26:0/2177704149 wait complete. 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 Processor -- start 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 -- start start 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284100ea0 0x7f4284198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284102a40 0x7f42841a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841a0a40 con 0x7f4284102080 2023-10-06T21:08:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841a0b80 con 0x7f4284100ea0 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f4288fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841a0d10 con 0x7f4284102a40 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.067+0000 7f428259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43766/0 (socket says 172.21.15.26:43766) 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 -- 172.21.15.26:0/3997258192 learned_addr learned my addr 172.21.15.26:0/3997258192 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f4282d9d700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284102a40 0x7f42841a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f427a59c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284100ea0 0x7f4284198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 -- 172.21.15.26:0/3997258192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284102a40 msgr2=0x7f42841a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284102a40 0x7f42841a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 -- 172.21.15.26:0/3997258192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284100ea0 msgr2=0x7f4284198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284100ea0 0x7f4284198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 -- 172.21.15.26:0/3997258192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c009d00 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f428259c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e970 tx=0x7f426c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c009790 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c00c7e0 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42841a0ff0 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.068+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003c70 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.069+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42841a1470 con 0x7f4284102080 2023-10-06T21:08:25.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.069+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428410c760 con 0x7f4284102080 2023-10-06T21:08:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.072+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f426c00cc50 con 0x7f4284102080 2023-10-06T21:08:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.073+0000 7f427bfff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 0x7f42700a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.073+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f426c014070 con 0x7f4284102080 2023-10-06T21:08:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.073+0000 7f427a59c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 0x7f42700a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.074+0000 7f427a59c700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 0x7f42700a3a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4284101bf0 tx=0x7f427400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:25.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.077+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f426c096510 con 0x7f4284102080 2023-10-06T21:08:25.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:24 smithi078 ceph-mon[134896]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:25.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:24 smithi078 ceph-mon[134896]: pgmap v275: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-10-06T21:08:25.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:24 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3890057437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:24 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2878058890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:24 smithi167 ceph-mon[134494]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:25.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:24 smithi167 ceph-mon[134494]: pgmap v275: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-10-06T21:08:25.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:24 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3890057437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:24 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2878058890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.280+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4284109060 con 0x7f42700a1550 2023-10-06T21:08:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.293+0000 7f427bfff700 1 -- 172.21.15.26:0/3997258192 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335937 (secure 0 0 0) 0x7f4284109060 con 0x7f42700a1550 2023-10-06T21:08:25.301 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 msgr2=0x7f42700a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 0x7f42700a3a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4284101bf0 tx=0x7f427400b410 comp rx=0 tx=0).stop 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 msgr2=0x7f4284199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e970 tx=0x7f426c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 shutdown_connections 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284102a40 0x7f42841a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4284100ea0 0x7f4284198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f42700a1550 0x7f42700a3a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 --2- 172.21.15.26:0/3997258192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4284102080 0x7f4284199400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 >> 172.21.15.26:0/3997258192 conn(0x7f42840fc420 msgr2=0x7f4284107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 shutdown_connections 2023-10-06T21:08:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.298+0000 7f4288fff700 1 -- 172.21.15.26:0/3997258192 wait complete. 2023-10-06T21:08:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:08:25.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:24 smithi026 ceph-mon[98490]: from='client.14637 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:08:25.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:24 smithi026 ceph-mon[98490]: pgmap v275: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-10-06T21:08:25.318 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:24 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3890057437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.318 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:24 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2878058890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:25.320 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2023-10-06T21:08:24.367573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613266,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":289,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336872,"kb_used_data":12652,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604952,"statfs":{"total":1151940427776,"available":1151595470848,"internally_reserved":0,"allocated":12955648,"data_stored":7065344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22898,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":108,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001990"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319267+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:08:18.319267+0000","last_peered":"2023-10-06T21:08:18.319267+0000","last_clean":"2023-10-06T21:08:18.319267+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:08:18.319267+0000","last_undegraded":"2023-10-06T21:08:18.319267+0000","last_fullsized":"2023-10-06T21:08:18.319267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922389+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:08:17.922389+0000","last_peered":"2023-10-06T21:08:17.922389+0000","last_clean":"2023-10-06T21:08:17.922389+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:08:17.922389+0000","last_undegraded":"2023-10-06T21:08:17.922389+0000","last_fullsized":"2023-10-06T21:08:17.922389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922246+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:08:17.922246+0000","last_peered":"2023-10-06T21:08:17.922246+0000","last_clean":"2023-10-06T21:08:17.922246+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:08:17.922246+0000","last_undegraded":"2023-10-06T21:08:17.922246+0000","last_fullsized":"2023-10-06T21:08:17.922246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.905583+0000","last_change":"2023-10-06T21:08:16.515864+0000","last_active":"2023-10-06T21:08:17.905583+0000","last_peered":"2023-10-06T21:08:17.905583+0000","last_clean":"2023-10-06T21:08:17.905583+0000","last_became_active":"2023-10-06T21:08:16.515523+0000","last_became_peered":"2023-10-06T21:08:16.515523+0000","last_unstale":"2023-10-06T21:08:17.905583+0000","last_undegraded":"2023-10-06T21:08:17.905583+0000","last_fullsized":"2023-10-06T21:08:17.905583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:41:18.651694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621143+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:08:17.621143+0000","last_peered":"2023-10-06T21:08:17.621143+0000","last_clean":"2023-10-06T21:08:17.621143+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:08:17.621143+0000","last_undegraded":"2023-10-06T21:08:17.621143+0000","last_fullsized":"2023-10-06T21:08:17.621143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922000+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:08:17.922000+0000","last_peered":"2023-10-06T21:08:17.922000+0000","last_clean":"2023-10-06T21:08:17.922000+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:08:17.922000+0000","last_undegraded":"2023-10-06T21:08:17.922000+0000","last_fullsized":"2023-10-06T21:08:17.922000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921288+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:08:17.921288+0000","last_peered":"2023-10-06T21:08:17.921288+0000","last_clean":"2023-10-06T21:08:17.921288+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:08:17.921288+0000","last_undegraded":"2023-10-06T21:08:17.921288+0000","last_fullsized":"2023-10-06T21:08:17.921288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921661+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:08:17.921661+0000","last_peered":"2023-10-06T21:08:17.921661+0000","last_clean":"2023-10-06T21:08:17.921661+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:08:17.921661+0000","last_undegraded":"2023-10-06T21:08:17.921661+0000","last_fullsized":"2023-10-06T21:08:17.921661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514106+0000","last_change":"2023-10-06T21:08:16.516691+0000","last_active":"2023-10-06T21:08:17.514106+0000","last_peered":"2023-10-06T21:08:17.514106+0000","last_clean":"2023-10-06T21:08:17.514106+0000","last_became_active":"2023-10-06T21:08:16.516223+0000","last_became_peered":"2023-10-06T21:08:16.516223+0000","last_unstale":"2023-10-06T21:08:17.514106+0000","last_undegraded":"2023-10-06T21:08:17.514106+0000","last_fullsized":"2023-10-06T21:08:17.514106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:02:33.410158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922236+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:08:17.922236+0000","last_peered":"2023-10-06T21:08:17.922236+0000","last_clean":"2023-10-06T21:08:17.922236+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:08:17.922236+0000","last_undegraded":"2023-10-06T21:08:17.922236+0000","last_fullsized":"2023-10-06T21:08:17.922236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921715+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:08:17.921715+0000","last_peered":"2023-10-06T21:08:17.921715+0000","last_clean":"2023-10-06T21:08:17.921715+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:08:17.921715+0000","last_undegraded":"2023-10-06T21:08:17.921715+0000","last_fullsized":"2023-10-06T21:08:17.921715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.905592+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:08:17.905592+0000","last_peered":"2023-10-06T21:08:17.905592+0000","last_clean":"2023-10-06T21:08:17.905592+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:08:17.905592+0000","last_undegraded":"2023-10-06T21:08:17.905592+0000","last_fullsized":"2023-10-06T21:08:17.905592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923588+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:08:17.923588+0000","last_peered":"2023-10-06T21:08:17.923588+0000","last_clean":"2023-10-06T21:08:17.923588+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:08:17.923588+0000","last_undegraded":"2023-10-06T21:08:17.923588+0000","last_fullsized":"2023-10-06T21:08:17.923588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921886+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:08:17.921886+0000","last_peered":"2023-10-06T21:08:17.921886+0000","last_clean":"2023-10-06T21:08:17.921886+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:08:17.921886+0000","last_undegraded":"2023-10-06T21:08:17.921886+0000","last_fullsized":"2023-10-06T21:08:17.921886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921641+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:08:17.921641+0000","last_peered":"2023-10-06T21:08:17.921641+0000","last_clean":"2023-10-06T21:08:17.921641+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:08:17.921641+0000","last_undegraded":"2023-10-06T21:08:17.921641+0000","last_fullsized":"2023-10-06T21:08:17.921641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513766+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:08:17.513766+0000","last_peered":"2023-10-06T21:08:17.513766+0000","last_clean":"2023-10-06T21:08:17.513766+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:08:17.513766+0000","last_undegraded":"2023-10-06T21:08:17.513766+0000","last_fullsized":"2023-10-06T21:08:17.513766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319305+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:08:18.319305+0000","last_peered":"2023-10-06T21:08:18.319305+0000","last_clean":"2023-10-06T21:08:18.319305+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:08:18.319305+0000","last_undegraded":"2023-10-06T21:08:18.319305+0000","last_fullsized":"2023-10-06T21:08:18.319305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514445+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:08:17.514445+0000","last_peered":"2023-10-06T21:08:17.514445+0000","last_clean":"2023-10-06T21:08:17.514445+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:08:17.514445+0000","last_undegraded":"2023-10-06T21:08:17.514445+0000","last_fullsized":"2023-10-06T21:08:17.514445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514952+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:08:17.514952+0000","last_peered":"2023-10-06T21:08:17.514952+0000","last_clean":"2023-10-06T21:08:17.514952+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:08:17.514952+0000","last_undegraded":"2023-10-06T21:08:17.514952+0000","last_fullsized":"2023-10-06T21:08:17.514952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.543742+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:16.543742+0000","last_peered":"2023-10-06T21:08:16.543742+0000","last_clean":"2023-10-06T21:08:16.543742+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:16.543742+0000","last_undegraded":"2023-10-06T21:08:16.543742+0000","last_fullsized":"2023-10-06T21:08:16.543742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922048+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:08:17.922048+0000","last_peered":"2023-10-06T21:08:17.922048+0000","last_clean":"2023-10-06T21:08:17.922048+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:08:17.922048+0000","last_undegraded":"2023-10-06T21:08:17.922048+0000","last_fullsized":"2023-10-06T21:08:17.922048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.541370+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:16.541370+0000","last_peered":"2023-10-06T21:08:16.541370+0000","last_clean":"2023-10-06T21:08:16.541370+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:16.541370+0000","last_undegraded":"2023-10-06T21:08:16.541370+0000","last_fullsized":"2023-10-06T21:08:16.541370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922312+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:08:17.922312+0000","last_peered":"2023-10-06T21:08:17.922312+0000","last_clean":"2023-10-06T21:08:17.922312+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:08:17.922312+0000","last_undegraded":"2023-10-06T21:08:17.922312+0000","last_fullsized":"2023-10-06T21:08:17.922312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.539828+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:16.539828+0000","last_peered":"2023-10-06T21:08:16.539828+0000","last_clean":"2023-10-06T21:08:16.539828+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:16.539828+0000","last_undegraded":"2023-10-06T21:08:16.539828+0000","last_fullsized":"2023-10-06T21:08:16.539828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513912+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:08:17.513912+0000","last_peered":"2023-10-06T21:08:17.513912+0000","last_clean":"2023-10-06T21:08:17.513912+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:08:17.513912+0000","last_undegraded":"2023-10-06T21:08:17.513912+0000","last_fullsized":"2023-10-06T21:08:17.513912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.541045+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:16.541045+0000","last_peered":"2023-10-06T21:08:16.541045+0000","last_clean":"2023-10-06T21:08:16.541045+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:16.541045+0000","last_undegraded":"2023-10-06T21:08:16.541045+0000","last_fullsized":"2023-10-06T21:08:16.541045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921279+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:08:17.921279+0000","last_peered":"2023-10-06T21:08:17.921279+0000","last_clean":"2023-10-06T21:08:17.921279+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:08:17.921279+0000","last_undegraded":"2023-10-06T21:08:17.921279+0000","last_fullsized":"2023-10-06T21:08:17.921279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.859123+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:08:21.859123+0000","last_peered":"2023-10-06T21:08:21.859123+0000","last_clean":"2023-10-06T21:08:21.859123+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:08:21.859123+0000","last_undegraded":"2023-10-06T21:08:21.859123+0000","last_fullsized":"2023-10-06T21:08:21.859123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621178+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:08:17.621178+0000","last_peered":"2023-10-06T21:08:17.621178+0000","last_clean":"2023-10-06T21:08:17.621178+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:08:17.621178+0000","last_undegraded":"2023-10-06T21:08:17.621178+0000","last_fullsized":"2023-10-06T21:08:17.621178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.855107+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:20.855107+0000","last_peered":"2023-10-06T21:08:20.855107+0000","last_clean":"2023-10-06T21:08:20.855107+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:20.855107+0000","last_undegraded":"2023-10-06T21:08:20.855107+0000","last_fullsized":"2023-10-06T21:08:20.855107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921275+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:08:17.921275+0000","last_peered":"2023-10-06T21:08:17.921275+0000","last_clean":"2023-10-06T21:08:17.921275+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:08:17.921275+0000","last_undegraded":"2023-10-06T21:08:17.921275+0000","last_fullsized":"2023-10-06T21:08:17.921275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922899+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:17.922899+0000","last_peered":"2023-10-06T21:08:17.922899+0000","last_clean":"2023-10-06T21:08:17.922899+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:17.922899+0000","last_undegraded":"2023-10-06T21:08:17.922899+0000","last_fullsized":"2023-10-06T21:08:17.922899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921512+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:08:17.921512+0000","last_peered":"2023-10-06T21:08:17.921512+0000","last_clean":"2023-10-06T21:08:17.921512+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:08:17.921512+0000","last_undegraded":"2023-10-06T21:08:17.921512+0000","last_fullsized":"2023-10-06T21:08:17.921512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.920971+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:17.920971+0000","last_peered":"2023-10-06T21:08:17.920971+0000","last_clean":"2023-10-06T21:08:17.920971+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:17.920971+0000","last_undegraded":"2023-10-06T21:08:17.920971+0000","last_fullsized":"2023-10-06T21:08:17.920971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514435+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:08:17.514435+0000","last_peered":"2023-10-06T21:08:17.514435+0000","last_clean":"2023-10-06T21:08:17.514435+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:08:17.514435+0000","last_undegraded":"2023-10-06T21:08:17.514435+0000","last_fullsized":"2023-10-06T21:08:17.514435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.847343+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:08:19.847343+0000","last_peered":"2023-10-06T21:08:19.847343+0000","last_clean":"2023-10-06T21:08:19.847343+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:08:19.847343+0000","last_undegraded":"2023-10-06T21:08:19.847343+0000","last_fullsized":"2023-10-06T21:08:19.847343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514172+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:08:17.514172+0000","last_peered":"2023-10-06T21:08:17.514172+0000","last_clean":"2023-10-06T21:08:17.514172+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:08:17.514172+0000","last_undegraded":"2023-10-06T21:08:17.514172+0000","last_fullsized":"2023-10-06T21:08:17.514172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846047+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:19.846047+0000","last_peered":"2023-10-06T21:08:19.846047+0000","last_clean":"2023-10-06T21:08:19.846047+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:19.846047+0000","last_undegraded":"2023-10-06T21:08:19.846047+0000","last_fullsized":"2023-10-06T21:08:19.846047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922161+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:08:17.922161+0000","last_peered":"2023-10-06T21:08:17.922161+0000","last_clean":"2023-10-06T21:08:17.922161+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:08:17.922161+0000","last_undegraded":"2023-10-06T21:08:17.922161+0000","last_fullsized":"2023-10-06T21:08:17.922161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846712+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:19.846712+0000","last_peered":"2023-10-06T21:08:19.846712+0000","last_clean":"2023-10-06T21:08:19.846712+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:19.846712+0000","last_undegraded":"2023-10-06T21:08:19.846712+0000","last_fullsized":"2023-10-06T21:08:19.846712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514034+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:08:17.514034+0000","last_peered":"2023-10-06T21:08:17.514034+0000","last_clean":"2023-10-06T21:08:17.514034+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:08:17.514034+0000","last_undegraded":"2023-10-06T21:08:17.514034+0000","last_fullsized":"2023-10-06T21:08:17.514034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'14","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:22.375164+0000","last_change":"2023-10-06T21:08:15.524431+0000","last_active":"2023-10-06T21:08:22.375164+0000","last_peered":"2023-10-06T21:08:22.375164+0000","last_clean":"2023-10-06T21:08:22.375164+0000","last_became_active":"2023-10-06T21:08:15.524064+0000","last_became_peered":"2023-10-06T21:08:15.524064+0000","last_unstale":"2023-10-06T21:08:22.375164+0000","last_undegraded":"2023-10-06T21:08:22.375164+0000","last_fullsized":"2023-10-06T21:08:22.375164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:09:11.585643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621270+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:08:17.621270+0000","last_peered":"2023-10-06T21:08:17.621270+0000","last_clean":"2023-10-06T21:08:17.621270+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:08:17.621270+0000","last_undegraded":"2023-10-06T21:08:17.621270+0000","last_fullsized":"2023-10-06T21:08:17.621270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.856131+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:20.856131+0000","last_peered":"2023-10-06T21:08:20.856131+0000","last_clean":"2023-10-06T21:08:20.856131+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:20.856131+0000","last_undegraded":"2023-10-06T21:08:20.856131+0000","last_fullsized":"2023-10-06T21:08:20.856131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923280+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:08:17.923280+0000","last_peered":"2023-10-06T21:08:17.923280+0000","last_clean":"2023-10-06T21:08:17.923280+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:08:17.923280+0000","last_undegraded":"2023-10-06T21:08:17.923280+0000","last_fullsized":"2023-10-06T21:08:17.923280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846427+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:19.846427+0000","last_peered":"2023-10-06T21:08:19.846427+0000","last_clean":"2023-10-06T21:08:19.846427+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:19.846427+0000","last_undegraded":"2023-10-06T21:08:19.846427+0000","last_fullsized":"2023-10-06T21:08:19.846427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921642+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:08:17.921642+0000","last_peered":"2023-10-06T21:08:17.921642+0000","last_clean":"2023-10-06T21:08:17.921642+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:08:17.921642+0000","last_undegraded":"2023-10-06T21:08:17.921642+0000","last_fullsized":"2023-10-06T21:08:17.921642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.860409+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:08:21.860409+0000","last_peered":"2023-10-06T21:08:21.860409+0000","last_clean":"2023-10-06T21:08:21.860409+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:08:21.860409+0000","last_undegraded":"2023-10-06T21:08:21.860409+0000","last_fullsized":"2023-10-06T21:08:21.860409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921207+0000","last_change":"2023-10-06T21:08:16.515157+0000","last_active":"2023-10-06T21:08:17.921207+0000","last_peered":"2023-10-06T21:08:17.921207+0000","last_clean":"2023-10-06T21:08:17.921207+0000","last_became_active":"2023-10-06T21:08:16.514847+0000","last_became_peered":"2023-10-06T21:08:16.514847+0000","last_unstale":"2023-10-06T21:08:17.921207+0000","last_undegraded":"2023-10-06T21:08:17.921207+0000","last_fullsized":"2023-10-06T21:08:17.921207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:34.986136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.845788+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:19.845788+0000","last_peered":"2023-10-06T21:08:19.845788+0000","last_clean":"2023-10-06T21:08:19.845788+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:19.845788+0000","last_undegraded":"2023-10-06T21:08:19.845788+0000","last_fullsized":"2023-10-06T21:08:19.845788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921104+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:08:17.921104+0000","last_peered":"2023-10-06T21:08:17.921104+0000","last_clean":"2023-10-06T21:08:17.921104+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:08:17.921104+0000","last_undegraded":"2023-10-06T21:08:17.921104+0000","last_fullsized":"2023-10-06T21:08:17.921104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.539931+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:16.539931+0000","last_peered":"2023-10-06T21:08:16.539931+0000","last_clean":"2023-10-06T21:08:16.539931+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:16.539931+0000","last_undegraded":"2023-10-06T21:08:16.539931+0000","last_fullsized":"2023-10-06T21:08:16.539931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319169+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:08:18.319169+0000","last_peered":"2023-10-06T21:08:18.319169+0000","last_clean":"2023-10-06T21:08:18.319169+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:08:18.319169+0000","last_undegraded":"2023-10-06T21:08:18.319169+0000","last_fullsized":"2023-10-06T21:08:18.319169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.501102+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:16.501102+0000","last_peered":"2023-10-06T21:08:16.501102+0000","last_clean":"2023-10-06T21:08:16.501102+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:16.501102+0000","last_undegraded":"2023-10-06T21:08:16.501102+0000","last_fullsized":"2023-10-06T21:08:16.501102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922282+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:08:17.922282+0000","last_peered":"2023-10-06T21:08:17.922282+0000","last_clean":"2023-10-06T21:08:17.922282+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:08:17.922282+0000","last_undegraded":"2023-10-06T21:08:17.922282+0000","last_fullsized":"2023-10-06T21:08:17.922282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921505+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:17.921505+0000","last_peered":"2023-10-06T21:08:17.921505+0000","last_clean":"2023-10-06T21:08:17.921505+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:17.921505+0000","last_undegraded":"2023-10-06T21:08:17.921505+0000","last_fullsized":"2023-10-06T21:08:17.921505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922161+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:08:17.922161+0000","last_peered":"2023-10-06T21:08:17.922161+0000","last_clean":"2023-10-06T21:08:17.922161+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:08:17.922161+0000","last_undegraded":"2023-10-06T21:08:17.922161+0000","last_fullsized":"2023-10-06T21:08:17.922161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921143+0000","last_change":"2023-10-06T21:08:16.512032+0000","last_active":"2023-10-06T21:08:17.921143+0000","last_peered":"2023-10-06T21:08:17.921143+0000","last_clean":"2023-10-06T21:08:17.921143+0000","last_became_active":"2023-10-06T21:08:16.511717+0000","last_became_peered":"2023-10-06T21:08:16.511717+0000","last_unstale":"2023-10-06T21:08:17.921143+0000","last_undegraded":"2023-10-06T21:08:17.921143+0000","last_fullsized":"2023-10-06T21:08:17.921143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:34.118930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922814+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:08:17.922814+0000","last_peered":"2023-10-06T21:08:17.922814+0000","last_clean":"2023-10-06T21:08:17.922814+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:08:17.922814+0000","last_undegraded":"2023-10-06T21:08:17.922814+0000","last_fullsized":"2023-10-06T21:08:17.922814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922753+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:08:17.922753+0000","last_peered":"2023-10-06T21:08:17.922753+0000","last_clean":"2023-10-06T21:08:17.922753+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:08:17.922753+0000","last_undegraded":"2023-10-06T21:08:17.922753+0000","last_fullsized":"2023-10-06T21:08:17.922753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.855802+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:21.855802+0000","last_peered":"2023-10-06T21:08:21.855802+0000","last_clean":"2023-10-06T21:08:21.855802+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:21.855802+0000","last_undegraded":"2023-10-06T21:08:21.855802+0000","last_fullsized":"2023-10-06T21:08:21.855802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.507232+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:16.507232+0000","last_peered":"2023-10-06T21:08:16.507232+0000","last_clean":"2023-10-06T21:08:16.507232+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:16.507232+0000","last_undegraded":"2023-10-06T21:08:16.507232+0000","last_fullsized":"2023-10-06T21:08:16.507232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922723+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:08:17.922723+0000","last_peered":"2023-10-06T21:08:17.922723+0000","last_clean":"2023-10-06T21:08:17.922723+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:08:17.922723+0000","last_undegraded":"2023-10-06T21:08:17.922723+0000","last_fullsized":"2023-10-06T21:08:17.922723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.855634+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:20.855634+0000","last_peered":"2023-10-06T21:08:20.855634+0000","last_clean":"2023-10-06T21:08:20.855634+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:20.855634+0000","last_undegraded":"2023-10-06T21:08:20.855634+0000","last_fullsized":"2023-10-06T21:08:20.855634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621223+0000","last_change":"2023-10-06T21:08:15.524380+0000","last_active":"2023-10-06T21:08:17.621223+0000","last_peered":"2023-10-06T21:08:17.621223+0000","last_clean":"2023-10-06T21:08:17.621223+0000","last_became_active":"2023-10-06T21:08:15.524059+0000","last_became_peered":"2023-10-06T21:08:15.524059+0000","last_unstale":"2023-10-06T21:08:17.621223+0000","last_undegraded":"2023-10-06T21:08:17.621223+0000","last_fullsized":"2023-10-06T21:08:17.621223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:23:54.360603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923175+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:08:17.923175+0000","last_peered":"2023-10-06T21:08:17.923175+0000","last_clean":"2023-10-06T21:08:17.923175+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:08:17.923175+0000","last_undegraded":"2023-10-06T21:08:17.923175+0000","last_fullsized":"2023-10-06T21:08:17.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.501641+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:16.501641+0000","last_peered":"2023-10-06T21:08:16.501641+0000","last_clean":"2023-10-06T21:08:16.501641+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:16.501641+0000","last_undegraded":"2023-10-06T21:08:16.501641+0000","last_fullsized":"2023-10-06T21:08:16.501641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922497+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:08:17.922497+0000","last_peered":"2023-10-06T21:08:17.922497+0000","last_clean":"2023-10-06T21:08:17.922497+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:08:17.922497+0000","last_undegraded":"2023-10-06T21:08:17.922497+0000","last_fullsized":"2023-10-06T21:08:17.922497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.848020+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:19.848020+0000","last_peered":"2023-10-06T21:08:19.848020+0000","last_clean":"2023-10-06T21:08:19.848020+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:19.848020+0000","last_undegraded":"2023-10-06T21:08:19.848020+0000","last_fullsized":"2023-10-06T21:08:19.848020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922542+0000","last_change":"2023-10-06T21:08:16.525576+0000","last_active":"2023-10-06T21:08:17.922542+0000","last_peered":"2023-10-06T21:08:17.922542+0000","last_clean":"2023-10-06T21:08:17.922542+0000","last_became_active":"2023-10-06T21:08:16.525250+0000","last_became_peered":"2023-10-06T21:08:16.525250+0000","last_unstale":"2023-10-06T21:08:17.922542+0000","last_undegraded":"2023-10-06T21:08:17.922542+0000","last_fullsized":"2023-10-06T21:08:17.922542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:37:03.571384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846599+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:19.846599+0000","last_peered":"2023-10-06T21:08:19.846599+0000","last_clean":"2023-10-06T21:08:19.846599+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:19.846599+0000","last_undegraded":"2023-10-06T21:08:19.846599+0000","last_fullsized":"2023-10-06T21:08:19.846599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621068+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:08:17.621068+0000","last_peered":"2023-10-06T21:08:17.621068+0000","last_clean":"2023-10-06T21:08:17.621068+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:08:17.621068+0000","last_undegraded":"2023-10-06T21:08:17.621068+0000","last_fullsized":"2023-10-06T21:08:17.621068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.847239+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:19.847239+0000","last_peered":"2023-10-06T21:08:19.847239+0000","last_clean":"2023-10-06T21:08:19.847239+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:19.847239+0000","last_undegraded":"2023-10-06T21:08:19.847239+0000","last_fullsized":"2023-10-06T21:08:19.847239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.318841+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:08:18.318841+0000","last_peered":"2023-10-06T21:08:18.318841+0000","last_clean":"2023-10-06T21:08:18.318841+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:08:18.318841+0000","last_undegraded":"2023-10-06T21:08:18.318841+0000","last_fullsized":"2023-10-06T21:08:18.318841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.355958+0000","last_change":"2023-10-06T21:08:15.521774+0000","last_active":"2023-10-06T21:08:18.355958+0000","last_peered":"2023-10-06T21:08:18.355958+0000","last_clean":"2023-10-06T21:08:18.355958+0000","last_became_active":"2023-10-06T21:08:15.521471+0000","last_became_peered":"2023-10-06T21:08:15.521471+0000","last_unstale":"2023-10-06T21:08:18.355958+0000","last_undegraded":"2023-10-06T21:08:18.355958+0000","last_fullsized":"2023-10-06T21:08:18.355958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:36.666948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922965+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:08:17.922965+0000","last_peered":"2023-10-06T21:08:17.922965+0000","last_clean":"2023-10-06T21:08:17.922965+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:08:17.922965+0000","last_undegraded":"2023-10-06T21:08:17.922965+0000","last_fullsized":"2023-10-06T21:08:17.922965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846469+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:19.846469+0000","last_peered":"2023-10-06T21:08:19.846469+0000","last_clean":"2023-10-06T21:08:19.846469+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:19.846469+0000","last_undegraded":"2023-10-06T21:08:19.846469+0000","last_fullsized":"2023-10-06T21:08:19.846469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921605+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:08:17.921605+0000","last_peered":"2023-10-06T21:08:17.921605+0000","last_clean":"2023-10-06T21:08:17.921605+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:08:17.921605+0000","last_undegraded":"2023-10-06T21:08:17.921605+0000","last_fullsized":"2023-10-06T21:08:17.921605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.858539+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:20.858539+0000","last_peered":"2023-10-06T21:08:20.858539+0000","last_clean":"2023-10-06T21:08:20.858539+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:20.858539+0000","last_undegraded":"2023-10-06T21:08:20.858539+0000","last_fullsized":"2023-10-06T21:08:20.858539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921797+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:08:17.921797+0000","last_peered":"2023-10-06T21:08:17.921797+0000","last_clean":"2023-10-06T21:08:17.921797+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:08:17.921797+0000","last_undegraded":"2023-10-06T21:08:17.921797+0000","last_fullsized":"2023-10-06T21:08:17.921797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921126+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:17.921126+0000","last_peered":"2023-10-06T21:08:17.921126+0000","last_clean":"2023-10-06T21:08:17.921126+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:17.921126+0000","last_undegraded":"2023-10-06T21:08:17.921126+0000","last_fullsized":"2023-10-06T21:08:17.921126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921350+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:08:17.921350+0000","last_peered":"2023-10-06T21:08:17.921350+0000","last_clean":"2023-10-06T21:08:17.921350+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:08:17.921350+0000","last_undegraded":"2023-10-06T21:08:17.921350+0000","last_fullsized":"2023-10-06T21:08:17.921350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922810+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:17.922810+0000","last_peered":"2023-10-06T21:08:17.922810+0000","last_clean":"2023-10-06T21:08:17.922810+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:17.922810+0000","last_undegraded":"2023-10-06T21:08:17.922810+0000","last_fullsized":"2023-10-06T21:08:17.922810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921741+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:08:17.921741+0000","last_peered":"2023-10-06T21:08:17.921741+0000","last_clean":"2023-10-06T21:08:17.921741+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:08:17.921741+0000","last_undegraded":"2023-10-06T21:08:17.921741+0000","last_fullsized":"2023-10-06T21:08:17.921741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.318893+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:08:18.318893+0000","last_peered":"2023-10-06T21:08:18.318893+0000","last_clean":"2023-10-06T21:08:18.318893+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:08:18.318893+0000","last_undegraded":"2023-10-06T21:08:18.318893+0000","last_fullsized":"2023-10-06T21:08:18.318893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514904+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:08:17.514904+0000","last_peered":"2023-10-06T21:08:17.514904+0000","last_clean":"2023-10-06T21:08:17.514904+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:08:17.514904+0000","last_undegraded":"2023-10-06T21:08:17.514904+0000","last_fullsized":"2023-10-06T21:08:17.514904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514757+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:08:17.514757+0000","last_peered":"2023-10-06T21:08:17.514757+0000","last_clean":"2023-10-06T21:08:17.514757+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:08:17.514757+0000","last_undegraded":"2023-10-06T21:08:17.514757+0000","last_fullsized":"2023-10-06T21:08:17.514757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921809+0000","last_change":"2023-10-06T21:08:16.523748+0000","last_active":"2023-10-06T21:08:17.921809+0000","last_peered":"2023-10-06T21:08:17.921809+0000","last_clean":"2023-10-06T21:08:17.921809+0000","last_became_active":"2023-10-06T21:08:16.523308+0000","last_became_peered":"2023-10-06T21:08:16.523308+0000","last_unstale":"2023-10-06T21:08:17.921809+0000","last_undegraded":"2023-10-06T21:08:17.921809+0000","last_fullsized":"2023-10-06T21:08:17.921809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:09:34.125869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923310+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:08:17.923310+0000","last_peered":"2023-10-06T21:08:17.923310+0000","last_clean":"2023-10-06T21:08:17.923310+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:08:17.923310+0000","last_undegraded":"2023-10-06T21:08:17.923310+0000","last_fullsized":"2023-10-06T21:08:17.923310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319139+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:08:18.319139+0000","last_peered":"2023-10-06T21:08:18.319139+0000","last_clean":"2023-10-06T21:08:18.319139+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:08:18.319139+0000","last_undegraded":"2023-10-06T21:08:18.319139+0000","last_fullsized":"2023-10-06T21:08:18.319139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.515131+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:08:17.515131+0000","last_peered":"2023-10-06T21:08:17.515131+0000","last_clean":"2023-10-06T21:08:17.515131+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:08:17.515131+0000","last_undegraded":"2023-10-06T21:08:17.515131+0000","last_fullsized":"2023-10-06T21:08:17.515131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922366+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:08:17.922366+0000","last_peered":"2023-10-06T21:08:17.922366+0000","last_clean":"2023-10-06T21:08:17.922366+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:08:17.922366+0000","last_undegraded":"2023-10-06T21:08:17.922366+0000","last_fullsized":"2023-10-06T21:08:17.922366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.920897+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:08:17.920897+0000","last_peered":"2023-10-06T21:08:17.920897+0000","last_clean":"2023-10-06T21:08:17.920897+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:08:17.920897+0000","last_undegraded":"2023-10-06T21:08:17.920897+0000","last_fullsized":"2023-10-06T21:08:17.920897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921837+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:08:17.921837+0000","last_peered":"2023-10-06T21:08:17.921837+0000","last_clean":"2023-10-06T21:08:17.921837+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:08:17.921837+0000","last_undegraded":"2023-10-06T21:08:17.921837+0000","last_fullsized":"2023-10-06T21:08:17.921837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513917+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:08:17.513917+0000","last_peered":"2023-10-06T21:08:17.513917+0000","last_clean":"2023-10-06T21:08:17.513917+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:08:17.513917+0000","last_undegraded":"2023-10-06T21:08:17.513917+0000","last_fullsized":"2023-10-06T21:08:17.513917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923540+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:08:17.923540+0000","last_peered":"2023-10-06T21:08:17.923540+0000","last_clean":"2023-10-06T21:08:17.923540+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:08:17.923540+0000","last_undegraded":"2023-10-06T21:08:17.923540+0000","last_fullsized":"2023-10-06T21:08:17.923540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319014+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:08:18.319014+0000","last_peered":"2023-10-06T21:08:18.319014+0000","last_clean":"2023-10-06T21:08:18.319014+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:08:18.319014+0000","last_undegraded":"2023-10-06T21:08:18.319014+0000","last_fullsized":"2023-10-06T21:08:18.319014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22775,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":67821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645328,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1032089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.436},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.873},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":439315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.96299999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2909999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.2310000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.252},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.1259999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":439135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.745},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:07:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332127,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1030691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.437},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79200000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83899999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.54900000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80300000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:07:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":6,"up_from":46,"seq":197568495649,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":438531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.89400000000000002,"5min":1.367,"15min":1.367},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.53300000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.73999999999999999},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2390000000000001,"5min":1.337,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98099999999999998,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86199999999999999,"5min":1.486,"15min":1.486},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.85299999999999998,"15min":0.86799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.123,"5min":1.456,"15min":1.456},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1759999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4570000000000001,"5min":1.51,"15min":1.51},"last":0.92700000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94699999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.073},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1.006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.111}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":440143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79200000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.435,"15min":0.41399999999999998},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.63400000000000001,"15min":0.62},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80700000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.84099999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.877}]},{"osd":10,"last update":"Fri Oct 6 21:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.42799999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.077,"5min":1.216,"15min":1.216},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0720000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.139,"15min":1.139},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.876}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.02},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.90200000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.97599999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0349999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.41299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84099999999999997,"5min":1.004,"15min":1.004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82999999999999996,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.623}]},{"osd":3,"last update":"Fri Oct 6 21:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.76500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1,"15min":1},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80500000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68100000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88200000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.48999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.746},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.61399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70699999999999996},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57599999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39200000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.441}]},{"osd":7,"last update":"Fri Oct 6 21:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83499999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53000000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757287,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1531904,"data_stored":1030044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70999999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":0.63900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.53500000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.747},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.78300000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Oct 6 21:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72999999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.879},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":18977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:08:25.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:08:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.561+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3786023556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc100ea0 msgr2=0x7f1ecc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.561+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3786023556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc1012c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009990 tx=0x7f1ec8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3786023556 shutdown_connections 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3786023556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3786023556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc1012c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3786023556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc102080 0x7f1ecc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3786023556 >> 172.21.15.26:0/3786023556 conn(0x7f1ecc0fc420 msgr2=0x7f1ecc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3786023556 shutdown_connections 2023-10-06T21:08:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.562+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3786023556 wait complete. 2023-10-06T21:08:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 Processor -- start 2023-10-06T21:08:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 -- start start 2023-10-06T21:08:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1a0980 con 0x7f1ecc102080 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1a0ac0 con 0x7f1ecc102a40 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed47ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1a0c50 con 0x7f1ecc100ea0 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed1d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed2d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.563+0000 7f1ed1d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43784/0 (socket says 172.21.15.26:43784) 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 -- 172.21.15.26:0/3685100387 learned_addr learned my addr 172.21.15.26:0/3685100387 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed259b700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 -- 172.21.15.26:0/3685100387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc100ea0 msgr2=0x7f1ecc198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 -- 172.21.15.26:0/3685100387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 msgr2=0x7f1ecc1a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc1a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 -- 172.21.15.26:0/3685100387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80095f0 con 0x7f1ecc102080 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed2d9c700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed1d9a700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00d980 tx=0x7f1ebc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc004020 con 0x7f1ecc102080 2023-10-06T21:08:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc00f690 con 0x7f1ecc102080 2023-10-06T21:08:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003a60 con 0x7f1ecc102080 2023-10-06T21:08:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.564+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc1a0f30 con 0x7f1ecc102080 2023-10-06T21:08:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.565+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc1a13b0 con 0x7f1ecc102080 2023-10-06T21:08:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.566+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc193550 con 0x7f1ecc102080 2023-10-06T21:08:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.568+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1ebc00fb00 con 0x7f1ecc102080 2023-10-06T21:08:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.569+0000 7f1ec37fe700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 0x7f1eb80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.569+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ebc0cc8f0 con 0x7f1ecc102080 2023-10-06T21:08:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.569+0000 7f1ed259b700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 0x7f1eb80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.570+0000 7f1ed259b700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 0x7f1eb80a3a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800b5c0 tx=0x7f1ec8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:25.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.573+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ebc095ea0 con 0x7f1ecc102080 2023-10-06T21:08:25.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.781+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ecc02d350 con 0x7f1eb80a15d0 2023-10-06T21:08:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.791+0000 7f1ec37fe700 1 -- 172.21.15.26:0/3685100387 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335937 (secure 0 0 0) 0x7f1ecc02d350 con 0x7f1eb80a15d0 2023-10-06T21:08:25.798 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 msgr2=0x7f1eb80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 0x7f1eb80a3a90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800b5c0 tx=0x7f1ec8005fd0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 msgr2=0x7f1ecc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00d980 tx=0x7f1ebc00dc90 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 shutdown_connections 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ecc102a40 0x7f1ecc1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ecc100ea0 0x7f1ecc198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1eb80a15d0 0x7f1eb80a3a90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 --2- 172.21.15.26:0/3685100387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ecc102080 0x7f1ecc199340 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 >> 172.21.15.26:0/3685100387 conn(0x7f1ecc0fc420 msgr2=0x7f1ecc1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 shutdown_connections 2023-10-06T21:08:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:25.796+0000 7f1ed47ff700 1 -- 172.21.15.26:0/3685100387 wait complete. 2023-10-06T21:08:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:08:25.818 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2023-10-06T21:08:24.367573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613266,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":155,"num_read_kb":135,"num_write":289,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336872,"kb_used_data":12652,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604952,"statfs":{"total":1151940427776,"available":1151595470848,"internally_reserved":0,"allocated":12955648,"data_stored":7065344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22898,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":108,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001990"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319267+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:08:18.319267+0000","last_peered":"2023-10-06T21:08:18.319267+0000","last_clean":"2023-10-06T21:08:18.319267+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:08:18.319267+0000","last_undegraded":"2023-10-06T21:08:18.319267+0000","last_fullsized":"2023-10-06T21:08:18.319267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922389+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:08:17.922389+0000","last_peered":"2023-10-06T21:08:17.922389+0000","last_clean":"2023-10-06T21:08:17.922389+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:08:17.922389+0000","last_undegraded":"2023-10-06T21:08:17.922389+0000","last_fullsized":"2023-10-06T21:08:17.922389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922246+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:08:17.922246+0000","last_peered":"2023-10-06T21:08:17.922246+0000","last_clean":"2023-10-06T21:08:17.922246+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:08:17.922246+0000","last_undegraded":"2023-10-06T21:08:17.922246+0000","last_fullsized":"2023-10-06T21:08:17.922246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.905583+0000","last_change":"2023-10-06T21:08:16.515864+0000","last_active":"2023-10-06T21:08:17.905583+0000","last_peered":"2023-10-06T21:08:17.905583+0000","last_clean":"2023-10-06T21:08:17.905583+0000","last_became_active":"2023-10-06T21:08:16.515523+0000","last_became_peered":"2023-10-06T21:08:16.515523+0000","last_unstale":"2023-10-06T21:08:17.905583+0000","last_undegraded":"2023-10-06T21:08:17.905583+0000","last_fullsized":"2023-10-06T21:08:17.905583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:41:18.651694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621143+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:08:17.621143+0000","last_peered":"2023-10-06T21:08:17.621143+0000","last_clean":"2023-10-06T21:08:17.621143+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:08:17.621143+0000","last_undegraded":"2023-10-06T21:08:17.621143+0000","last_fullsized":"2023-10-06T21:08:17.621143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922000+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:08:17.922000+0000","last_peered":"2023-10-06T21:08:17.922000+0000","last_clean":"2023-10-06T21:08:17.922000+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:08:17.922000+0000","last_undegraded":"2023-10-06T21:08:17.922000+0000","last_fullsized":"2023-10-06T21:08:17.922000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921288+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:08:17.921288+0000","last_peered":"2023-10-06T21:08:17.921288+0000","last_clean":"2023-10-06T21:08:17.921288+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:08:17.921288+0000","last_undegraded":"2023-10-06T21:08:17.921288+0000","last_fullsized":"2023-10-06T21:08:17.921288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921661+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:08:17.921661+0000","last_peered":"2023-10-06T21:08:17.921661+0000","last_clean":"2023-10-06T21:08:17.921661+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:08:17.921661+0000","last_undegraded":"2023-10-06T21:08:17.921661+0000","last_fullsized":"2023-10-06T21:08:17.921661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514106+0000","last_change":"2023-10-06T21:08:16.516691+0000","last_active":"2023-10-06T21:08:17.514106+0000","last_peered":"2023-10-06T21:08:17.514106+0000","last_clean":"2023-10-06T21:08:17.514106+0000","last_became_active":"2023-10-06T21:08:16.516223+0000","last_became_peered":"2023-10-06T21:08:16.516223+0000","last_unstale":"2023-10-06T21:08:17.514106+0000","last_undegraded":"2023-10-06T21:08:17.514106+0000","last_fullsized":"2023-10-06T21:08:17.514106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:02:33.410158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922236+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:08:17.922236+0000","last_peered":"2023-10-06T21:08:17.922236+0000","last_clean":"2023-10-06T21:08:17.922236+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:08:17.922236+0000","last_undegraded":"2023-10-06T21:08:17.922236+0000","last_fullsized":"2023-10-06T21:08:17.922236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921715+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:08:17.921715+0000","last_peered":"2023-10-06T21:08:17.921715+0000","last_clean":"2023-10-06T21:08:17.921715+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:08:17.921715+0000","last_undegraded":"2023-10-06T21:08:17.921715+0000","last_fullsized":"2023-10-06T21:08:17.921715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.905592+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:08:17.905592+0000","last_peered":"2023-10-06T21:08:17.905592+0000","last_clean":"2023-10-06T21:08:17.905592+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:08:17.905592+0000","last_undegraded":"2023-10-06T21:08:17.905592+0000","last_fullsized":"2023-10-06T21:08:17.905592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923588+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:08:17.923588+0000","last_peered":"2023-10-06T21:08:17.923588+0000","last_clean":"2023-10-06T21:08:17.923588+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:08:17.923588+0000","last_undegraded":"2023-10-06T21:08:17.923588+0000","last_fullsized":"2023-10-06T21:08:17.923588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921886+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:08:17.921886+0000","last_peered":"2023-10-06T21:08:17.921886+0000","last_clean":"2023-10-06T21:08:17.921886+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:08:17.921886+0000","last_undegraded":"2023-10-06T21:08:17.921886+0000","last_fullsized":"2023-10-06T21:08:17.921886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921641+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:08:17.921641+0000","last_peered":"2023-10-06T21:08:17.921641+0000","last_clean":"2023-10-06T21:08:17.921641+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:08:17.921641+0000","last_undegraded":"2023-10-06T21:08:17.921641+0000","last_fullsized":"2023-10-06T21:08:17.921641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513766+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:08:17.513766+0000","last_peered":"2023-10-06T21:08:17.513766+0000","last_clean":"2023-10-06T21:08:17.513766+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:08:17.513766+0000","last_undegraded":"2023-10-06T21:08:17.513766+0000","last_fullsized":"2023-10-06T21:08:17.513766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319305+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:08:18.319305+0000","last_peered":"2023-10-06T21:08:18.319305+0000","last_clean":"2023-10-06T21:08:18.319305+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:08:18.319305+0000","last_undegraded":"2023-10-06T21:08:18.319305+0000","last_fullsized":"2023-10-06T21:08:18.319305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514445+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:08:17.514445+0000","last_peered":"2023-10-06T21:08:17.514445+0000","last_clean":"2023-10-06T21:08:17.514445+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:08:17.514445+0000","last_undegraded":"2023-10-06T21:08:17.514445+0000","last_fullsized":"2023-10-06T21:08:17.514445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514952+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:08:17.514952+0000","last_peered":"2023-10-06T21:08:17.514952+0000","last_clean":"2023-10-06T21:08:17.514952+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:08:17.514952+0000","last_undegraded":"2023-10-06T21:08:17.514952+0000","last_fullsized":"2023-10-06T21:08:17.514952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.543742+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:16.543742+0000","last_peered":"2023-10-06T21:08:16.543742+0000","last_clean":"2023-10-06T21:08:16.543742+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:16.543742+0000","last_undegraded":"2023-10-06T21:08:16.543742+0000","last_fullsized":"2023-10-06T21:08:16.543742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922048+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:08:17.922048+0000","last_peered":"2023-10-06T21:08:17.922048+0000","last_clean":"2023-10-06T21:08:17.922048+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:08:17.922048+0000","last_undegraded":"2023-10-06T21:08:17.922048+0000","last_fullsized":"2023-10-06T21:08:17.922048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.541370+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:16.541370+0000","last_peered":"2023-10-06T21:08:16.541370+0000","last_clean":"2023-10-06T21:08:16.541370+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:16.541370+0000","last_undegraded":"2023-10-06T21:08:16.541370+0000","last_fullsized":"2023-10-06T21:08:16.541370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922312+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:08:17.922312+0000","last_peered":"2023-10-06T21:08:17.922312+0000","last_clean":"2023-10-06T21:08:17.922312+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:08:17.922312+0000","last_undegraded":"2023-10-06T21:08:17.922312+0000","last_fullsized":"2023-10-06T21:08:17.922312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.539828+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:16.539828+0000","last_peered":"2023-10-06T21:08:16.539828+0000","last_clean":"2023-10-06T21:08:16.539828+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:16.539828+0000","last_undegraded":"2023-10-06T21:08:16.539828+0000","last_fullsized":"2023-10-06T21:08:16.539828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513912+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:08:17.513912+0000","last_peered":"2023-10-06T21:08:17.513912+0000","last_clean":"2023-10-06T21:08:17.513912+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:08:17.513912+0000","last_undegraded":"2023-10-06T21:08:17.513912+0000","last_fullsized":"2023-10-06T21:08:17.513912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.541045+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:16.541045+0000","last_peered":"2023-10-06T21:08:16.541045+0000","last_clean":"2023-10-06T21:08:16.541045+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:16.541045+0000","last_undegraded":"2023-10-06T21:08:16.541045+0000","last_fullsized":"2023-10-06T21:08:16.541045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921279+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:08:17.921279+0000","last_peered":"2023-10-06T21:08:17.921279+0000","last_clean":"2023-10-06T21:08:17.921279+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:08:17.921279+0000","last_undegraded":"2023-10-06T21:08:17.921279+0000","last_fullsized":"2023-10-06T21:08:17.921279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.859123+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:08:21.859123+0000","last_peered":"2023-10-06T21:08:21.859123+0000","last_clean":"2023-10-06T21:08:21.859123+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:08:21.859123+0000","last_undegraded":"2023-10-06T21:08:21.859123+0000","last_fullsized":"2023-10-06T21:08:21.859123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621178+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:08:17.621178+0000","last_peered":"2023-10-06T21:08:17.621178+0000","last_clean":"2023-10-06T21:08:17.621178+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:08:17.621178+0000","last_undegraded":"2023-10-06T21:08:17.621178+0000","last_fullsized":"2023-10-06T21:08:17.621178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.855107+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:20.855107+0000","last_peered":"2023-10-06T21:08:20.855107+0000","last_clean":"2023-10-06T21:08:20.855107+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:20.855107+0000","last_undegraded":"2023-10-06T21:08:20.855107+0000","last_fullsized":"2023-10-06T21:08:20.855107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921275+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:08:17.921275+0000","last_peered":"2023-10-06T21:08:17.921275+0000","last_clean":"2023-10-06T21:08:17.921275+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:08:17.921275+0000","last_undegraded":"2023-10-06T21:08:17.921275+0000","last_fullsized":"2023-10-06T21:08:17.921275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922899+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:17.922899+0000","last_peered":"2023-10-06T21:08:17.922899+0000","last_clean":"2023-10-06T21:08:17.922899+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:17.922899+0000","last_undegraded":"2023-10-06T21:08:17.922899+0000","last_fullsized":"2023-10-06T21:08:17.922899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921512+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:08:17.921512+0000","last_peered":"2023-10-06T21:08:17.921512+0000","last_clean":"2023-10-06T21:08:17.921512+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:08:17.921512+0000","last_undegraded":"2023-10-06T21:08:17.921512+0000","last_fullsized":"2023-10-06T21:08:17.921512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.920971+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:17.920971+0000","last_peered":"2023-10-06T21:08:17.920971+0000","last_clean":"2023-10-06T21:08:17.920971+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:17.920971+0000","last_undegraded":"2023-10-06T21:08:17.920971+0000","last_fullsized":"2023-10-06T21:08:17.920971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514435+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:08:17.514435+0000","last_peered":"2023-10-06T21:08:17.514435+0000","last_clean":"2023-10-06T21:08:17.514435+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:08:17.514435+0000","last_undegraded":"2023-10-06T21:08:17.514435+0000","last_fullsized":"2023-10-06T21:08:17.514435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.847343+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:08:19.847343+0000","last_peered":"2023-10-06T21:08:19.847343+0000","last_clean":"2023-10-06T21:08:19.847343+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:08:19.847343+0000","last_undegraded":"2023-10-06T21:08:19.847343+0000","last_fullsized":"2023-10-06T21:08:19.847343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514172+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:08:17.514172+0000","last_peered":"2023-10-06T21:08:17.514172+0000","last_clean":"2023-10-06T21:08:17.514172+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:08:17.514172+0000","last_undegraded":"2023-10-06T21:08:17.514172+0000","last_fullsized":"2023-10-06T21:08:17.514172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846047+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:19.846047+0000","last_peered":"2023-10-06T21:08:19.846047+0000","last_clean":"2023-10-06T21:08:19.846047+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:19.846047+0000","last_undegraded":"2023-10-06T21:08:19.846047+0000","last_fullsized":"2023-10-06T21:08:19.846047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922161+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:08:17.922161+0000","last_peered":"2023-10-06T21:08:17.922161+0000","last_clean":"2023-10-06T21:08:17.922161+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:08:17.922161+0000","last_undegraded":"2023-10-06T21:08:17.922161+0000","last_fullsized":"2023-10-06T21:08:17.922161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846712+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:19.846712+0000","last_peered":"2023-10-06T21:08:19.846712+0000","last_clean":"2023-10-06T21:08:19.846712+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:19.846712+0000","last_undegraded":"2023-10-06T21:08:19.846712+0000","last_fullsized":"2023-10-06T21:08:19.846712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514034+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:08:17.514034+0000","last_peered":"2023-10-06T21:08:17.514034+0000","last_clean":"2023-10-06T21:08:17.514034+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:08:17.514034+0000","last_undegraded":"2023-10-06T21:08:17.514034+0000","last_fullsized":"2023-10-06T21:08:17.514034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'14","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:22.375164+0000","last_change":"2023-10-06T21:08:15.524431+0000","last_active":"2023-10-06T21:08:22.375164+0000","last_peered":"2023-10-06T21:08:22.375164+0000","last_clean":"2023-10-06T21:08:22.375164+0000","last_became_active":"2023-10-06T21:08:15.524064+0000","last_became_peered":"2023-10-06T21:08:15.524064+0000","last_unstale":"2023-10-06T21:08:22.375164+0000","last_undegraded":"2023-10-06T21:08:22.375164+0000","last_fullsized":"2023-10-06T21:08:22.375164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:09:11.585643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621270+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:08:17.621270+0000","last_peered":"2023-10-06T21:08:17.621270+0000","last_clean":"2023-10-06T21:08:17.621270+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:08:17.621270+0000","last_undegraded":"2023-10-06T21:08:17.621270+0000","last_fullsized":"2023-10-06T21:08:17.621270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.856131+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:20.856131+0000","last_peered":"2023-10-06T21:08:20.856131+0000","last_clean":"2023-10-06T21:08:20.856131+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:20.856131+0000","last_undegraded":"2023-10-06T21:08:20.856131+0000","last_fullsized":"2023-10-06T21:08:20.856131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923280+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:08:17.923280+0000","last_peered":"2023-10-06T21:08:17.923280+0000","last_clean":"2023-10-06T21:08:17.923280+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:08:17.923280+0000","last_undegraded":"2023-10-06T21:08:17.923280+0000","last_fullsized":"2023-10-06T21:08:17.923280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846427+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:19.846427+0000","last_peered":"2023-10-06T21:08:19.846427+0000","last_clean":"2023-10-06T21:08:19.846427+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:19.846427+0000","last_undegraded":"2023-10-06T21:08:19.846427+0000","last_fullsized":"2023-10-06T21:08:19.846427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921642+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:08:17.921642+0000","last_peered":"2023-10-06T21:08:17.921642+0000","last_clean":"2023-10-06T21:08:17.921642+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:08:17.921642+0000","last_undegraded":"2023-10-06T21:08:17.921642+0000","last_fullsized":"2023-10-06T21:08:17.921642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.860409+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:08:21.860409+0000","last_peered":"2023-10-06T21:08:21.860409+0000","last_clean":"2023-10-06T21:08:21.860409+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:08:21.860409+0000","last_undegraded":"2023-10-06T21:08:21.860409+0000","last_fullsized":"2023-10-06T21:08:21.860409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921207+0000","last_change":"2023-10-06T21:08:16.515157+0000","last_active":"2023-10-06T21:08:17.921207+0000","last_peered":"2023-10-06T21:08:17.921207+0000","last_clean":"2023-10-06T21:08:17.921207+0000","last_became_active":"2023-10-06T21:08:16.514847+0000","last_became_peered":"2023-10-06T21:08:16.514847+0000","last_unstale":"2023-10-06T21:08:17.921207+0000","last_undegraded":"2023-10-06T21:08:17.921207+0000","last_fullsized":"2023-10-06T21:08:17.921207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:56:34.986136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.845788+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:19.845788+0000","last_peered":"2023-10-06T21:08:19.845788+0000","last_clean":"2023-10-06T21:08:19.845788+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:19.845788+0000","last_undegraded":"2023-10-06T21:08:19.845788+0000","last_fullsized":"2023-10-06T21:08:19.845788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921104+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:08:17.921104+0000","last_peered":"2023-10-06T21:08:17.921104+0000","last_clean":"2023-10-06T21:08:17.921104+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:08:17.921104+0000","last_undegraded":"2023-10-06T21:08:17.921104+0000","last_fullsized":"2023-10-06T21:08:17.921104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.539931+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:16.539931+0000","last_peered":"2023-10-06T21:08:16.539931+0000","last_clean":"2023-10-06T21:08:16.539931+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:16.539931+0000","last_undegraded":"2023-10-06T21:08:16.539931+0000","last_fullsized":"2023-10-06T21:08:16.539931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319169+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:08:18.319169+0000","last_peered":"2023-10-06T21:08:18.319169+0000","last_clean":"2023-10-06T21:08:18.319169+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:08:18.319169+0000","last_undegraded":"2023-10-06T21:08:18.319169+0000","last_fullsized":"2023-10-06T21:08:18.319169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.501102+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:16.501102+0000","last_peered":"2023-10-06T21:08:16.501102+0000","last_clean":"2023-10-06T21:08:16.501102+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:16.501102+0000","last_undegraded":"2023-10-06T21:08:16.501102+0000","last_fullsized":"2023-10-06T21:08:16.501102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922282+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:08:17.922282+0000","last_peered":"2023-10-06T21:08:17.922282+0000","last_clean":"2023-10-06T21:08:17.922282+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:08:17.922282+0000","last_undegraded":"2023-10-06T21:08:17.922282+0000","last_fullsized":"2023-10-06T21:08:17.922282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921505+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:17.921505+0000","last_peered":"2023-10-06T21:08:17.921505+0000","last_clean":"2023-10-06T21:08:17.921505+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:17.921505+0000","last_undegraded":"2023-10-06T21:08:17.921505+0000","last_fullsized":"2023-10-06T21:08:17.921505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922161+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:08:17.922161+0000","last_peered":"2023-10-06T21:08:17.922161+0000","last_clean":"2023-10-06T21:08:17.922161+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:08:17.922161+0000","last_undegraded":"2023-10-06T21:08:17.922161+0000","last_fullsized":"2023-10-06T21:08:17.922161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921143+0000","last_change":"2023-10-06T21:08:16.512032+0000","last_active":"2023-10-06T21:08:17.921143+0000","last_peered":"2023-10-06T21:08:17.921143+0000","last_clean":"2023-10-06T21:08:17.921143+0000","last_became_active":"2023-10-06T21:08:16.511717+0000","last_became_peered":"2023-10-06T21:08:16.511717+0000","last_unstale":"2023-10-06T21:08:17.921143+0000","last_undegraded":"2023-10-06T21:08:17.921143+0000","last_fullsized":"2023-10-06T21:08:17.921143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:02:34.118930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922814+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:08:17.922814+0000","last_peered":"2023-10-06T21:08:17.922814+0000","last_clean":"2023-10-06T21:08:17.922814+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:08:17.922814+0000","last_undegraded":"2023-10-06T21:08:17.922814+0000","last_fullsized":"2023-10-06T21:08:17.922814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922753+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:08:17.922753+0000","last_peered":"2023-10-06T21:08:17.922753+0000","last_clean":"2023-10-06T21:08:17.922753+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:08:17.922753+0000","last_undegraded":"2023-10-06T21:08:17.922753+0000","last_fullsized":"2023-10-06T21:08:17.922753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:21.855802+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:21.855802+0000","last_peered":"2023-10-06T21:08:21.855802+0000","last_clean":"2023-10-06T21:08:21.855802+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:21.855802+0000","last_undegraded":"2023-10-06T21:08:21.855802+0000","last_fullsized":"2023-10-06T21:08:21.855802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.507232+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:16.507232+0000","last_peered":"2023-10-06T21:08:16.507232+0000","last_clean":"2023-10-06T21:08:16.507232+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:16.507232+0000","last_undegraded":"2023-10-06T21:08:16.507232+0000","last_fullsized":"2023-10-06T21:08:16.507232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922723+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:08:17.922723+0000","last_peered":"2023-10-06T21:08:17.922723+0000","last_clean":"2023-10-06T21:08:17.922723+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:08:17.922723+0000","last_undegraded":"2023-10-06T21:08:17.922723+0000","last_fullsized":"2023-10-06T21:08:17.922723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.855634+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:20.855634+0000","last_peered":"2023-10-06T21:08:20.855634+0000","last_clean":"2023-10-06T21:08:20.855634+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:20.855634+0000","last_undegraded":"2023-10-06T21:08:20.855634+0000","last_fullsized":"2023-10-06T21:08:20.855634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621223+0000","last_change":"2023-10-06T21:08:15.524380+0000","last_active":"2023-10-06T21:08:17.621223+0000","last_peered":"2023-10-06T21:08:17.621223+0000","last_clean":"2023-10-06T21:08:17.621223+0000","last_became_active":"2023-10-06T21:08:15.524059+0000","last_became_peered":"2023-10-06T21:08:15.524059+0000","last_unstale":"2023-10-06T21:08:17.621223+0000","last_undegraded":"2023-10-06T21:08:17.621223+0000","last_fullsized":"2023-10-06T21:08:17.621223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:23:54.360603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923175+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:08:17.923175+0000","last_peered":"2023-10-06T21:08:17.923175+0000","last_clean":"2023-10-06T21:08:17.923175+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:08:17.923175+0000","last_undegraded":"2023-10-06T21:08:17.923175+0000","last_fullsized":"2023-10-06T21:08:17.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-06T21:08:16.501641+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:16.501641+0000","last_peered":"2023-10-06T21:08:16.501641+0000","last_clean":"2023-10-06T21:08:16.501641+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:16.501641+0000","last_undegraded":"2023-10-06T21:08:16.501641+0000","last_fullsized":"2023-10-06T21:08:16.501641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922497+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:08:17.922497+0000","last_peered":"2023-10-06T21:08:17.922497+0000","last_clean":"2023-10-06T21:08:17.922497+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:08:17.922497+0000","last_undegraded":"2023-10-06T21:08:17.922497+0000","last_fullsized":"2023-10-06T21:08:17.922497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.848020+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:19.848020+0000","last_peered":"2023-10-06T21:08:19.848020+0000","last_clean":"2023-10-06T21:08:19.848020+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:19.848020+0000","last_undegraded":"2023-10-06T21:08:19.848020+0000","last_fullsized":"2023-10-06T21:08:19.848020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922542+0000","last_change":"2023-10-06T21:08:16.525576+0000","last_active":"2023-10-06T21:08:17.922542+0000","last_peered":"2023-10-06T21:08:17.922542+0000","last_clean":"2023-10-06T21:08:17.922542+0000","last_became_active":"2023-10-06T21:08:16.525250+0000","last_became_peered":"2023-10-06T21:08:16.525250+0000","last_unstale":"2023-10-06T21:08:17.922542+0000","last_undegraded":"2023-10-06T21:08:17.922542+0000","last_fullsized":"2023-10-06T21:08:17.922542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:37:03.571384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846599+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:19.846599+0000","last_peered":"2023-10-06T21:08:19.846599+0000","last_clean":"2023-10-06T21:08:19.846599+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:19.846599+0000","last_undegraded":"2023-10-06T21:08:19.846599+0000","last_fullsized":"2023-10-06T21:08:19.846599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.621068+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:08:17.621068+0000","last_peered":"2023-10-06T21:08:17.621068+0000","last_clean":"2023-10-06T21:08:17.621068+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:08:17.621068+0000","last_undegraded":"2023-10-06T21:08:17.621068+0000","last_fullsized":"2023-10-06T21:08:17.621068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.847239+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:19.847239+0000","last_peered":"2023-10-06T21:08:19.847239+0000","last_clean":"2023-10-06T21:08:19.847239+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:19.847239+0000","last_undegraded":"2023-10-06T21:08:19.847239+0000","last_fullsized":"2023-10-06T21:08:19.847239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.318841+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:08:18.318841+0000","last_peered":"2023-10-06T21:08:18.318841+0000","last_clean":"2023-10-06T21:08:18.318841+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:08:18.318841+0000","last_undegraded":"2023-10-06T21:08:18.318841+0000","last_fullsized":"2023-10-06T21:08:18.318841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.355958+0000","last_change":"2023-10-06T21:08:15.521774+0000","last_active":"2023-10-06T21:08:18.355958+0000","last_peered":"2023-10-06T21:08:18.355958+0000","last_clean":"2023-10-06T21:08:18.355958+0000","last_became_active":"2023-10-06T21:08:15.521471+0000","last_became_peered":"2023-10-06T21:08:15.521471+0000","last_unstale":"2023-10-06T21:08:18.355958+0000","last_undegraded":"2023-10-06T21:08:18.355958+0000","last_fullsized":"2023-10-06T21:08:18.355958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:09:36.666948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922965+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:08:17.922965+0000","last_peered":"2023-10-06T21:08:17.922965+0000","last_clean":"2023-10-06T21:08:17.922965+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:08:17.922965+0000","last_undegraded":"2023-10-06T21:08:17.922965+0000","last_fullsized":"2023-10-06T21:08:17.922965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:19.846469+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:19.846469+0000","last_peered":"2023-10-06T21:08:19.846469+0000","last_clean":"2023-10-06T21:08:19.846469+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:19.846469+0000","last_undegraded":"2023-10-06T21:08:19.846469+0000","last_fullsized":"2023-10-06T21:08:19.846469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921605+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:08:17.921605+0000","last_peered":"2023-10-06T21:08:17.921605+0000","last_clean":"2023-10-06T21:08:17.921605+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:08:17.921605+0000","last_undegraded":"2023-10-06T21:08:17.921605+0000","last_fullsized":"2023-10-06T21:08:17.921605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:20.858539+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:20.858539+0000","last_peered":"2023-10-06T21:08:20.858539+0000","last_clean":"2023-10-06T21:08:20.858539+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:20.858539+0000","last_undegraded":"2023-10-06T21:08:20.858539+0000","last_fullsized":"2023-10-06T21:08:20.858539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921797+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:08:17.921797+0000","last_peered":"2023-10-06T21:08:17.921797+0000","last_clean":"2023-10-06T21:08:17.921797+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:08:17.921797+0000","last_undegraded":"2023-10-06T21:08:17.921797+0000","last_fullsized":"2023-10-06T21:08:17.921797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921126+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:17.921126+0000","last_peered":"2023-10-06T21:08:17.921126+0000","last_clean":"2023-10-06T21:08:17.921126+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:17.921126+0000","last_undegraded":"2023-10-06T21:08:17.921126+0000","last_fullsized":"2023-10-06T21:08:17.921126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921350+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:08:17.921350+0000","last_peered":"2023-10-06T21:08:17.921350+0000","last_clean":"2023-10-06T21:08:17.921350+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:08:17.921350+0000","last_undegraded":"2023-10-06T21:08:17.921350+0000","last_fullsized":"2023-10-06T21:08:17.921350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922810+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:17.922810+0000","last_peered":"2023-10-06T21:08:17.922810+0000","last_clean":"2023-10-06T21:08:17.922810+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:17.922810+0000","last_undegraded":"2023-10-06T21:08:17.922810+0000","last_fullsized":"2023-10-06T21:08:17.922810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921741+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:08:17.921741+0000","last_peered":"2023-10-06T21:08:17.921741+0000","last_clean":"2023-10-06T21:08:17.921741+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:08:17.921741+0000","last_undegraded":"2023-10-06T21:08:17.921741+0000","last_fullsized":"2023-10-06T21:08:17.921741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.318893+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:08:18.318893+0000","last_peered":"2023-10-06T21:08:18.318893+0000","last_clean":"2023-10-06T21:08:18.318893+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:08:18.318893+0000","last_undegraded":"2023-10-06T21:08:18.318893+0000","last_fullsized":"2023-10-06T21:08:18.318893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514904+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:08:17.514904+0000","last_peered":"2023-10-06T21:08:17.514904+0000","last_clean":"2023-10-06T21:08:17.514904+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:08:17.514904+0000","last_undegraded":"2023-10-06T21:08:17.514904+0000","last_fullsized":"2023-10-06T21:08:17.514904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.514757+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:08:17.514757+0000","last_peered":"2023-10-06T21:08:17.514757+0000","last_clean":"2023-10-06T21:08:17.514757+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:08:17.514757+0000","last_undegraded":"2023-10-06T21:08:17.514757+0000","last_fullsized":"2023-10-06T21:08:17.514757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921809+0000","last_change":"2023-10-06T21:08:16.523748+0000","last_active":"2023-10-06T21:08:17.921809+0000","last_peered":"2023-10-06T21:08:17.921809+0000","last_clean":"2023-10-06T21:08:17.921809+0000","last_became_active":"2023-10-06T21:08:16.523308+0000","last_became_peered":"2023-10-06T21:08:16.523308+0000","last_unstale":"2023-10-06T21:08:17.921809+0000","last_undegraded":"2023-10-06T21:08:17.921809+0000","last_fullsized":"2023-10-06T21:08:17.921809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:09:34.125869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923310+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:08:17.923310+0000","last_peered":"2023-10-06T21:08:17.923310+0000","last_clean":"2023-10-06T21:08:17.923310+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:08:17.923310+0000","last_undegraded":"2023-10-06T21:08:17.923310+0000","last_fullsized":"2023-10-06T21:08:17.923310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319139+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:08:18.319139+0000","last_peered":"2023-10-06T21:08:18.319139+0000","last_clean":"2023-10-06T21:08:18.319139+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:08:18.319139+0000","last_undegraded":"2023-10-06T21:08:18.319139+0000","last_fullsized":"2023-10-06T21:08:18.319139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.515131+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:08:17.515131+0000","last_peered":"2023-10-06T21:08:17.515131+0000","last_clean":"2023-10-06T21:08:17.515131+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:08:17.515131+0000","last_undegraded":"2023-10-06T21:08:17.515131+0000","last_fullsized":"2023-10-06T21:08:17.515131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.922366+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:08:17.922366+0000","last_peered":"2023-10-06T21:08:17.922366+0000","last_clean":"2023-10-06T21:08:17.922366+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:08:17.922366+0000","last_undegraded":"2023-10-06T21:08:17.922366+0000","last_fullsized":"2023-10-06T21:08:17.922366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.920897+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:08:17.920897+0000","last_peered":"2023-10-06T21:08:17.920897+0000","last_clean":"2023-10-06T21:08:17.920897+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:08:17.920897+0000","last_undegraded":"2023-10-06T21:08:17.920897+0000","last_fullsized":"2023-10-06T21:08:17.920897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.921837+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:08:17.921837+0000","last_peered":"2023-10-06T21:08:17.921837+0000","last_clean":"2023-10-06T21:08:17.921837+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:08:17.921837+0000","last_undegraded":"2023-10-06T21:08:17.921837+0000","last_fullsized":"2023-10-06T21:08:17.921837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.513917+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:08:17.513917+0000","last_peered":"2023-10-06T21:08:17.513917+0000","last_clean":"2023-10-06T21:08:17.513917+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:08:17.513917+0000","last_undegraded":"2023-10-06T21:08:17.513917+0000","last_fullsized":"2023-10-06T21:08:17.513917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:17.923540+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:08:17.923540+0000","last_peered":"2023-10-06T21:08:17.923540+0000","last_clean":"2023-10-06T21:08:17.923540+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:08:17.923540+0000","last_undegraded":"2023-10-06T21:08:17.923540+0000","last_fullsized":"2023-10-06T21:08:17.923540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-06T21:08:18.319014+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:08:18.319014+0000","last_peered":"2023-10-06T21:08:18.319014+0000","last_clean":"2023-10-06T21:08:18.319014+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:08:18.319014+0000","last_undegraded":"2023-10-06T21:08:18.319014+0000","last_fullsized":"2023-10-06T21:08:18.319014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22775,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":19,"num_read_kb":16,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":67821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645328,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1032089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.436},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.873},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":439315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.96299999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2909999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.2310000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.252},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.1259999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070489,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":439135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.745},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:07:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332127,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1030691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.437},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79200000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83899999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.54900000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80300000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:07:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":6,"up_from":46,"seq":197568495649,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":438531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.38700000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.68700000000000006,"15min":0.67400000000000004},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.55700000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.89400000000000002,"5min":1.367,"15min":1.367},"last":0.72899999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.53300000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.751,"15min":0.73999999999999999},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2390000000000001,"5min":1.337,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98099999999999998,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86199999999999999,"5min":1.486,"15min":1.486},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.49399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 21:07:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.85299999999999998,"15min":0.86799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.123,"5min":1.456,"15min":1.456},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.21,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83799999999999997,"15min":0.84699999999999998},"min":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1759999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.90300000000000002,"15min":0.90700000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4570000000000001,"5min":1.51,"15min":1.51},"last":0.92700000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.94699999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.073},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":1.006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.111}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":440143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51000000000000001},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79200000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.435,"15min":0.41399999999999998},"min":{"1min":0.34000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.47299999999999998,"15min":0.46300000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.63400000000000001,"15min":0.62},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 21:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.53800000000000003,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80700000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.84099999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.877}]},{"osd":10,"last update":"Fri Oct 6 21:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.751},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.42799999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81999999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.58499999999999996,"15min":0.54700000000000004},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.077,"5min":1.216,"15min":1.216},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65100000000000002,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0720000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72199999999999998,"15min":0.72099999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.139,"15min":1.139},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.876}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.02},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.90200000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.97599999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0349999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.41299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84099999999999997,"5min":1.004,"15min":1.004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82999999999999996,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.67900000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.623}]},{"osd":3,"last update":"Fri Oct 6 21:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81299999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.76500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.63800000000000001},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1,"15min":1},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80500000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68100000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.56999999999999995},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88200000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:07:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.48999999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93300000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.629,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.746},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.61399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.61299999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.70699999999999996},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.58799999999999997}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.55100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920813,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.374},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.35699999999999998},"min":{"1min":0.26700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57599999999999996,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54900000000000004,"15min":0.53100000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39200000000000002}]},{"osd":5,"last update":"Fri Oct 6 21:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Oct 6 21:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.441}]},{"osd":7,"last update":"Fri Oct 6 21:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83499999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60299999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:07:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53000000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757287,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28512,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716640,"statfs":{"total":95995035648,"available":95965839360,"internally_reserved":0,"allocated":1531904,"data_stored":1030044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70999999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":0.63900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.53500000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.65000000000000002,"15min":0.65400000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.747},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.78300000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Oct 6 21:07:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72999999999999998}]},{"osd":10,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.879},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":18977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:08:25.826 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T21:08:25.827 INFO:tasks.cephadm:Setting up client nodes... 2023-10-06T21:08:25.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-06T21:08:26.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:25 smithi026 ceph-mon[98490]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:26.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:25 smithi078 ceph-mon[134896]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:26.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:25 smithi167 ceph-mon[134494]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:27.152 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:26 smithi026 ceph-mon[98490]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:27.152 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:26 smithi026 ceph-mon[98490]: pgmap v276: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-10-06T21:08:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:26 smithi167 ceph-mon[134494]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:26 smithi167 ceph-mon[134494]: pgmap v276: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-10-06T21:08:27.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:26 smithi078 ceph-mon[134896]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:08:27.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:26 smithi078 ceph-mon[134896]: pgmap v276: 97 pgs: 97 active+clean; 599 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 -- 172.21.15.26:0/3498189815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 msgr2=0x7fc5f01013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/3498189815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f01013d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40099c0 tx=0x7fc5e4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 -- 172.21.15.26:0/3498189815 shutdown_connections 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/3498189815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 0x7fc5f0109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/3498189815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f01013d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/3498189815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 0x7fc5f0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.116+0000 7fc5f6e35700 1 -- 172.21.15.26:0/3498189815 >> 172.21.15.26:0/3498189815 conn(0x7fc5f00fc570 msgr2=0x7fc5f00fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.117+0000 7fc5f6e35700 1 -- 172.21.15.26:0/3498189815 shutdown_connections 2023-10-06T21:08:28.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.117+0000 7fc5f6e35700 1 -- 172.21.15.26:0/3498189815 wait complete. 2023-10-06T21:08:28.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 Processor -- start 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 -- start start 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 0x7fc5f00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 0x7fc5f00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0073f60 con 0x7fc5f0100fb0 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00740d0 con 0x7fc5f01021b0 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f6e35700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01a0cf0 con 0x7fc5f0102b50 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.118+0000 7fc5f4bd1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43804/0 (socket says 172.21.15.26:43804) 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 -- 172.21.15.26:0/2662960024 learned_addr learned my addr 172.21.15.26:0/2662960024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:28.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5effff700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 0x7fc5f00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f53d2700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 0x7fc5f00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 -- 172.21.15.26:0/2662960024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 msgr2=0x7fc5f00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 0x7fc5f00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 -- 172.21.15.26:0/2662960024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 msgr2=0x7fc5f00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 0x7fc5f00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 -- 172.21.15.26:0/2662960024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40095f0 con 0x7fc5f0100fb0 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5f4bd1700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40098e0 tx=0x7fc5e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.119+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e401d070 con 0x7fc5f0100fb0 2023-10-06T21:08:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.120+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e4005050 con 0x7fc5f0100fb0 2023-10-06T21:08:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.120+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01a0f70 con 0x7fc5f0100fb0 2023-10-06T21:08:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.120+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4004b40 con 0x7fc5f0100fb0 2023-10-06T21:08:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.120+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1320 con 0x7fc5f0100fb0 2023-10-06T21:08:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.121+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0107f70 con 0x7fc5f0100fb0 2023-10-06T21:08:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.123+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc5e4004ce0 con 0x7fc5f0100fb0 2023-10-06T21:08:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.124+0000 7fc5edffb700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 0x7fc5e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.124+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5e40cdb40 con 0x7fc5f0100fb0 2023-10-06T21:08:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.124+0000 7fc5effff700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 0x7fc5e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:28.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.125+0000 7fc5effff700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 0x7fc5e00a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc007c60 tx=0x7fc5dc0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.128+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5e4097180 con 0x7fc5f0100fb0 2023-10-06T21:08:28.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.426+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc5f010c800 con 0x7fc5f0100fb0 2023-10-06T21:08:28.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.429+0000 7fc5edffb700 1 -- 172.21.15.26:0/2662960024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc5e409a7a0 con 0x7fc5f0100fb0 2023-10-06T21:08:28.431 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2023-10-06T21:08:28.431 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBMdyBlxsKFGRAAzRsezklt7cRbYVDgKZVSfw== 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 msgr2=0x7fc5e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 0x7fc5e00a3a50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc007c60 tx=0x7fc5dc0073d0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 msgr2=0x7fc5f0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40098e0 tx=0x7fc5e400f690 comp rx=0 tx=0).stop 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 shutdown_connections 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5f0102b50 0x7fc5f00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5e00a1590 0x7fc5e00a3a50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f0100fb0 0x7fc5f0074c50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 --2- 172.21.15.26:0/2662960024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5f01021b0 0x7fc5f00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 >> 172.21.15.26:0/2662960024 conn(0x7fc5f00fc570 msgr2=0x7fc5f00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 shutdown_connections 2023-10-06T21:08:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:28.431+0000 7fc5f6e35700 1 -- 172.21.15.26:0/2662960024 wait complete. 2023-10-06T21:08:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:28 smithi078 ceph-mon[134896]: pgmap v277: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2023-10-06T21:08:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:28 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:08:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:28 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:08:29.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:28 smithi026 ceph-mon[98490]: pgmap v277: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2023-10-06T21:08:29.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:28 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:08:29.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:28 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:08:29.177 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:08:29.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-06T21:08:29.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T21:08:29.281 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T21:08:29.281 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-06T21:08:29.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-06T21:08:29.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:28 smithi167 ceph-mon[134494]: pgmap v277: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2023-10-06T21:08:29.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:28 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-06T21:08:29.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:28 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2662960024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-06T21:08:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 -- 172.21.15.26:0/3404144059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 msgr2=0x7f20fc101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:32.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 --2- 172.21.15.26:0/3404144059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc101400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20f00099c0 tx=0x7f20f0009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 -- 172.21.15.26:0/3404144059 shutdown_connections 2023-10-06T21:08:32.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 --2- 172.21.15.26:0/3404144059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc101940 0x7f20fc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 --2- 172.21.15.26:0/3404144059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc106f90 0x7f20fc107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 --2- 172.21.15.26:0/3404144059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc101400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 -- 172.21.15.26:0/3404144059 >> 172.21.15.26:0/3404144059 conn(0x7f20fc0780a0 msgr2=0x7f20fc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 -- 172.21.15.26:0/3404144059 shutdown_connections 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.581+0000 7f2100b40700 1 -- 172.21.15.26:0/3404144059 wait complete. 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.582+0000 7f2100b40700 1 Processor -- start 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.582+0000 7f2100b40700 1 -- start start 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc101940 0x7f20fc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc106f90 0x7f20fc19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e150 con 0x7f20fc100f80 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e2c0 con 0x7f20fc106f90 2023-10-06T21:08:32.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f2100b40700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e480 con 0x7f20fc101940 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33350/0 (socket says 172.21.15.26:33350) 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f20fa59c700 1 -- 172.21.15.26:0/2501653882 learned_addr learned my addr 172.21.15.26:0/2501653882 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f20f9d9b700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc101940 0x7f20fc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.583+0000 7f20fad9d700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc106f90 0x7f20fc19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 -- 172.21.15.26:0/2501653882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc101940 msgr2=0x7f20fc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc101940 0x7f20fc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 -- 172.21.15.26:0/2501653882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc106f90 msgr2=0x7f20fc19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc106f90 0x7f20fc19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 -- 172.21.15.26:0/2501653882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00095f0 con 0x7f20fc100f80 2023-10-06T21:08:32.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20fa59c700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d980 tx=0x7f20ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:32.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004220 con 0x7f20fc100f80 2023-10-06T21:08:33.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc19e760 con 0x7f20fc100f80 2023-10-06T21:08:33.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ec007640 con 0x7f20fc100f80 2023-10-06T21:08:33.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec005690 con 0x7f20fc100f80 2023-10-06T21:08:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.584+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc1a5fc0 con 0x7f20fc100f80 2023-10-06T21:08:33.276 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:31 smithi078 ceph-mon[134896]: pgmap v278: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2023-10-06T21:08:33.276 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:32 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2501653882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:08:33.276 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:31 smithi167 ceph-mon[134494]: pgmap v278: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2023-10-06T21:08:33.276 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:32 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2501653882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.586+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc04ebb0 con 0x7f20fc100f80 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.587+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f20ec004750 con 0x7f20fc100f80 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.591+0000 7f20eb7fe700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 0x7f20e40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.591+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f20ec0ccb60 con 0x7f20fc100f80 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.591+0000 7f20f9d9b700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 0x7f20e40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.591+0000 7f20f9d9b700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 0x7f20e40a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009990 tx=0x7f20f0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.594+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f20ec096110 con 0x7f20fc100f80 2023-10-06T21:08:33.278 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:31 smithi026 ceph-mon[98490]: pgmap v278: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2023-10-06T21:08:33.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.859+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f20fc19a4f0 con 0x7f20fc100f80 2023-10-06T21:08:33.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.866+0000 7f20eb7fe700 1 -- 172.21.15.26:0/2501653882 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260407 (secure 0 0 0) 0x7f20ec016020 con 0x7f20fc100f80 2023-10-06T21:08:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.870+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 msgr2=0x7f20e40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.870+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 0x7f20e40a39d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009990 tx=0x7f20f0005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.870+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 msgr2=0x7f20fc199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.870+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d980 tx=0x7f20ec00dc90 comp rx=0 tx=0).stop 2023-10-06T21:08:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 shutdown_connections 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20fc106f90 0x7f20fc19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f20e40a1510 0x7f20e40a39d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100f80 0x7f20fc199190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 --2- 172.21.15.26:0/2501653882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20fc101940 0x7f20fc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 >> 172.21.15.26:0/2501653882 conn(0x7f20fc0780a0 msgr2=0x7f20fc10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 shutdown_connections 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:31.871+0000 7f2100b40700 1 -- 172.21.15.26:0/2501653882 wait complete. 2023-10-06T21:08:33.428 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2830651730},{"type":"v1","addr":"172.21.15.26:6801","nonce":2830651730}]},"active_addr":"172.21.15.26:6801/2830651730","active_change":"2023-10-06T21:01:44.284699+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1157730691}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2246884211}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3325517294}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3867812316}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":282364966}]}]} 2023-10-06T21:08:34.051 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-06T21:08:34.199 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-06T21:08:34.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:08:34.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:32 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2501653882' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-06T21:08:34.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:33 smithi026 ceph-mon[98490]: pgmap v279: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2023-10-06T21:08:34.201 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:33 smithi078 ceph-mon[134896]: pgmap v279: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2023-10-06T21:08:34.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:33 smithi167 ceph-mon[134494]: pgmap v279: 97 pgs: 97 active+clean; 622 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2023-10-06T21:08:34.950 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:35.582 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:35 smithi026 ceph-mon[98490]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-10-06T21:08:35.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:35 smithi167 ceph-mon[134494]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-10-06T21:08:35.900 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:35 smithi078 ceph-mon[134896]: pgmap v280: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-10-06T21:08:36.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 -- 172.21.15.26:0/1816327230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 msgr2=0x7fc57c10ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 --2- 172.21.15.26:0/1816327230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c10ddb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b0d0 tx=0x7fc57400b3e0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 -- 172.21.15.26:0/1816327230 shutdown_connections 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 --2- 172.21.15.26:0/1816327230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c10ddb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 --2- 172.21.15.26:0/1816327230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c106f90 0x7fc57c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 --2- 172.21.15.26:0/1816327230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c100f80 0x7fc57c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 -- 172.21.15.26:0/1816327230 >> 172.21.15.26:0/1816327230 conn(0x7fc57c0780a0 msgr2=0x7fc57c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.429+0000 7fc581a51700 1 -- 172.21.15.26:0/1816327230 shutdown_connections 2023-10-06T21:08:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.430+0000 7fc581a51700 1 -- 172.21.15.26:0/1816327230 wait complete. 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 Processor -- start 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 -- start start 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c106f90 0x7fc57c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c19e210 con 0x7fc57c101940 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c19e380 con 0x7fc57c100f80 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.431+0000 7fc581a51700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c19e540 con 0x7fc57c106f90 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:36.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40742/0 (socket says 172.21.15.26:40742) 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57affd700 1 -- 172.21.15.26:0/2223320577 learned_addr learned my addr 172.21.15.26:0/2223320577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57b7fe700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c106f90 0x7fc57c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 -- 172.21.15.26:0/2223320577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c106f90 msgr2=0x7fc57c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c106f90 0x7fc57c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 -- 172.21.15.26:0/2223320577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 msgr2=0x7fc57c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 -- 172.21.15.26:0/2223320577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574009d00 con 0x7fc57c101940 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57affd700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.432+0000 7fc57a7fc700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c199790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc57000e9f0 tx=0x7fc57000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.433+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000c9c0 con 0x7fc57c101940 2023-10-06T21:08:36.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.433+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c19e820 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.433+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc570010430 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.433+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc570018680 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.433+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c1a6080 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.434+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c04ebb0 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.438+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc570010860 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.439+0000 7fc580a4f700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 0x7fc5640a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.440+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc570014070 con 0x7fc57c101940 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.440+0000 7fc57affd700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 0x7fc5640a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.440+0000 7fc57affd700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 0x7fc5640a39d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc56c011550 tx=0x7fc56c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:36.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.441+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc570096690 con 0x7fc57c101940 2023-10-06T21:08:36.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.651+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc57c110ef0 con 0x7fc57c101940 2023-10-06T21:08:36.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.653+0000 7fc580a4f700 1 -- 172.21.15.26:0/2223320577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22069 (secure 0 0 0) 0x7fc570099cb0 con 0x7fc57c101940 2023-10-06T21:08:36.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:36.697 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:17.510362+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 msgr2=0x7fc5640a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 0x7fc5640a39d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc56c011550 tx=0x7fc56c009260 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 msgr2=0x7fc57c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c199790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc57000e9f0 tx=0x7fc57000ed00 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 shutdown_connections 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc57c106f90 0x7fc57c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c100f80 0x7fc57c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5640a1510 0x7fc5640a39d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 --2- 172.21.15.26:0/2223320577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc57c101940 0x7fc57c199790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 >> 172.21.15.26:0/2223320577 conn(0x7fc57c0780a0 msgr2=0x7fc57c10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 shutdown_connections 2023-10-06T21:08:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:36.656+0000 7fc581a51700 1 -- 172.21.15.26:0/2223320577 wait complete. 2023-10-06T21:08:37.128 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-06T21:08:37.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-06T21:08:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:37 smithi167 ceph-mon[134494]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-10-06T21:08:37.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:37 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2223320577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:37 smithi026 ceph-mon[98490]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-10-06T21:08:37.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:37 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2223320577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:37.902 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:37 smithi078 ceph-mon[134896]: pgmap v281: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-10-06T21:08:37.902 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:37 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2223320577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:38 smithi078 ceph-mon[134896]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-10-06T21:08:39.225 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:38 smithi026 ceph-mon[98490]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-10-06T21:08:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:38 smithi167 ceph-mon[134494]: pgmap v282: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-10-06T21:08:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 -- 172.21.15.26:0/4016032088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 msgr2=0x7f99f0107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/4016032088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f0107db0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009990 tx=0x7f99e0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 -- 172.21.15.26:0/4016032088 shutdown_connections 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/4016032088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 0x7f99f0110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/4016032088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/4016032088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f0107db0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 -- 172.21.15.26:0/4016032088 >> 172.21.15.26:0/4016032088 conn(0x7f99f00fec20 msgr2=0x7f99f0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 -- 172.21.15.26:0/4016032088 shutdown_connections 2023-10-06T21:08:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.830+0000 7f99f7c2b700 1 -- 172.21.15.26:0/4016032088 wait complete. 2023-10-06T21:08:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.831+0000 7f99f7c2b700 1 Processor -- start 2023-10-06T21:08:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.831+0000 7f99f7c2b700 1 -- start start 2023-10-06T21:08:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:39.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 0x7f99f019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:39.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e250 con 0x7f99f0103d60 2023-10-06T21:08:39.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e3c0 con 0x7f99f0108490 2023-10-06T21:08:39.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f7c2b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e580 con 0x7f99f01033b0 2023-10-06T21:08:39.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f51c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49380/0 (socket says 172.21.15.26:49380) 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 -- 172.21.15.26:0/1046197117 learned_addr learned my addr 172.21.15.26:0/1046197117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f61c8700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 0x7f99f019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 -- 172.21.15.26:0/1046197117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 msgr2=0x7f99f019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 0x7f99f019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 -- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 msgr2=0x7f99f01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f59c7700 1 -- 172.21.15.26:0/1046197117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00095f0 con 0x7f99f01033b0 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.832+0000 7f99f51c6700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99f59c7700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e9a0 tx=0x7f99ec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec009790 con 0x7f99f01033b0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f019e860 con 0x7f99f01033b0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ec010430 con 0x7f99f01033b0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec018680 con 0x7f99f01033b0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.833+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01a60c0 con 0x7f99f01033b0 2023-10-06T21:08:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.834+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f004ebb0 con 0x7f99f01033b0 2023-10-06T21:08:39.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.837+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f99ec0105a0 con 0x7f99f01033b0 2023-10-06T21:08:39.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.838+0000 7f99e6ffd700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 0x7f99dc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:39.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.838+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99ec014070 con 0x7f99f01033b0 2023-10-06T21:08:39.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.838+0000 7f99f51c6700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 0x7f99dc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:39.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.839+0000 7f99f51c6700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 0x7f99dc0a39d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99f019a8b0 tx=0x7f99e0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:39.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:39.840+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f99ec096770 con 0x7f99f01033b0 2023-10-06T21:08:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.056+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99f019a5f0 con 0x7f99f01033b0 2023-10-06T21:08:40.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.058+0000 7f99e6ffd700 1 -- 172.21.15.26:0/1046197117 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22069 (secure 0 0 0) 0x7f99ec026090 con 0x7f99f01033b0 2023-10-06T21:08:40.060 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:08:40.060 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:17.510362+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 msgr2=0x7f99dc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 0x7f99dc0a39d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99f019a8b0 tx=0x7f99e0005fd0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 msgr2=0x7f99f0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e9a0 tx=0x7f99ec00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 shutdown_connections 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f0108490 0x7f99f019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f01033b0 0x7f99f0199290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99dc0a1510 0x7f99dc0a39d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 --2- 172.21.15.26:0/1046197117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f0103d60 0x7f99f01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:40.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 >> 172.21.15.26:0/1046197117 conn(0x7f99f00fec20 msgr2=0x7f99f00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 shutdown_connections 2023-10-06T21:08:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:40.060+0000 7f99f7c2b700 1 -- 172.21.15.26:0/1046197117 wait complete. 2023-10-06T21:08:40.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:40 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1046197117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:40.502 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:40 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1046197117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:40.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:40 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1046197117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:08:40.590 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-06T21:08:40.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-06T21:08:40.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-06T21:08:40.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-06T21:08:40.592 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-06T21:08:40.592 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-06T21:08:40.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-06T21:08:40.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-06T21:08:40.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-06T21:08:40.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-06T21:08:40.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-06T21:08:40.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-06T21:08:41.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:41 smithi167 ceph-mon[134494]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:08:41.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:41 smithi026 ceph-mon[98490]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:08:41.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:41 smithi078 ceph-mon[134896]: pgmap v283: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:08:42.153 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:42.393 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:42.627 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:42.629 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:43.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:43 smithi167 ceph-mon[134494]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-10-06T21:08:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:43 smithi026 ceph-mon[98490]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-10-06T21:08:43.854 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:43.855 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:43.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:43 smithi078 ceph-mon[134896]: pgmap v284: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-10-06T21:08:44.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-10-06T21:08:44.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-10-06T21:08:44.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:44 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-06T21:08:44.582 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-10-06T21:08:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:44 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-10-06T21:08:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-10-06T21:08:44.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:44 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-10-06T21:08:45.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 -- 172.21.15.26:0/2317133133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0101940 msgr2=0x7f07a010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2317133133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0101940 0x7f07a010ddb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f079800b0d0 tx=0x7f079800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 -- 172.21.15.26:0/2317133133 shutdown_connections 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2317133133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0101940 0x7f07a010ddb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2317133133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0106f90 0x7f07a0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2317133133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a0100f80 0x7f07a0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 -- 172.21.15.26:0/2317133133 >> 172.21.15.26:0/2317133133 conn(0x7f07a00780a0 msgr2=0x7f07a00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 -- 172.21.15.26:0/2317133133 shutdown_connections 2023-10-06T21:08:45.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.098+0000 7f07a7d28700 1 -- 172.21.15.26:0/2317133133 wait complete. 2023-10-06T21:08:45.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.099+0000 7f07a7d28700 1 Processor -- start 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 -- start start 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0100f80 0x7f07a01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a0101940 0x7f07a0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 0x7f07a019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a019e2a0 con 0x7f07a0106f90 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a019e410 con 0x7f07a0101940 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.100+0000 7f07a7d28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a019e5d0 con 0x7f07a0100f80 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 0x7f07a019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 0x7f07a019dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51852/0 (socket says 172.21.15.26:51852) 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 -- 172.21.15.26:0/2041778650 learned_addr learned my addr 172.21.15.26:0/2041778650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:45.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a52c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a0101940 0x7f07a0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a5ac4700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0100f80 0x7f07a01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 -- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0100f80 msgr2=0x7f07a01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07a0100f80 0x7f07a01992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 -- 172.21.15.26:0/2041778650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a0101940 msgr2=0x7f07a0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a0101940 0x7f07a0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798009d00 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.101+0000 7f07a62c5700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 0x7f07a019dc90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0798007870 tx=0x7f07980078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.102+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800e040 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.102+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a019e850 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.102+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0798005070 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.102+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079801b6c0 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.102+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01a6110 con 0x7f07a0106f90 2023-10-06T21:08:45.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.103+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07a0111150 con 0x7f07a0106f90 2023-10-06T21:08:45.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.107+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f07980051e0 con 0x7f07a0106f90 2023-10-06T21:08:45.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.108+0000 7f0796ffd700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f078c0a1510 0x7f078c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.108+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f07980ce6c0 con 0x7f07a0106f90 2023-10-06T21:08:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.108+0000 7f07a5ac4700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f078c0a1510 0x7f078c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.109+0000 7f0796ffd700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] conn(0x7f078c0ab0d0 0x7f078c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.109+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f078c0adc40 con 0x7f078c0ab0d0 2023-10-06T21:08:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.109+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07980cea30 con 0x7f07a0106f90 2023-10-06T21:08:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.109+0000 7f07a52c3700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] conn(0x7f078c0ab0d0 0x7f078c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.109+0000 7f07a5ac4700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f078c0a1510 0x7f078c0a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f079c009be0 tx=0x7f079c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.110+0000 7f07a52c3700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] conn(0x7f078c0ab0d0 0x7f078c0ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:45.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.110+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== osd.2 v2:172.21.15.167:6800/2625559471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f078c0adc40 con 0x7f078c0ab0d0 2023-10-06T21:08:45.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.128+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 --> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f07a004ebb0 con 0x7f078c0ab0d0 2023-10-06T21:08:45.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.133+0000 7f0796ffd700 1 -- 172.21.15.26:0/2041778650 <== osd.2 v2:172.21.15.167:6800/2625559471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f07a004ebb0 con 0x7f078c0ab0d0 2023-10-06T21:08:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] conn(0x7f078c0ab0d0 msgr2=0x7f078c0ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] conn(0x7f078c0ab0d0 0x7f078c0ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f078c0a1510 msgr2=0x7f078c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f078c0a1510 0x7f078c0a39d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f079c009be0 tx=0x7f079c009260 comp rx=0 tx=0).stop 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 msgr2=0x7f07a019dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 --2- 172.21.15.26:0/2041778650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07a0106f90 0x7f07a019dc90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0798007870 tx=0x7f07980078a0 comp rx=0 tx=0).stop 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a62c5700 1 -- 172.21.15.26:0/2041778650 reap_dead start 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 shutdown_connections 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 >> 172.21.15.26:0/2041778650 conn(0x7f07a00780a0 msgr2=0x7f07a010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 shutdown_connections 2023-10-06T21:08:45.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:45.134+0000 7f07a7d28700 1 -- 172.21.15.26:0/2041778650 wait complete. 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: pgmap v285: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-10-06T21:08:45.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-10-06T21:08:45.694 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-10-06T21:08:45.694 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-06T21:08:45.694 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:45 smithi026 ceph-mon[98490]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:08:45.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: pgmap v285: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-06T21:08:45.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:45 smithi167 ceph-mon[134494]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:08:45.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: pgmap v285: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-10-06T21:08:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:45 smithi078 ceph-mon[134896]: osdmap e86: 12 total, 12 up, 12 in 2023-10-06T21:08:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:46.620 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:46.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:46 smithi167 ceph-mon[134494]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:08:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.808+0000 7f1abb9e2700 1 -- 172.21.15.26:0/3637763082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 msgr2=0x7f1ab4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.808+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/3637763082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4101400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40099c0 tx=0x7f1aa4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 -- 172.21.15.26:0/3637763082 shutdown_connections 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/3637763082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/3637763082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab4106f90 0x7f1ab4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/3637763082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4101400 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 -- 172.21.15.26:0/3637763082 >> 172.21.15.26:0/3637763082 conn(0x7f1ab40780a0 msgr2=0x7f1ab40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 -- 172.21.15.26:0/3637763082 shutdown_connections 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.810+0000 7f1abb9e2700 1 -- 172.21.15.26:0/3637763082 wait complete. 2023-10-06T21:08:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.811+0000 7f1abb9e2700 1 Processor -- start 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.811+0000 7f1abb9e2700 1 -- start start 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab4197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab4106f90 0x7f1ab419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419bf80 con 0x7f1ab4100f80 2023-10-06T21:08:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419c0f0 con 0x7f1ab4101940 2023-10-06T21:08:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1abb9e2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419c2b0 con 0x7f1ab4106f90 2023-10-06T21:08:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab977e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab4197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab4197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39590/0 (socket says 172.21.15.26:39590) 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab977e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51868/0 (socket says 172.21.15.26:51868) 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 -- 172.21.15.26:0/1334415834 learned_addr learned my addr 172.21.15.26:0/1334415834 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab9f7f700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab4106f90 0x7f1ab419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 -- 172.21.15.26:0/1334415834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab4106f90 msgr2=0x7f1ab419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab4106f90 0x7f1ab419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 -- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 msgr2=0x7f1ab4197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab4100f80 0x7f1ab4197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.812+0000 7f1ab8f7d700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40095f0 con 0x7f1ab4101940 2023-10-06T21:08:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1ab8f7d700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab4197550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401d070 con 0x7f1ab4101940 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa4005050 con 0x7f1ab4101940 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab419c530 con 0x7f1ab4101940 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4004b70 con 0x7f1ab4101940 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.813+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab41a3df0 con 0x7f1ab4101940 2023-10-06T21:08:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.814+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ab404ebb0 con 0x7f1ab4101940 2023-10-06T21:08:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.816+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1aa40051c0 con 0x7f1ab4101940 2023-10-06T21:08:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1aaa7fc700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1aa00a1590 0x7f1aa00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1aa40cddd0 con 0x7f1ab4101940 2023-10-06T21:08:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1aaa7fc700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] conn(0x7f1aa00ab150 0x7f1aa00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa00adcc0 con 0x7f1aa00ab150 2023-10-06T21:08:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa40ce140 con 0x7f1ab4101940 2023-10-06T21:08:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.818+0000 7f1ab977e700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1aa00a1590 0x7f1aa00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.819+0000 7f1ab9f7f700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] conn(0x7f1aa00ab150 0x7f1aa00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.819+0000 7f1ab977e700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1aa00a1590 0x7f1aa00a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009be0 tx=0x7f1ab0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.819+0000 7f1ab9f7f700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] conn(0x7f1aa00ab150 0x7f1aa00ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.820+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== osd.4 v2:172.21.15.78:6808/3086176217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1aa00adcc0 con 0x7f1aa00ab150 2023-10-06T21:08:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:46 smithi026 ceph-mon[98490]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:08:46.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.837+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 --> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ab4111150 con 0x7f1aa00ab150 2023-10-06T21:08:46.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.838+0000 7f1aaa7fc700 1 -- 172.21.15.26:0/1334415834 <== osd.4 v2:172.21.15.78:6808/3086176217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1aac00f040 con 0x7f1aa00ab150 2023-10-06T21:08:46.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.838+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] conn(0x7f1aa00ab150 msgr2=0x7f1aa00ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.838+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] conn(0x7f1aa00ab150 0x7f1aa00ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1aa00a1590 msgr2=0x7f1aa00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1aa00a1590 0x7f1aa00a3a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009be0 tx=0x7f1ab0009260 comp rx=0 tx=0).stop 2023-10-06T21:08:46.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 msgr2=0x7f1ab4197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:46.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 --2- 172.21.15.26:0/1334415834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab4101940 0x7f1ab4197550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa400bfd0 comp rx=0 tx=0).stop 2023-10-06T21:08:46.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1ab9f7f700 1 -- 172.21.15.26:0/1334415834 reap_dead start 2023-10-06T21:08:46.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 shutdown_connections 2023-10-06T21:08:46.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 >> 172.21.15.26:0/1334415834 conn(0x7f1ab40780a0 msgr2=0x7f1ab40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:46.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 shutdown_connections 2023-10-06T21:08:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:46.839+0000 7f1abb9e2700 1 -- 172.21.15.26:0/1334415834 wait complete. 2023-10-06T21:08:46.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:46 smithi078 ceph-mon[134896]: osdmap e87: 12 total, 12 up, 12 in 2023-10-06T21:08:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:77309411387 2023-10-06T21:08:46.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-06T21:08:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:47.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:47 smithi026 ceph-mon[98490]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2023-10-06T21:08:47.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:47 smithi167 ceph-mon[134494]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2023-10-06T21:08:47.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:47 smithi078 ceph-mon[134896]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s 2023-10-06T21:08:47.993 INFO:teuthology.orchestra.run.smithi026.stdout:141733920818 2023-10-06T21:08:47.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-06T21:08:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 -- 172.21.15.26:0/1240495214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800689a0 msgr2=0x7f4f80068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 --2- 172.21.15.26:0/1240495214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800689a0 0x7f4f80068d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009990 tx=0x7f4f70009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 -- 172.21.15.26:0/1240495214 shutdown_connections 2023-10-06T21:08:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 --2- 172.21.15.26:0/1240495214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f8010da10 0x7f4f8010fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 --2- 172.21.15.26:0/1240495214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800689a0 0x7f4f80068d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 --2- 172.21.15.26:0/1240495214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f800692c0 0x7f4f8010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 -- 172.21.15.26:0/1240495214 >> 172.21.15.26:0/1240495214 conn(0x7f4f80075e00 msgr2=0x7f4f80076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 -- 172.21.15.26:0/1240495214 shutdown_connections 2023-10-06T21:08:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.388+0000 7f4f85312700 1 -- 172.21.15.26:0/1240495214 wait complete. 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.389+0000 7f4f85312700 1 Processor -- start 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 -- start start 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f800689a0 0x7f4f80199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f800692c0 0x7f4f80199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f8010da10 0x7f4f8019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019e1d0 con 0x7f4f8010da10 2023-10-06T21:08:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019e340 con 0x7f4f800689a0 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f85312700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019e500 con 0x7f4f800692c0 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f7effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f800689a0 0x7f4f80199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f7effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f800689a0 0x7f4f80199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39604/0 (socket says 172.21.15.26:39604) 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f7effd700 1 -- 172.21.15.26:0/691993883 learned_addr learned my addr 172.21.15.26:0/691993883 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f7f7fe700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f8010da10 0x7f4f8019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:48.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.390+0000 7f4f7e7fc700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f800692c0 0x7f4f80199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 -- 172.21.15.26:0/691993883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f800692c0 msgr2=0x7f4f80199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f800692c0 0x7f4f80199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 -- 172.21.15.26:0/691993883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f800689a0 msgr2=0x7f4f80199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f800689a0 0x7f4f80199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700095f0 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f7f7fe700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f8010da10 0x7f4f8019dbc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800ef90 tx=0x7f4f7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78018070 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f7800f040 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8019e780 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.391+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800cc30 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.392+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801a6040 con 0x7f4f8010da10 2023-10-06T21:08:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.392+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f801133b0 con 0x7f4f8010da10 2023-10-06T21:08:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.394+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4f78003a00 con 0x7f4f8010da10 2023-10-06T21:08:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f67fff700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f680a1590 0x7f4f680a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4f780cd2a0 con 0x7f4f8010da10 2023-10-06T21:08:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f7effd700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f680a1590 0x7f4f680a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f67fff700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] conn(0x7f4f680ab150 0x7f4f680ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f680adcc0 con 0x7f4f680ab150 2023-10-06T21:08:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.396+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f780cd610 con 0x7f4f8010da10 2023-10-06T21:08:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.397+0000 7f4f7e7fc700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] conn(0x7f4f680ab150 0x7f4f680ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.397+0000 7f4f7effd700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f680a1590 0x7f4f680a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006010 tx=0x7f4f7000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.397+0000 7f4f7e7fc700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] conn(0x7f4f680ab150 0x7f4f680ad5f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.399+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== osd.0 v2:172.21.15.26:6802/3569177052 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4f680adcc0 con 0x7f4f680ab150 2023-10-06T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.420+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 --> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4f8004ebb0 con 0x7f4f680ab150 2023-10-06T21:08:48.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f67fff700 1 -- 172.21.15.26:0/691993883 <== osd.0 v2:172.21.15.26:6802/3569177052 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4f8004ebb0 con 0x7f4f680ab150 2023-10-06T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] conn(0x7f4f680ab150 msgr2=0x7f4f680ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] conn(0x7f4f680ab150 0x7f4f680ad5f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f680a1590 msgr2=0x7f4f680a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4f680a1590 0x7f4f680a3a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006010 tx=0x7f4f7000b260 comp rx=0 tx=0).stop 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f8010da10 msgr2=0x7f4f8019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f85312700 1 --2- 172.21.15.26:0/691993883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f8010da10 0x7f4f8019dbc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800ef90 tx=0x7f4f7800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.421+0000 7f4f7f7fe700 1 -- 172.21.15.26:0/691993883 reap_dead start 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.422+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 shutdown_connections 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.422+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 >> 172.21.15.26:0/691993883 conn(0x7f4f80075e00 msgr2=0x7f4f8010eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.422+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 shutdown_connections 2023-10-06T21:08:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:48.422+0000 7f4f85312700 1 -- 172.21.15.26:0/691993883 wait complete. 2023-10-06T21:08:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:48 smithi078 ceph-mon[134896]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:49.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:48 smithi167 ceph-mon[134494]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:49.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:48 smithi026 ceph-mon[98490]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:49.719 INFO:teuthology.orchestra.run.smithi026.stdout:34359738435 2023-10-06T21:08:49.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 -- 172.21.15.26:0/811599601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 msgr2=0x7ff95c10d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 --2- 172.21.15.26:0/811599601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 0x7ff95c10d4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff950009990 tx=0x7ff950009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 -- 172.21.15.26:0/811599601 shutdown_connections 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 --2- 172.21.15.26:0/811599601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 0x7ff95c10fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 --2- 172.21.15.26:0/811599601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 --2- 172.21.15.26:0/811599601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 0x7ff95c10d4d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 -- 172.21.15.26:0/811599601 >> 172.21.15.26:0/811599601 conn(0x7ff95c075e00 msgr2=0x7ff95c076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 -- 172.21.15.26:0/811599601 shutdown_connections 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.419+0000 7ff961a28700 1 -- 172.21.15.26:0/811599601 wait complete. 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.420+0000 7ff961a28700 1 Processor -- start 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 -- start start 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 0x7ff95c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 0x7ff95c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c19e1e0 con 0x7ff95c0689a0 2023-10-06T21:08:50.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c19e350 con 0x7ff95c10da10 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.421+0000 7ff961a28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c19e510 con 0x7ff95c0692c0 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51910/0 (socket says 172.21.15.26:51910) 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95affd700 1 -- 172.21.15.26:0/3736942484 learned_addr learned my addr 172.21.15.26:0/3736942484 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 0x7ff95c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95a7fc700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 0x7ff95c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 -- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 msgr2=0x7ff95c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff95c0692c0 0x7ff95c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 -- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 msgr2=0x7ff95c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95b7fe700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009520 con 0x7ff95c10da10 2023-10-06T21:08:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.422+0000 7ff95affd700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c0689a0 0x7ff95c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff95b7fe700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 0x7ff95c19dbd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ef90 tx=0x7ff95400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954018070 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95400f040 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400cc30 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9500095f0 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.423+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c1a5e80 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.424+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff95c113390 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.428+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff954003a00 con 0x7ff95c10da10 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.430+0000 7ff960a26700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9440a1590 0x7ff9440a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9540cd2a0 con 0x7ff95c10da10 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff95affd700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9440a1590 0x7ff9440a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff960a26700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] conn(0x7ff9440ab150 0x7ff9440ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9440adcc0 con 0x7ff9440ab150 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9540cd610 con 0x7ff95c10da10 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.431+0000 7ff95a7fc700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] conn(0x7ff9440ab150 0x7ff9440ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.432+0000 7ff95affd700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9440a1590 0x7ff9440a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009fd0 tx=0x7ff94c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.432+0000 7ff95a7fc700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] conn(0x7ff9440ab150 0x7ff9440ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.433+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== osd.8 v2:172.21.15.167:6816/263711994 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff9440adcc0 con 0x7ff9440ab150 2023-10-06T21:08:50.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.450+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 --> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff95c04ebb0 con 0x7ff9440ab150 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff960a26700 1 -- 172.21.15.26:0/3736942484 <== osd.8 v2:172.21.15.167:6816/263711994 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff95c04ebb0 con 0x7ff9440ab150 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] conn(0x7ff9440ab150 msgr2=0x7ff9440ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] conn(0x7ff9440ab150 0x7ff9440ad5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9440a1590 msgr2=0x7ff9440a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff9440a1590 0x7ff9440a3a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009fd0 tx=0x7ff94c009260 comp rx=0 tx=0).stop 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 msgr2=0x7ff95c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 --2- 172.21.15.26:0/3736942484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff95c10da10 0x7ff95c19dbd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ef90 tx=0x7ff95400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff95b7fe700 1 -- 172.21.15.26:0/3736942484 reap_dead start 2023-10-06T21:08:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 shutdown_connections 2023-10-06T21:08:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 >> 172.21.15.26:0/3736942484 conn(0x7ff95c075e00 msgr2=0x7ff95c10eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 shutdown_connections 2023-10-06T21:08:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.451+0000 7ff961a28700 1 -- 172.21.15.26:0/3736942484 wait complete. 2023-10-06T21:08:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 -- 172.21.15.26:0/3644155246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 msgr2=0x7f1680101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 --2- 172.21.15.26:0/3644155246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f1680101400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16740099c0 tx=0x7f1674009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 -- 172.21.15.26:0/3644155246 shutdown_connections 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 --2- 172.21.15.26:0/3644155246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680101940 0x7f168010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 --2- 172.21.15.26:0/3644155246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1680106f90 0x7f1680107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 --2- 172.21.15.26:0/3644155246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f1680101400 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 -- 172.21.15.26:0/3644155246 >> 172.21.15.26:0/3644155246 conn(0x7f16800780a0 msgr2=0x7f16800784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 -- 172.21.15.26:0/3644155246 shutdown_connections 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.167+0000 7f16856f7700 1 -- 172.21.15.26:0/3644155246 wait complete. 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.168+0000 7f16856f7700 1 Processor -- start 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.168+0000 7f16856f7700 1 -- start start 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f16801991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1680101940 0x7f1680199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680106f90 0x7f168019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168019e190 con 0x7f1680100f80 2023-10-06T21:08:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168019e300 con 0x7f1680101940 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f16856f7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168019e4c0 con 0x7f1680106f90 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f167effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f16801991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f167effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f16801991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51896/0 (socket says 172.21.15.26:51896) 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f167f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680106f90 0x7f168019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f167effd700 1 -- 172.21.15.26:0/3779848856 learned_addr learned my addr 172.21.15.26:0/3779848856 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.169+0000 7f167e7fc700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1680101940 0x7f1680199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 -- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1680101940 msgr2=0x7f1680199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1680101940 0x7f1680199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 -- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 msgr2=0x7f16801991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f16801991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740095f0 con 0x7f1680106f90 2023-10-06T21:08:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167effd700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1680100f80 0x7f16801991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f167f7fe700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680106f90 0x7f168019db80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f167800efc0 tx=0x7f167800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678018070 con 0x7f1680106f90 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168019e740 con 0x7f1680106f90 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f167800f040 con 0x7f1680106f90 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167800cc80 con 0x7f1680106f90 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.170+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16801a6000 con 0x7f1680106f90 2023-10-06T21:08:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.171+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1680111190 con 0x7f1680106f90 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.174+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1678003a00 con 0x7f1680106f90 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f1667fff700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f16680a1510 0x7f16680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f16780cd320 con 0x7f1680106f90 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f167effd700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f16680a1510 0x7f16680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f1667fff700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] conn(0x7f16680ab0d0 0x7f16680ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16680adc40 con 0x7f16680ab0d0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.175+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16780cd690 con 0x7f1680106f90 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.176+0000 7f167effd700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f16680a1510 0x7f16680a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1670011550 tx=0x7f1670009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.176+0000 7f167e7fc700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] conn(0x7f16680ab0d0 0x7f16680ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.176+0000 7f167e7fc700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] conn(0x7f16680ab0d0 0x7f16680ad570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.177+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== osd.7 v2:172.21.15.78:6816/2278627403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f16680adc40 con 0x7f16680ab0d0 2023-10-06T21:08:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.194+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 --> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f168004ebb0 con 0x7f16680ab0d0 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.195+0000 7f1667fff700 1 -- 172.21.15.26:0/3779848856 <== osd.7 v2:172.21.15.78:6816/2278627403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f168004ebb0 con 0x7f16680ab0d0 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.195+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] conn(0x7f16680ab0d0 msgr2=0x7f16680ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.195+0000 7f16856f7700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] conn(0x7f16680ab0d0 0x7f16680ad570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f16680a1510 msgr2=0x7f16680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f16680a1510 0x7f16680a39d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1670011550 tx=0x7f1670009260 comp rx=0 tx=0).stop 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680106f90 msgr2=0x7f168019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 --2- 172.21.15.26:0/3779848856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1680106f90 0x7f168019db80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f167800efc0 tx=0x7f167800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f167f7fe700 1 -- 172.21.15.26:0/3779848856 reap_dead start 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 shutdown_connections 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 >> 172.21.15.26:0/3779848856 conn(0x7f16800780a0 msgr2=0x7f168010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 shutdown_connections 2023-10-06T21:08:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:50.196+0000 7f16856f7700 1 -- 172.21.15.26:0/3779848856 wait complete. 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 -- 172.21.15.26:0/1427425146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 msgr2=0x7fa210107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1427425146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa210107370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa2040099c0 tx=0x7fa204009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 -- 172.21.15.26:0/1427425146 shutdown_connections 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1427425146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa210101940 0x7fa21010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1427425146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa210107370 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1427425146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 0x7fa210101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 -- 172.21.15.26:0/1427425146 >> 172.21.15.26:0/1427425146 conn(0x7fa2100780a0 msgr2=0x7fa2100784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 -- 172.21.15.26:0/1427425146 shutdown_connections 2023-10-06T21:08:51.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.060+0000 7fa2170c7700 1 -- 172.21.15.26:0/1427425146 wait complete. 2023-10-06T21:08:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.061+0000 7fa2170c7700 1 Processor -- start 2023-10-06T21:08:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.061+0000 7fa2170c7700 1 -- start start 2023-10-06T21:08:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.061+0000 7fa2170c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 0x7fa210199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa2170c7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa210101940 0x7fa2101996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa2170c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa21019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa2170c7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21019e130 con 0x7fa210106f90 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa2170c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21019e2a0 con 0x7fa210100f80 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa2170c7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21019e460 con 0x7fa210101940 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa214e63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 0x7fa210199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa214e63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 0x7fa210199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44198/0 (socket says 172.21.15.26:44198) 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa214e63700 1 -- 172.21.15.26:0/1758768519 learned_addr learned my addr 172.21.15.26:0/1758768519 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa21019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa20ffff700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa210101940 0x7fa2101996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 -- 172.21.15.26:0/1758768519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa210101940 msgr2=0x7fa2101996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa210101940 0x7fa2101996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 -- 172.21.15.26:0/1758768519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 msgr2=0x7fa210199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa210100f80 0x7fa210199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.062+0000 7fa215664700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040095f0 con 0x7fa210106f90 2023-10-06T21:08:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa215664700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa21019db20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208018070 con 0x7fa210106f90 2023-10-06T21:08:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21019e6e0 con 0x7fa210106f90 2023-10-06T21:08:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20800f040 con 0x7fa210106f90 2023-10-06T21:08:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20800cc80 con 0x7fa210106f90 2023-10-06T21:08:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.063+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2101a5fa0 con 0x7fa210106f90 2023-10-06T21:08:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.064+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2101111a0 con 0x7fa210106f90 2023-10-06T21:08:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.066+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa208003a00 con 0x7fa210106f90 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.067+0000 7fa20dffb700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa2000a1590 0x7fa2000a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2080cd2f0 con 0x7fa210106f90 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa214e63700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa2000a1590 0x7fa2000a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa20dffb700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] conn(0x7fa2000ab150 0x7fa2000ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2000adcc0 con 0x7fa2000ab150 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2080cd660 con 0x7fa210106f90 2023-10-06T21:08:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa20ffff700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] conn(0x7fa2000ab150 0x7fa2000ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.068+0000 7fa214e63700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa2000a1590 0x7fa2000a3a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2040036c0 tx=0x7fa204005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.069+0000 7fa20ffff700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] conn(0x7fa2000ab150 0x7fa2000ad5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.070+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== osd.6 v2:172.21.15.26:6818/1485072774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa2000adcc0 con 0x7fa2000ab150 2023-10-06T21:08:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.087+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 --> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa21004ebb0 con 0x7fa2000ab150 2023-10-06T21:08:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.087+0000 7fa20dffb700 1 -- 172.21.15.26:0/1758768519 <== osd.6 v2:172.21.15.26:6818/1485072774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa21004ebb0 con 0x7fa2000ab150 2023-10-06T21:08:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] conn(0x7fa2000ab150 msgr2=0x7fa2000ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] conn(0x7fa2000ab150 0x7fa2000ad5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa2000a1590 msgr2=0x7fa2000a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa2000a1590 0x7fa2000a3a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2040036c0 tx=0x7fa204005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 msgr2=0x7fa21019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 --2- 172.21.15.26:0/1758768519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa210106f90 0x7fa21019db20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa215664700 1 -- 172.21.15.26:0/1758768519 reap_dead start 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 shutdown_connections 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 >> 172.21.15.26:0/1758768519 conn(0x7fa2100780a0 msgr2=0x7fa21010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 shutdown_connections 2023-10-06T21:08:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:51.088+0000 7fa2170c7700 1 -- 172.21.15.26:0/1758768519 wait complete. 2023-10-06T21:08:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:51 smithi026 ceph-mon[98490]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:51.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:51 smithi167 ceph-mon[134494]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:51.802 INFO:teuthology.orchestra.run.smithi026.stdout:253403070496 2023-10-06T21:08:51.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-06T21:08:51.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:51 smithi078 ceph-mon[134896]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 5 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:52.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 -- 172.21.15.26:0/677324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8100f80 msgr2=0x7f96e8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 --2- 172.21.15.26:0/677324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8100f80 0x7f96e8101400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009920 tx=0x7f96d8009c30 comp rx=0 tx=0).stop 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 -- 172.21.15.26:0/677324510 shutdown_connections 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 --2- 172.21.15.26:0/677324510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8101940 0x7f96e810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 --2- 172.21.15.26:0/677324510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 0x7f96e8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 --2- 172.21.15.26:0/677324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8100f80 0x7f96e8101400 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 -- 172.21.15.26:0/677324510 >> 172.21.15.26:0/677324510 conn(0x7f96e80780a0 msgr2=0x7f96e80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 -- 172.21.15.26:0/677324510 shutdown_connections 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.508+0000 7f96ef59a700 1 -- 172.21.15.26:0/677324510 wait complete. 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.509+0000 7f96ef59a700 1 Processor -- start 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 -- start start 2023-10-06T21:08:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8100f80 0x7f96e8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 0x7f96e8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 0x7f96e819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819e1d0 con 0x7f96e8101940 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819e340 con 0x7f96e8100f80 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ef59a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819e500 con 0x7f96e8106f90 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ecb35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 0x7f96e8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ecb35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 0x7f96e8199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40744/0 (socket says 172.21.15.26:40744) 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96edb37700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 0x7f96e819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.510+0000 7f96ecb35700 1 -- 172.21.15.26:0/210308157 learned_addr learned my addr 172.21.15.26:0/210308157 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ed336700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8100f80 0x7f96e8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 -- 172.21.15.26:0/210308157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 msgr2=0x7f96e819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 0x7f96e819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 -- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8100f80 msgr2=0x7f96e8199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8100f80 0x7f96e8199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8009520 con 0x7f96e8101940 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96edb37700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96e8106f90 0x7f96e819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ed336700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e8100f80 0x7f96e8199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.511+0000 7f96ecb35700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 0x7f96e8199750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009840 tx=0x7f96d800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.512+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d801c070 con 0x7f96e8101940 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.512+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d800bb20 con 0x7f96e8101940 2023-10-06T21:08:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.512+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e819e780 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.512+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d80038e0 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.512+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e81a5e70 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.513+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96e81111a0 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.515+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f96d8020030 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.516+0000 7f96de7fc700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96d40a1510 0x7f96d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96d80ce450 con 0x7f96e8101940 2023-10-06T21:08:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96ed336700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96d40a1510 0x7f96d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96de7fc700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] conn(0x7f96d40ab0d0 0x7f96d40ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96d40adc40 con 0x7f96d40ab0d0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d80ce840 con 0x7f96e8101940 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.517+0000 7f96edb37700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] conn(0x7f96d40ab0d0 0x7f96d40ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.518+0000 7f96ed336700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96d40a1510 0x7f96d40a39d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96e4011490 tx=0x7f96e4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.518+0000 7f96edb37700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] conn(0x7f96d40ab0d0 0x7f96d40ad570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.519+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== osd.1 v2:172.21.15.78:6800/4119914658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f96d40adc40 con 0x7f96d40ab0d0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.537+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 --> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96e804ebb0 con 0x7f96d40ab0d0 2023-10-06T21:08:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.537+0000 7f96de7fc700 1 -- 172.21.15.26:0/210308157 <== osd.1 v2:172.21.15.78:6800/4119914658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96e804ebb0 con 0x7f96d40ab0d0 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] conn(0x7f96d40ab0d0 msgr2=0x7f96d40ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] conn(0x7f96d40ab0d0 0x7f96d40ad570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96d40a1510 msgr2=0x7f96d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96d40a1510 0x7f96d40a39d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96e4011490 tx=0x7f96e4009260 comp rx=0 tx=0).stop 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 msgr2=0x7f96e8199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 --2- 172.21.15.26:0/210308157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e8101940 0x7f96e8199750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009840 tx=0x7f96d800fa00 comp rx=0 tx=0).stop 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96edb37700 1 -- 172.21.15.26:0/210308157 reap_dead start 2023-10-06T21:08:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 shutdown_connections 2023-10-06T21:08:52.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 >> 172.21.15.26:0/210308157 conn(0x7f96e80780a0 msgr2=0x7f96e80ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:52.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 shutdown_connections 2023-10-06T21:08:52.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:52.538+0000 7f96ef59a700 1 -- 172.21.15.26:0/210308157 wait complete. 2023-10-06T21:08:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:219043332133 2023-10-06T21:08:53.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-06T21:08:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:197568495656 2023-10-06T21:08:53.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-06T21:08:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 -- 172.21.15.26:0/3546013491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 msgr2=0x7fb93c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 --2- 172.21.15.26:0/3546013491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c1013b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009990 tx=0x7fb92c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 -- 172.21.15.26:0/3546013491 shutdown_connections 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 --2- 172.21.15.26:0/3546013491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c1060b0 0x7fb93c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 --2- 172.21.15.26:0/3546013491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c1013b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 --2- 172.21.15.26:0/3546013491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 0x7fb93c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 -- 172.21.15.26:0/3546013491 >> 172.21.15.26:0/3546013491 conn(0x7fb93c0fc860 msgr2=0x7fb93c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 -- 172.21.15.26:0/3546013491 shutdown_connections 2023-10-06T21:08:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.565+0000 7fb93bfff700 1 -- 172.21.15.26:0/3546013491 wait complete. 2023-10-06T21:08:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.567+0000 7fb93bfff700 1 Processor -- start 2023-10-06T21:08:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.567+0000 7fb93bfff700 1 -- start start 2023-10-06T21:08:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 0x7fb93c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c1060b0 0x7fb93c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19bee0 con 0x7fb93c100fd0 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19c050 con 0x7fb93c101980 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19c210 con 0x7fb93c1060b0 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 0x7fb93c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40756/0 (socket says 172.21.15.26:40756) 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93affd700 1 -- 172.21.15.26:0/2760170199 learned_addr learned my addr 172.21.15.26:0/2760170199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93b7fe700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c1060b0 0x7fb93c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 -- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c1060b0 msgr2=0x7fb93c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c1060b0 0x7fb93c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 -- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 msgr2=0x7fb93c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93a7fc700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0095f0 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.568+0000 7fb93affd700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb93c100fd0 0x7fb93c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb93a7fc700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 0x7fb93c197460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb93000e9c0 tx=0x7fb93000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000c9c0 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c19c4f0 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb930010430 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930018680 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.569+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c1a3d50 con 0x7fb93c101980 2023-10-06T21:08:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.570+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb93c110f90 con 0x7fb93c101980 2023-10-06T21:08:53.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.572+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb93000cb20 con 0x7fb93c101980 2023-10-06T21:08:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.573+0000 7fb923fff700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb9240a1510 0x7fb9240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb930014070 con 0x7fb93c101980 2023-10-06T21:08:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb93affd700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb9240a1510 0x7fb9240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb923fff700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] conn(0x7fb9240ab0d0 0x7fb9240ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9240adc40 con 0x7fb9240ab0d0 2023-10-06T21:08:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9300cd600 con 0x7fb93c101980 2023-10-06T21:08:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb93b7fe700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] conn(0x7fb9240ab0d0 0x7fb9240ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.574+0000 7fb93affd700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb9240a1510 0x7fb9240a39d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006010 tx=0x7fb92c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.575+0000 7fb93b7fe700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] conn(0x7fb9240ab0d0 0x7fb9240ad570 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.576+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== osd.11 v2:172.21.15.167:6824/4274564614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb9240adc40 con 0x7fb9240ab0d0 2023-10-06T21:08:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.593+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 --> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb93c04f400 con 0x7fb9240ab0d0 2023-10-06T21:08:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.594+0000 7fb923fff700 1 -- 172.21.15.26:0/2760170199 <== osd.11 v2:172.21.15.167:6824/4274564614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb93c04f400 con 0x7fb9240ab0d0 2023-10-06T21:08:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.594+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] conn(0x7fb9240ab0d0 msgr2=0x7fb9240ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.594+0000 7fb93bfff700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] conn(0x7fb9240ab0d0 0x7fb9240ad570 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb9240a1510 msgr2=0x7fb9240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fb9240a1510 0x7fb9240a39d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006010 tx=0x7fb92c00b260 comp rx=0 tx=0).stop 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 msgr2=0x7fb93c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 --2- 172.21.15.26:0/2760170199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb93c101980 0x7fb93c197460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb93000e9c0 tx=0x7fb93000ecd0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93b7fe700 1 -- 172.21.15.26:0/2760170199 reap_dead start 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 shutdown_connections 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 >> 172.21.15.26:0/2760170199 conn(0x7fb93c0fc860 msgr2=0x7fb93c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 shutdown_connections 2023-10-06T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.595+0000 7fb93bfff700 1 -- 172.21.15.26:0/2760170199 wait complete. 2023-10-06T21:08:53.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:53 smithi026 ceph-mon[98490]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 -- 172.21.15.26:0/3699945461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50068c00 msgr2=0x7efe50068fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 --2- 172.21.15.26:0/3699945461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50068c00 0x7efe50068fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009990 tx=0x7efe4c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 -- 172.21.15.26:0/3699945461 shutdown_connections 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 --2- 172.21.15.26:0/3699945461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 0x7efe5010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 --2- 172.21.15.26:0/3699945461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50068c00 0x7efe50068fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 --2- 172.21.15.26:0/3699945461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe500695b0 0x7efe50105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 -- 172.21.15.26:0/3699945461 >> 172.21.15.26:0/3699945461 conn(0x7efe50075e00 msgr2=0x7efe50076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 -- 172.21.15.26:0/3699945461 shutdown_connections 2023-10-06T21:08:53.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.777+0000 7efe5732d700 1 -- 172.21.15.26:0/3699945461 wait complete. 2023-10-06T21:08:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.778+0000 7efe5732d700 1 Processor -- start 2023-10-06T21:08:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.778+0000 7efe5732d700 1 -- start start 2023-10-06T21:08:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe50068c00 0x7efe501992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500695b0 0x7efe50199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 0x7efe5019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5019e2a0 con 0x7efe500695b0 2023-10-06T21:08:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5019e410 con 0x7efe50068c00 2023-10-06T21:08:53.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe5732d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5019e5d0 con 0x7efe501062b0 2023-10-06T21:08:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:53 smithi167 ceph-mon[134494]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe548c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500695b0 0x7efe50199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe550c9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe50068c00 0x7efe501992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe558ca700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 0x7efe5019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe548c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500695b0 0x7efe50199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40774/0 (socket says 172.21.15.26:40774) 2023-10-06T21:08:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe558ca700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 0x7efe5019dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:57618/0 (socket says 172.21.15.26:57618) 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.779+0000 7efe548c8700 1 -- 172.21.15.26:0/4071929180 learned_addr learned my addr 172.21.15.26:0/4071929180 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 -- 172.21.15.26:0/4071929180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 msgr2=0x7efe5019dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe501062b0 0x7efe5019dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 -- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500695b0 msgr2=0x7efe50199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe500695b0 0x7efe50199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0095f0 con 0x7efe50068c00 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe550c9700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe50068c00 0x7efe501992e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0098e0 tx=0x7efe4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01d070 con 0x7efe50068c00 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe4c005030 con 0x7efe50068c00 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c004ab0 con 0x7efe50068c00 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.780+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5019e850 con 0x7efe50068c00 2023-10-06T21:08:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.781+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe501a5f40 con 0x7efe50068c00 2023-10-06T21:08:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.781+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe50111100 con 0x7efe50068c00 2023-10-06T21:08:53.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.784+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7efe4c005410 con 0x7efe50068c00 2023-10-06T21:08:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.785+0000 7efe467fc700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efe3c0a1590 0x7efe3c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.785+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efe4c0ce280 con 0x7efe50068c00 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.785+0000 7efe548c8700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efe3c0a1590 0x7efe3c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe467fc700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] conn(0x7efe3c0ab150 0x7efe3c0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe3c0adcc0 con 0x7efe3c0ab150 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe4c0ce670 con 0x7efe50068c00 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe558ca700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] conn(0x7efe3c0ab150 0x7efe3c0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe548c8700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efe3c0a1590 0x7efe3c0a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efe5019a900 tx=0x7efe4000b460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.786+0000 7efe558ca700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] conn(0x7efe3c0ab150 0x7efe3c0ad5f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:53.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.787+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== osd.3 v2:172.21.15.26:6810/1210213318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7efe3c0adcc0 con 0x7efe3c0ab150 2023-10-06T21:08:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.808+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 --> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe5004ebb0 con 0x7efe3c0ab150 2023-10-06T21:08:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.808+0000 7efe467fc700 1 -- 172.21.15.26:0/4071929180 <== osd.3 v2:172.21.15.26:6810/1210213318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efe5004ebb0 con 0x7efe3c0ab150 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] conn(0x7efe3c0ab150 msgr2=0x7efe3c0ad5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] conn(0x7efe3c0ab150 0x7efe3c0ad5f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efe3c0a1590 msgr2=0x7efe3c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7efe3c0a1590 0x7efe3c0a3a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efe5019a900 tx=0x7efe4000b460 comp rx=0 tx=0).stop 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe50068c00 msgr2=0x7efe501992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 --2- 172.21.15.26:0/4071929180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe50068c00 0x7efe501992e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0098e0 tx=0x7efe4c004730 comp rx=0 tx=0).stop 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe558ca700 1 -- 172.21.15.26:0/4071929180 reap_dead start 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 shutdown_connections 2023-10-06T21:08:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 >> 172.21.15.26:0/4071929180 conn(0x7efe50075e00 msgr2=0x7efe5010c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 shutdown_connections 2023-10-06T21:08:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:53.809+0000 7efe5732d700 1 -- 172.21.15.26:0/4071929180 wait complete. 2023-10-06T21:08:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:53.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:53 smithi078 ceph-mon[134896]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 7.0 KiB/s, 0 objects/s recovering 2023-10-06T21:08:54.416 INFO:teuthology.orchestra.run.smithi026.stdout:55834574912 2023-10-06T21:08:54.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-06T21:08:54.626 INFO:teuthology.orchestra.run.smithi026.stdout:343597383699 2023-10-06T21:08:54.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-06T21:08:54.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 -- 172.21.15.26:0/1352699278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 msgr2=0x7f8fe4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 --2- 172.21.15.26:0/1352699278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4101400 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40099c0 tx=0x7f8fd4009cd0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 -- 172.21.15.26:0/1352699278 shutdown_connections 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 --2- 172.21.15.26:0/1352699278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe4101940 0x7f8fe410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 --2- 172.21.15.26:0/1352699278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe4106f90 0x7f8fe4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 --2- 172.21.15.26:0/1352699278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4101400 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.316+0000 7f8feb95c700 1 -- 172.21.15.26:0/1352699278 >> 172.21.15.26:0/1352699278 conn(0x7f8fe40780a0 msgr2=0x7f8fe40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.317+0000 7f8feb95c700 1 -- 172.21.15.26:0/1352699278 shutdown_connections 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.317+0000 7f8feb95c700 1 -- 172.21.15.26:0/1352699278 wait complete. 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 Processor -- start 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 -- start start 2023-10-06T21:08:54.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe4101940 0x7f8fe41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe4106f90 0x7f8fe419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe419bf40 con 0x7f8fe4100f80 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe419c0b0 con 0x7f8fe4101940 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8feb95c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe419c270 con 0x7f8fe4106f90 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.318+0000 7f8fe96f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40792/0 (socket says 172.21.15.26:40792) 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 -- 172.21.15.26:0/3118358834 learned_addr learned my addr 172.21.15.26:0/3118358834 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe9ef9700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe4106f90 0x7f8fe419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe8ef7700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe4101940 0x7f8fe41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 -- 172.21.15.26:0/3118358834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe4106f90 msgr2=0x7f8fe419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe4106f90 0x7f8fe419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 -- 172.21.15.26:0/3118358834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe4101940 msgr2=0x7f8fe41974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8fe4101940 0x7f8fe41974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40095f0 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fe96f8700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4196f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9a0 tx=0x7f8fe000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.319+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0009790 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.320+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe419c550 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.320+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe0010430 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.320+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0018680 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.320+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41a3db0 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.320+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fe404ebb0 con 0x7f8fe4100f80 2023-10-06T21:08:54.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.323+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8fe00105a0 con 0x7f8fe4100f80 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fda7fc700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8fd00a1510 0x7f8fd00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8fe0014070 con 0x7f8fe4100f80 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fe8ef7700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8fd00a1510 0x7f8fd00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fda7fc700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] conn(0x7f8fd00ab0d0 0x7f8fd00ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fd00adc40 con 0x7f8fd00ab0d0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.324+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe00cd6b0 con 0x7f8fe4100f80 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.325+0000 7f8fe9ef9700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] conn(0x7f8fd00ab0d0 0x7f8fd00ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.325+0000 7f8fe8ef7700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8fd00a1510 0x7f8fd00a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009990 tx=0x7f8fd4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.325+0000 7f8fe9ef9700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] conn(0x7f8fd00ab0d0 0x7f8fd00ad570 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.326+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== osd.9 v2:172.21.15.26:6826/157001209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8fd00adc40 con 0x7f8fd00ab0d0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.343+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 --> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8fe41111a0 con 0x7f8fd00ab0d0 2023-10-06T21:08:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.343+0000 7f8fda7fc700 1 -- 172.21.15.26:0/3118358834 <== osd.9 v2:172.21.15.26:6826/157001209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8fdc00f040 con 0x7f8fd00ab0d0 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] conn(0x7f8fd00ab0d0 msgr2=0x7f8fd00ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] conn(0x7f8fd00ab0d0 0x7f8fd00ad570 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8fd00a1510 msgr2=0x7f8fd00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8fd00a1510 0x7f8fd00a39d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009990 tx=0x7f8fd4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 msgr2=0x7f8fe4196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 --2- 172.21.15.26:0/3118358834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fe4100f80 0x7f8fe4196f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9a0 tx=0x7f8fe000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8fe9ef9700 1 -- 172.21.15.26:0/3118358834 reap_dead start 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 shutdown_connections 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 >> 172.21.15.26:0/3118358834 conn(0x7f8fe40780a0 msgr2=0x7f8fe40ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 shutdown_connections 2023-10-06T21:08:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.344+0000 7f8feb95c700 1 -- 172.21.15.26:0/3118358834 wait complete. 2023-10-06T21:08:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 -- 172.21.15.26:0/1974047693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 msgr2=0x7f183810d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 --2- 172.21.15.26:0/1974047693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f183810d3f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1828009990 tx=0x7f1828009ca0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 -- 172.21.15.26:0/1974047693 shutdown_connections 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 --2- 172.21.15.26:0/1974047693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f183810d930 0x7f183810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 --2- 172.21.15.26:0/1974047693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18380730e0 0x7f18380734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 --2- 172.21.15.26:0/1974047693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f183810d3f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 -- 172.21.15.26:0/1974047693 >> 172.21.15.26:0/1974047693 conn(0x7f18380780a0 msgr2=0x7f18380784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 -- 172.21.15.26:0/1974047693 shutdown_connections 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.776+0000 7f183fe27700 1 -- 172.21.15.26:0/1974047693 wait complete. 2023-10-06T21:08:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.778+0000 7f183fe27700 1 Processor -- start 2023-10-06T21:08:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.778+0000 7f183fe27700 1 -- start start 2023-10-06T21:08:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.778+0000 7f183fe27700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18380730e0 0x7f18381992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183fe27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f18381997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183fe27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f183810d930 0x7f183819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183fe27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819e270 con 0x7f1838073a00 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183fe27700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819e3e0 con 0x7f183810d930 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183fe27700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819e5a0 con 0x7f18380730e0 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183d3c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f18381997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f183810d930 0x7f183819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183dbc3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18380730e0 0x7f18381992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183d3c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f18381997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40836/0 (socket says 172.21.15.26:40836) 2023-10-06T21:08:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183d3c2700 1 -- 172.21.15.26:0/3699840568 learned_addr learned my addr 172.21.15.26:0/3699840568 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 -- 172.21.15.26:0/3699840568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18380730e0 msgr2=0x7f18381992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18380730e0 0x7f18381992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 -- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 msgr2=0x7f18381997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f18381997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183e3c4700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009520 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.779+0000 7f183d3c2700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1838073a00 0x7f18381997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f183e3c4700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f183810d930 0x7f183819dc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18380752c0 tx=0x7f183000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830018070 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18280095f0 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183000f040 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000cc10 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.780+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18381a5f10 con 0x7f183810d930 2023-10-06T21:08:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.781+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18381132f0 con 0x7f183810d930 2023-10-06T21:08:54.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.783+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1830003a00 con 0x7f183810d930 2023-10-06T21:08:54.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f182effd700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f18240a1510 0x7f18240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18300cc960 con 0x7f183810d930 2023-10-06T21:08:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f183dbc3700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f18240a1510 0x7f18240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f182effd700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] conn(0x7f18240ab0d0 0x7f18240ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18240adc40 con 0x7f18240ab0d0 2023-10-06T21:08:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.785+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18300ccd50 con 0x7f183810d930 2023-10-06T21:08:54.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.786+0000 7f183d3c2700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] conn(0x7f18240ab0d0 0x7f18240ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:54.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.786+0000 7f183dbc3700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f18240a1510 0x7f18240a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1834009df0 tx=0x7f1834009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.786+0000 7f183d3c2700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] conn(0x7f18240ab0d0 0x7f18240ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:54.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.787+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== osd.10 v2:172.21.15.78:6824/1966991917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f18240adc40 con 0x7f18240ab0d0 2023-10-06T21:08:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.821+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 --> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f183804ebb0 con 0x7f18240ab0d0 2023-10-06T21:08:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f182effd700 1 -- 172.21.15.26:0/3699840568 <== osd.10 v2:172.21.15.78:6824/1966991917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f183804ebb0 con 0x7f18240ab0d0 2023-10-06T21:08:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] conn(0x7f18240ab0d0 msgr2=0x7f18240ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] conn(0x7f18240ab0d0 0x7f18240ad570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f18240a1510 msgr2=0x7f18240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f18240a1510 0x7f18240a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1834009df0 tx=0x7f1834009260 comp rx=0 tx=0).stop 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f183810d930 msgr2=0x7f183819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.822+0000 7f183fe27700 1 --2- 172.21.15.26:0/3699840568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f183810d930 0x7f183819dc60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18380752c0 tx=0x7f183000ef90 comp rx=0 tx=0).stop 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.823+0000 7f183e3c4700 1 -- 172.21.15.26:0/3699840568 reap_dead start 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.823+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 shutdown_connections 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.823+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 >> 172.21.15.26:0/3699840568 conn(0x7f18380780a0 msgr2=0x7f18380ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.823+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 shutdown_connections 2023-10-06T21:08:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.823+0000 7f183fe27700 1 -- 172.21.15.26:0/3699840568 wait complete. 2023-10-06T21:08:54.847 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:55.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 -- 172.21.15.26:0/2043996225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c111de0 msgr2=0x7fcc2c1141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 --2- 172.21.15.26:0/2043996225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c111de0 0x7fcc2c1141d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcc200098f0 tx=0x7fcc20009c00 comp rx=0 tx=0).stop 2023-10-06T21:08:55.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 -- 172.21.15.26:0/2043996225 shutdown_connections 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 --2- 172.21.15.26:0/2043996225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c111de0 0x7fcc2c1141d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 --2- 172.21.15.26:0/2043996225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c0753e0 0x7fcc2c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 --2- 172.21.15.26:0/2043996225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc2c075d00 0x7fcc2c1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 -- 172.21.15.26:0/2043996225 >> 172.21.15.26:0/2043996225 conn(0x7fcc2c0fec50 msgr2=0x7fcc2c101070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 -- 172.21.15.26:0/2043996225 shutdown_connections 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.710+0000 7fcc330be700 1 -- 172.21.15.26:0/2043996225 wait complete. 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.712+0000 7fcc330be700 1 Processor -- start 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.712+0000 7fcc330be700 1 -- start start 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc330be700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0753e0 0x7fcc2c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:55.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc330be700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 0x7fcc2c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc30e5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0753e0 0x7fcc2c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc30e5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0753e0 0x7fcc2c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40814/0 (socket says 172.21.15.26:40814) 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 0x7fcc2c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.713+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 0x7fcc2c19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44298/0 (socket says 172.21.15.26:44298) 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc330be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc2c111de0 0x7fcc2c1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1a25c0 con 0x7fcc2c0753e0 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc30e5a700 1 -- 172.21.15.26:0/3961888474 learned_addr learned my addr 172.21.15.26:0/3961888474 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1a2730 con 0x7fcc2c075d00 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1a28f0 con 0x7fcc2c111de0 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc3165b700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc2c111de0 0x7fcc2c1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc2bfff700 1 -- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc2c111de0 msgr2=0x7fcc2c1a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc2bfff700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc2c111de0 0x7fcc2c1a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc2bfff700 1 -- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0753e0 msgr2=0x7fcc2c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc2bfff700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0753e0 0x7fcc2c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.714+0000 7fcc2bfff700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0095f0 con 0x7fcc2c075d00 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc2bfff700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 0x7fcc2c19db40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00eb90 tx=0x7fcc1c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00ccf0 con 0x7fcc2c075d00 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1c00ce50 con 0x7fcc2c075d00 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c010430 con 0x7fcc2c075d00 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc20009520 con 0x7fcc2c075d00 2023-10-06T21:08:55.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.715+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c1aa260 con 0x7fcc2c075d00 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.716+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc2c1177a0 con 0x7fcc2c075d00 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.718+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fcc1c003f50 con 0x7fcc2c075d00 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc29ffb700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcc140a1610 0x7fcc140a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcc1c014070 con 0x7fcc2c075d00 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc29ffb700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] conn(0x7fcc140ab1d0 0x7fcc140ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc140add40 con 0x7fcc140ab1d0 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc30e5a700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcc140a1610 0x7fcc140a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.720+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c0cd650 con 0x7fcc2c075d00 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.721+0000 7fcc3165b700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] conn(0x7fcc140ab1d0 0x7fcc140ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.721+0000 7fcc30e5a700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcc140a1610 0x7fcc140a3ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009c10 tx=0x7fcc2400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.721+0000 7fcc3165b700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] conn(0x7fcc140ab1d0 0x7fcc140ad670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.722+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== osd.5 v2:172.21.15.167:6808/644360584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcc140add40 con 0x7fcc140ab1d0 2023-10-06T21:08:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.740+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 --> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc2c04ebb0 con 0x7fcc140ab1d0 2023-10-06T21:08:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.741+0000 7fcc29ffb700 1 -- 172.21.15.26:0/3961888474 <== osd.5 v2:172.21.15.167:6808/644360584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcc2c04ebb0 con 0x7fcc140ab1d0 2023-10-06T21:08:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] conn(0x7fcc140ab1d0 msgr2=0x7fcc140ad670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] conn(0x7fcc140ab1d0 0x7fcc140ad670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcc140a1610 msgr2=0x7fcc140a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcc140a1610 0x7fcc140a3ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009c10 tx=0x7fcc2400b040 comp rx=0 tx=0).stop 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 msgr2=0x7fcc2c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 --2- 172.21.15.26:0/3961888474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc2c075d00 0x7fcc2c19db40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00eb90 tx=0x7fcc1c00eea0 comp rx=0 tx=0).stop 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc3165b700 1 -- 172.21.15.26:0/3961888474 reap_dead start 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 shutdown_connections 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 >> 172.21.15.26:0/3961888474 conn(0x7fcc2c0fec50 msgr2=0x7fcc2c112a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 shutdown_connections 2023-10-06T21:08:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:54.742+0000 7fcc330be700 1 -- 172.21.15.26:0/3961888474 wait complete. 2023-10-06T21:08:55.625 INFO:teuthology.orchestra.run.smithi026.stdout:309237645335 2023-10-06T21:08:55.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-06T21:08:55.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:55 smithi026 ceph-mon[98490]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 205 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:55.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:55 smithi167 ceph-mon[134494]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 205 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:55.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:55 smithi078 ceph-mon[134896]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 205 B/s wr, 5 op/s; 5.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:56.410 INFO:teuthology.orchestra.run.smithi026.stdout:287762808859 2023-10-06T21:08:56.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-06T21:08:56.428 INFO:teuthology.orchestra.run.smithi026.stdout:163208757294 2023-10-06T21:08:56.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-06T21:08:56.743 INFO:teuthology.orchestra.run.smithi026.stdout:107374182455 2023-10-06T21:08:56.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-06T21:08:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:57.761 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:57 smithi026 ceph-mon[98490]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 187 B/s wr, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2023-10-06T21:08:57.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:57 smithi167 ceph-mon[134494]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 187 B/s wr, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2023-10-06T21:08:57.819 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:57.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:57 smithi078 ceph-mon[134896]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 187 B/s wr, 5 op/s; 5.1 KiB/s, 0 objects/s recovering 2023-10-06T21:08:57.906 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:58.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 -- 172.21.15.26:0/4123912893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8106f90 msgr2=0x7f8de8107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:58.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 --2- 172.21.15.26:0/4123912893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8106f90 0x7f8de8107370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b0a0 tx=0x7f8de000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 -- 172.21.15.26:0/4123912893 shutdown_connections 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 --2- 172.21.15.26:0/4123912893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 0x7f8de810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 --2- 172.21.15.26:0/4123912893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8106f90 0x7f8de8107370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 --2- 172.21.15.26:0/4123912893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8100f80 0x7f8de8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 -- 172.21.15.26:0/4123912893 >> 172.21.15.26:0/4123912893 conn(0x7f8de80780a0 msgr2=0x7f8de80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 -- 172.21.15.26:0/4123912893 shutdown_connections 2023-10-06T21:08:58.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.274+0000 7f8df00ea700 1 -- 172.21.15.26:0/4123912893 wait complete. 2023-10-06T21:08:58.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.276+0000 7f8df00ea700 1 Processor -- start 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.277+0000 7f8df00ea700 1 -- start start 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.278+0000 7f8df00ea700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.278+0000 7f8df00ea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 0x7f8de8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8dede86700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8df00ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 0x7f8de819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8df00ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de819e3b0 con 0x7f8de8106f90 2023-10-06T21:08:58.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8df00ea700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de819e520 con 0x7f8de8101940 2023-10-06T21:08:58.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8df00ea700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de819e6e0 con 0x7f8de8100f80 2023-10-06T21:08:58.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8dede86700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:57702/0 (socket says 172.21.15.26:57702) 2023-10-06T21:08:58.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.279+0000 7f8dede86700 1 -- 172.21.15.26:0/2045909003 learned_addr learned my addr 172.21.15.26:0/2045909003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:58.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8ded685700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 0x7f8de8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:58.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dee687700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 0x7f8de819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dede86700 1 -- 172.21.15.26:0/2045909003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 msgr2=0x7f8de8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dede86700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 0x7f8de8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dede86700 1 -- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 msgr2=0x7f8de819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dede86700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 0x7f8de819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dede86700 1 -- 172.21.15.26:0/2045909003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc009520 con 0x7f8de8100f80 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.280+0000 7f8dee687700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 0x7f8de819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8dede86700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8de00062a0 tx=0x7f8de000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de000e040 con 0x7f8de8100f80 2023-10-06T21:08:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de00094a0 con 0x7f8de8100f80 2023-10-06T21:08:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de0009d00 con 0x7f8de8100f80 2023-10-06T21:08:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de001b6c0 con 0x7f8de8100f80 2023-10-06T21:08:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.281+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de81a6050 con 0x7f8de8100f80 2023-10-06T21:08:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.283+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de804ebb0 con 0x7f8de8100f80 2023-10-06T21:08:58.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.286+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8de001d470 con 0x7f8de8100f80 2023-10-06T21:08:58.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.289+0000 7f8de6ffd700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 0x7f8dd00a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:58.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.289+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8de00ce170 con 0x7f8de8100f80 2023-10-06T21:08:58.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.289+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8de00ce550 con 0x7f8de8100f80 2023-10-06T21:08:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.290+0000 7f8ded685700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 0x7f8dd00a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.292+0000 7f8ded685700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 0x7f8dd00a3ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8de819aa10 tx=0x7f8ddc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:58.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.533+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8de8110e80 con 0x7f8de8100f80 2023-10-06T21:08:58.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.533+0000 7f8de6ffd700 1 -- 172.21.15.26:0/2045909003 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8de00976c0 con 0x7f8de8100f80 2023-10-06T21:08:58.536 INFO:teuthology.orchestra.run.smithi026.stdout:141733920820 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 msgr2=0x7f8dd00a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 0x7f8dd00a3ad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8de819aa10 tx=0x7f8ddc009260 comp rx=0 tx=0).stop 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 msgr2=0x7f8de81993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8de00062a0 tx=0x7f8de000be10 comp rx=0 tx=0).stop 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 shutdown_connections 2023-10-06T21:08:58.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8dd00a1610 0x7f8dd00a3ad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de8106f90 0x7f8de819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8de8100f80 0x7f8de81993f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 --2- 172.21.15.26:0/2045909003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8de8101940 0x7f8de8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.536+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 >> 172.21.15.26:0/2045909003 conn(0x7f8de80780a0 msgr2=0x7f8de810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.537+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 shutdown_connections 2023-10-06T21:08:58.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:58.537+0000 7f8df00ea700 1 -- 172.21.15.26:0/2045909003 wait complete. 2023-10-06T21:08:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:58.825 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:08:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:58 smithi026 ceph-mon[98490]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:58 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2045909003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:08:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:58 smithi078 ceph-mon[134896]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:58 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2045909003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:08:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:58 smithi167 ceph-mon[134494]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s; 4.7 KiB/s, 0 objects/s recovering 2023-10-06T21:08:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:58 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2045909003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-06T21:08:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 -- 172.21.15.26:0/4259140672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8101940 msgr2=0x7f10d810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/4259140672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8101940 0x7f10d810ddb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f10d000b0d0 tx=0x7f10d000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 -- 172.21.15.26:0/4259140672 shutdown_connections 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/4259140672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8101940 0x7f10d810ddb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/4259140672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8106f90 0x7f10d8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/4259140672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8100f80 0x7f10d8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 -- 172.21.15.26:0/4259140672 >> 172.21.15.26:0/4259140672 conn(0x7f10d80780a0 msgr2=0x7f10d80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 -- 172.21.15.26:0/4259140672 shutdown_connections 2023-10-06T21:08:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.569+0000 7f10dc8b1700 1 -- 172.21.15.26:0/4259140672 wait complete. 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.570+0000 7f10dc8b1700 1 Processor -- start 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 -- start start 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 0x7f10d80731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 0x7f10d80736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8106f90 0x7f10d8073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81a9aa0 con 0x7f10d8100f80 2023-10-06T21:08:59.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81a9be0 con 0x7f10d8106f90 2023-10-06T21:08:59.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10dc8b1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d81a9d20 con 0x7f10d8101940 2023-10-06T21:08:59.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10d659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 0x7f10d80731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10d5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 0x7f10d80736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10d5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 0x7f10d80736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:57714/0 (socket says 172.21.15.26:57714) 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10d5d9b700 1 -- 172.21.15.26:0/1017293864 learned_addr learned my addr 172.21.15.26:0/1017293864 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.571+0000 7f10d6d9d700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8106f90 0x7f10d8073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 -- 172.21.15.26:0/1017293864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 msgr2=0x7f10d80736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 0x7f10d80736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 -- 172.21.15.26:0/1017293864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8106f90 msgr2=0x7f10d8073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8106f90 0x7f10d8073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 -- 172.21.15.26:0/1017293864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009d00 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10d659c700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 0x7f10d80731b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d750 tx=0x7f10c800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004020 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d81a9fb0 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c8004830 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.572+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c80054c0 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.573+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d81aa470 con 0x7f10d8100f80 2023-10-06T21:08:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.574+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d804ebb0 con 0x7f10d8100f80 2023-10-06T21:08:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.576+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f10c80049a0 con 0x7f10d8100f80 2023-10-06T21:08:59.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.578+0000 7f10c77fe700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 0x7f10c00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:08:59.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.578+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f10c80cc5e0 con 0x7f10d8100f80 2023-10-06T21:08:59.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.578+0000 7f10d5d9b700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 0x7f10c00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:08:59.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.579+0000 7f10d5d9b700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 0x7f10c00a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10d8074950 tx=0x7f10cc007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:08:59.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.583+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f10c8095ae0 con 0x7f10d8100f80 2023-10-06T21:08:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.804+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f10d8074690 con 0x7f10d8100f80 2023-10-06T21:08:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.804+0000 7f10c77fe700 1 -- 172.21.15.26:0/1017293864 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10c8099100 con 0x7f10d8100f80 2023-10-06T21:08:59.807 INFO:teuthology.orchestra.run.smithi026.stdout:77309411390 2023-10-06T21:08:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 msgr2=0x7f10c00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 0x7f10c00a3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10d8074950 tx=0x7f10cc007480 comp rx=0 tx=0).stop 2023-10-06T21:08:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 msgr2=0x7f10d80731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 0x7f10d80731b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d750 tx=0x7f10c800da60 comp rx=0 tx=0).stop 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 shutdown_connections 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10d8106f90 0x7f10d8073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f10c00a1590 0x7f10c00a3a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d8100f80 0x7f10d80731b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 --2- 172.21.15.26:0/1017293864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8101940 0x7f10d80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:08:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 >> 172.21.15.26:0/1017293864 conn(0x7f10d80780a0 msgr2=0x7f10d8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:08:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 shutdown_connections 2023-10-06T21:08:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:08:59.807+0000 7f10dc8b1700 1 -- 172.21.15.26:0/1017293864 wait complete. 2023-10-06T21:09:00.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:08:59 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1017293864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:09:00.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:08:59 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1017293864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:09:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:08:59 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1017293864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-06T21:09:00.454 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920820 for osd.4 2023-10-06T21:09:00.455 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:00.953 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:00.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:00 smithi026 ceph-mon[98490]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2023-10-06T21:09:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:00 smithi078 ceph-mon[134896]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2023-10-06T21:09:01.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:00 smithi167 ceph-mon[134494]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s; 1 B/s, 0 objects/s recovering 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 -- 172.21.15.26:0/1556592358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f90 msgr2=0x7fcf74107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 --2- 172.21.15.26:0/1556592358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f90 0x7fcf74107370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf700099c0 tx=0x7fcf70009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 -- 172.21.15.26:0/1556592358 shutdown_connections 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 --2- 172.21.15.26:0/1556592358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74101940 0x7fcf7410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 --2- 172.21.15.26:0/1556592358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74106f90 0x7fcf74107370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 --2- 172.21.15.26:0/1556592358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74100f80 0x7fcf74101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 -- 172.21.15.26:0/1556592358 >> 172.21.15.26:0/1556592358 conn(0x7fcf740780a0 msgr2=0x7fcf740784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 -- 172.21.15.26:0/1556592358 shutdown_connections 2023-10-06T21:09:01.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.728+0000 7fcf7c241700 1 -- 172.21.15.26:0/1556592358 wait complete. 2023-10-06T21:09:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.729+0000 7fcf7c241700 1 Processor -- start 2023-10-06T21:09:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.729+0000 7fcf7c241700 1 -- start start 2023-10-06T21:09:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:01.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74101940 0x7fcf741974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74106f90 0x7fcf7419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419bf60 con 0x7fcf74100f80 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419c0d0 con 0x7fcf74106f90 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7c241700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419c290 con 0x7fcf74101940 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41388/0 (socket says 172.21.15.26:41388) 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 -- 172.21.15.26:0/3477170398 learned_addr learned my addr 172.21.15.26:0/3477170398 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf797dc700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74101940 0x7fcf741974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf7a7de700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74106f90 0x7fcf7419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:01.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 -- 172.21.15.26:0/3477170398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74101940 msgr2=0x7fcf741974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74101940 0x7fcf741974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 -- 172.21.15.26:0/3477170398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74106f90 msgr2=0x7fcf7419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74106f90 0x7fcf7419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.730+0000 7fcf79fdd700 1 -- 172.21.15.26:0/3477170398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700095f0 con 0x7fcf74100f80 2023-10-06T21:09:01.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf79fdd700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b580 tx=0x7fcf7000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7001d070 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7419c510 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf70022470 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70017630 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.731+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741a3dd0 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.732+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7404ebb0 con 0x7fcf74100f80 2023-10-06T21:09:01.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.734+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fcf700225e0 con 0x7fcf74100f80 2023-10-06T21:09:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.736+0000 7fcf6affd700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 0x7fcf600a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.736+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcf700cded0 con 0x7fcf74100f80 2023-10-06T21:09:01.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.736+0000 7fcf797dc700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 0x7fcf600a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:01.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.737+0000 7fcf797dc700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 0x7fcf600a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf741985c0 tx=0x7fcf64009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.740+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcf70097420 con 0x7fcf74100f80 2023-10-06T21:09:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.970+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf74110e50 con 0x7fcf74100f80 2023-10-06T21:09:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.970+0000 7fcf6affd700 1 -- 172.21.15.26:0/3477170398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf700228c0 con 0x7fcf74100f80 2023-10-06T21:09:01.973 INFO:teuthology.orchestra.run.smithi026.stdout:34359738437 2023-10-06T21:09:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 msgr2=0x7fcf600a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 0x7fcf600a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf741985c0 tx=0x7fcf64009260 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 msgr2=0x7fcf74196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b580 tx=0x7fcf7000bfd0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 shutdown_connections 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf74106f90 0x7fcf7419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fcf600a1510 0x7fcf600a39d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf74100f80 0x7fcf74196fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 --2- 172.21.15.26:0/3477170398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf74101940 0x7fcf741974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 >> 172.21.15.26:0/3477170398 conn(0x7fcf740780a0 msgr2=0x7fcf740ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 shutdown_connections 2023-10-06T21:09:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:01.973+0000 7fcf7c241700 1 -- 172.21.15.26:0/3477170398 wait complete. 2023-10-06T21:09:02.045 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:02 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3477170398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:09:02.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:02 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3477170398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:09:02.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:02 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3477170398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-06T21:09:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:03.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:03 smithi026 ceph-mon[98490]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:09:03.095 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:03 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2349353343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:09:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 -- 172.21.15.26:0/2802023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c1062b0 msgr2=0x7fc07c10de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 --2- 172.21.15.26:0/2802023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c1062b0 0x7fc07c10de30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc074009f90 tx=0x7fc07400b330 comp rx=0 tx=0).stop 2023-10-06T21:09:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 -- 172.21.15.26:0/2802023963 shutdown_connections 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 --2- 172.21.15.26:0/2802023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c1062b0 0x7fc07c10de30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 --2- 172.21.15.26:0/2802023963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 --2- 172.21.15.26:0/2802023963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c0695b0 0x7fc07c105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 -- 172.21.15.26:0/2802023963 >> 172.21.15.26:0/2802023963 conn(0x7fc07c075e00 msgr2=0x7fc07c076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 -- 172.21.15.26:0/2802023963 shutdown_connections 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.749+0000 7fc083572700 1 -- 172.21.15.26:0/2802023963 wait complete. 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.751+0000 7fc083572700 1 Processor -- start 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.751+0000 7fc083572700 1 -- start start 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c0695b0 0x7fc07c100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:03.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c1062b0 0x7fc07c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a55b0 con 0x7fc07c0695b0 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a5720 con 0x7fc07c068c00 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc083572700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1a58e0 con 0x7fc07c1062b0 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc080b0d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c0695b0 0x7fc07c100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39498/0 (socket says 172.21.15.26:39498) 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 -- 172.21.15.26:0/2349353343 learned_addr learned my addr 172.21.15.26:0/2349353343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc081b0f700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c1062b0 0x7fc07c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 -- 172.21.15.26:0/2349353343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c1062b0 msgr2=0x7fc07c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c1062b0 0x7fc07c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 -- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c0695b0 msgr2=0x7fc07c100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c0695b0 0x7fc07c100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.752+0000 7fc08130e700 1 -- 172.21.15.26:0/2349353343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c009520 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc08130e700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc07800e990 tx=0x7fc07800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078004220 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc078004cf0 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078005720 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc074009c40 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.753+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1a5ee0 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.755+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c04ebb0 con 0x7fc07c068c00 2023-10-06T21:09:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.757+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc078004380 con 0x7fc07c068c00 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.759+0000 7fc0727fc700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 0x7fc0680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.760+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc0780ccb90 con 0x7fc07c068c00 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.760+0000 7fc080b0d700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 0x7fc0680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.760+0000 7fc080b0d700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 0x7fc0680a39d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc07c101720 tx=0x7fc06c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.762+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc078096050 con 0x7fc07c068c00 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.991+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc07c101460 con 0x7fc07c068c00 2023-10-06T21:09:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.992+0000 7fc0727fc700 1 -- 172.21.15.26:0/2349353343 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc07801f020 con 0x7fc07c068c00 2023-10-06T21:09:03.116 INFO:teuthology.orchestra.run.smithi026.stdout:343597383701 2023-10-06T21:09:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 msgr2=0x7fc0680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 0x7fc0680a39d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc07c101720 tx=0x7fc06c009260 comp rx=0 tx=0).stop 2023-10-06T21:09:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 msgr2=0x7fc07c0fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc07800e990 tx=0x7fc07800eca0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 shutdown_connections 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc07c1062b0 0x7fc07c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c068c00 0x7fc07c0fff30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc0680a1510 0x7fc0680a39d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 --2- 172.21.15.26:0/2349353343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc07c0695b0 0x7fc07c100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 >> 172.21.15.26:0/2349353343 conn(0x7fc07c075e00 msgr2=0x7fc07c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.994+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 shutdown_connections 2023-10-06T21:09:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:02.995+0000 7fc083572700 1 -- 172.21.15.26:0/2349353343 wait complete. 2023-10-06T21:09:03.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411390 for osd.2 2023-10-06T21:09:03.157 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:03.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:03 smithi167 ceph-mon[134494]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:09:03.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:03 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2349353343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:09:03.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:03 smithi078 ceph-mon[134896]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:09:03.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:03 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2349353343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-06T21:09:03.686 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738435 got 34359738437 for osd.0 2023-10-06T21:09:03.687 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:04.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383699 got 343597383701 for osd.11 2023-10-06T21:09:04.299 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 -- 172.21.15.26:0/436818019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948105e70 msgr2=0x7fc948106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/436818019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948105e70 0x7fc948106250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc944009920 tx=0x7fc944009c30 comp rx=0 tx=0).stop 2023-10-06T21:09:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 -- 172.21.15.26:0/436818019 shutdown_connections 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/436818019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948100820 0x7fc94810cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/436818019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948105e70 0x7fc948106250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/436818019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 -- 172.21.15.26:0/436818019 >> 172.21.15.26:0/436818019 conn(0x7fc948075e00 msgr2=0x7fc948076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 -- 172.21.15.26:0/436818019 shutdown_connections 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.797+0000 7fc94f2b7700 1 -- 172.21.15.26:0/436818019 wait complete. 2023-10-06T21:09:04.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.799+0000 7fc94f2b7700 1 Processor -- start 2023-10-06T21:09:04.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.799+0000 7fc94f2b7700 1 -- start start 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.800+0000 7fc94f2b7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.800+0000 7fc94f2b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 0x7fc948199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94f2b7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948105e70 0x7fc94819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94f2b7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819e2b0 con 0x7fc948105e70 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94f2b7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819e420 con 0x7fc948100820 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94f2b7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94819e5e0 con 0x7fc9480ffe60 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94d053700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94c852700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 0x7fc948199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94d053700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41840/0 (socket says 172.21.15.26:41840) 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94d053700 1 -- 172.21.15.26:0/2500147630 learned_addr learned my addr 172.21.15.26:0/2500147630 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.801+0000 7fc94d854700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948105e70 0x7fc94819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 -- 172.21.15.26:0/2500147630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 msgr2=0x7fc9481992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 -- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948105e70 msgr2=0x7fc94819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:04.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948105e70 0x7fc94819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:04.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 -- 172.21.15.26:0/2500147630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380095f0 con 0x7fc948100820 2023-10-06T21:09:04.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.802+0000 7fc94c852700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 0x7fc948199830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc93800daa0 tx=0x7fc93800ddb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:04.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.803+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938012070 con 0x7fc948100820 2023-10-06T21:09:04.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.803+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc944009520 con 0x7fc948100820 2023-10-06T21:09:04.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.803+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93800ce30 con 0x7fc948100820 2023-10-06T21:09:04.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.803+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380039b0 con 0x7fc948100820 2023-10-06T21:09:04.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.803+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9481a5f50 con 0x7fc948100820 2023-10-06T21:09:04.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.805+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94804ebb0 con 0x7fc948100820 2023-10-06T21:09:04.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.807+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc93800c970 con 0x7fc948100820 2023-10-06T21:09:04.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.808+0000 7fc93e7fc700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 0x7fc9340a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:04.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.808+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9380cc350 con 0x7fc948100820 2023-10-06T21:09:04.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.809+0000 7fc94d053700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 0x7fc9340a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.809+0000 7fc94d053700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 0x7fc9340a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c010 tx=0x7fc944019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.812+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc938095810 con 0x7fc948100820 2023-10-06T21:09:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 -- 172.21.15.26:0/3330790866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038068ee0 msgr2=0x7f7038069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 --2- 172.21.15.26:0/3330790866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038068ee0 0x7f7038069360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7028009990 tx=0x7f7028009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 -- 172.21.15.26:0/3330790866 shutdown_connections 2023-10-06T21:09:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 --2- 172.21.15.26:0/3330790866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7038103960 0x7f7038103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 --2- 172.21.15.26:0/3330790866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038105ef0 0x7f70380689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 --2- 172.21.15.26:0/3330790866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038068ee0 0x7f7038069360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 -- 172.21.15.26:0/3330790866 >> 172.21.15.26:0/3330790866 conn(0x7f7038075e00 msgr2=0x7f7038076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 -- 172.21.15.26:0/3330790866 shutdown_connections 2023-10-06T21:09:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.010+0000 7f703f543700 1 -- 172.21.15.26:0/3330790866 wait complete. 2023-10-06T21:09:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.012+0000 7f703f543700 1 Processor -- start 2023-10-06T21:09:05.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.012+0000 7f703f543700 1 -- start start 2023-10-06T21:09:05.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.012+0000 7f703f543700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703f543700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7038103960 0x7f703806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703f543700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038105ef0 0x7f70381b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703f543700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70381b2060 con 0x7f7038105ef0 2023-10-06T21:09:05.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703f543700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70381b21a0 con 0x7f7038068ee0 2023-10-06T21:09:05.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703f543700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70381b2360 con 0x7f7038103960 2023-10-06T21:09:05.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39562/0 (socket says 172.21.15.26:39562) 2023-10-06T21:09:05.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 -- 172.21.15.26:0/3056644843 learned_addr learned my addr 172.21.15.26:0/3056644843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703dae0700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038105ef0 0x7f70381b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 -- 172.21.15.26:0/3056644843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7038103960 msgr2=0x7f703806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7038103960 0x7f703806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 -- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038105ef0 msgr2=0x7f70381b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038105ef0 0x7f70381b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.013+0000 7f703d2df700 1 -- 172.21.15.26:0/3056644843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034009520 con 0x7f7038068ee0 2023-10-06T21:09:05.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f703d2df700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f703400eb30 tx=0x7f703400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034009880 con 0x7f7038068ee0 2023-10-06T21:09:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70280095f0 con 0x7f7038068ee0 2023-10-06T21:09:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70340044e0 con 0x7f7038068ee0 2023-10-06T21:09:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034003e90 con 0x7f7038068ee0 2023-10-06T21:09:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.014+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70381b2a00 con 0x7f7038068ee0 2023-10-06T21:09:05.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.015+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703804ebb0 con 0x7f7038068ee0 2023-10-06T21:09:05.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.017+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f7034004000 con 0x7f7038068ee0 2023-10-06T21:09:05.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.019+0000 7f702e7fc700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 0x7f70240a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.019+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7034014070 con 0x7f7038068ee0 2023-10-06T21:09:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.019+0000 7f703cade700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 0x7f70240a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.020+0000 7f703cade700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 0x7f70240a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70280098e0 tx=0x7f7028005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.020+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7034096710 con 0x7f7038068ee0 2023-10-06T21:09:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 -- 172.21.15.26:0/4013081785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 msgr2=0x7f33e4105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/4013081785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4105b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f33d80098f0 tx=0x7f33d8009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 -- 172.21.15.26:0/4013081785 shutdown_connections 2023-10-06T21:09:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/4013081785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e41062b0 0x7f33e410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/4013081785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e4068c00 0x7f33e4068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/4013081785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4105b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 -- 172.21.15.26:0/4013081785 >> 172.21.15.26:0/4013081785 conn(0x7f33e4075e00 msgr2=0x7f33e4076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 -- 172.21.15.26:0/4013081785 shutdown_connections 2023-10-06T21:09:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.046+0000 7f33e9bcc700 1 -- 172.21.15.26:0/4013081785 wait complete. 2023-10-06T21:09:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.047+0000 7f33e9bcc700 1 Processor -- start 2023-10-06T21:09:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 -- start start 2023-10-06T21:09:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e4068c00 0x7f33e40fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41a54e0 con 0x7f33e41062b0 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41a5650 con 0x7f33e40695b0 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e9bcc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41a5810 con 0x7f33e4068c00 2023-10-06T21:09:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41460/0 (socket says 172.21.15.26:41460) 2023-10-06T21:09:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39594/0 (socket says 172.21.15.26:39594) 2023-10-06T21:09:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e3fff700 1 -- 172.21.15.26:0/2325038021 learned_addr learned my addr 172.21.15.26:0/2325038021 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.048+0000 7f33e37fe700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e4068c00 0x7f33e40fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 -- 172.21.15.26:0/2325038021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e4068c00 msgr2=0x7f33e40fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e4068c00 0x7f33e40fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 -- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 msgr2=0x7f33e41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 -- 172.21.15.26:0/2325038021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40095f0 con 0x7f33e40695b0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e3fff700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.049+0000 7f33e2ffd700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33d8000c00 tx=0x7f33d800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.050+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d801c070 con 0x7f33e40695b0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.050+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d8009520 con 0x7f33e40695b0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.050+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33d800bb20 con 0x7f33e40695b0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.050+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d80038e0 con 0x7f33e40695b0 2023-10-06T21:09:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.050+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e41a5d20 con 0x7f33e40695b0 2023-10-06T21:09:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.051+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e41094a0 con 0x7f33e40695b0 2023-10-06T21:09:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.053+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f33d8020030 con 0x7f33e40695b0 2023-10-06T21:09:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.054+0000 7f33e0ff9700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 0x7f33cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.054+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f33d80cdfe0 con 0x7f33e40695b0 2023-10-06T21:09:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.054+0000 7f33e37fe700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 0x7f33cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.055+0000 7f33e37fe700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 0x7f33cc0a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005fd0 tx=0x7f33d4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.060 INFO:teuthology.orchestra.run.smithi026.stdout:219043332136 2023-10-06T21:09:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 -- 172.21.15.26:0/2101831127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728105ef0 msgr2=0x7ff7280689a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 --2- 172.21.15.26:0/2101831127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728105ef0 0x7ff7280689a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff718009990 tx=0x7ff718009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 -- 172.21.15.26:0/2101831127 shutdown_connections 2023-10-06T21:09:05.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 --2- 172.21.15.26:0/2101831127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff728103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 --2- 172.21.15.26:0/2101831127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728105ef0 0x7ff7280689a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 --2- 172.21.15.26:0/2101831127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728068ee0 0x7ff728069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 -- 172.21.15.26:0/2101831127 >> 172.21.15.26:0/2101831127 conn(0x7ff728075e00 msgr2=0x7ff728076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 -- 172.21.15.26:0/2101831127 shutdown_connections 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.651+0000 7ff72df20700 1 -- 172.21.15.26:0/2101831127 wait complete. 2023-10-06T21:09:05.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.653+0000 7ff72df20700 1 Processor -- start 2023-10-06T21:09:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.653+0000 7ff72df20700 1 -- start start 2023-10-06T21:09:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.653+0000 7ff72df20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 0x7ff72819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.653+0000 7ff72df20700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff72df20700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728105ef0 0x7ff7281a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff72df20700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a24f0 con 0x7ff728103960 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff72df20700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a2660 con 0x7ff728068ee0 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff72df20700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281a2820 con 0x7ff728105ef0 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff7277fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 0x7ff72819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41418/0 (socket says 172.21.15.26:41418) 2023-10-06T21:09:05.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 -- 172.21.15.26:0/821955536 learned_addr learned my addr 172.21.15.26:0/821955536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff727fff700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728105ef0 0x7ff7281a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 -- 172.21.15.26:0/821955536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728105ef0 msgr2=0x7ff7281a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728105ef0 0x7ff7281a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 -- 172.21.15.26:0/821955536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 msgr2=0x7ff72819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 0x7ff72819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff726ffd700 1 -- 172.21.15.26:0/821955536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009c40 con 0x7ff728103960 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.654+0000 7ff7277fe700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 0x7ff72819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff726ffd700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9a0 tx=0x7ff71c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c009790 con 0x7ff728103960 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7180095f0 con 0x7ff728103960 2023-10-06T21:09:05.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff71c00cbd0 con 0x7ff728103960 2023-10-06T21:09:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c018680 con 0x7ff728103960 2023-10-06T21:09:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.655+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7281aa190 con 0x7ff728103960 2023-10-06T21:09:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.656+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72804ebb0 con 0x7ff728103960 2023-10-06T21:09:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.660+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff71c00cd40 con 0x7ff728103960 2023-10-06T21:09:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.662+0000 7ff724ff9700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 0x7ff7100a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.662+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff71c014070 con 0x7ff728103960 2023-10-06T21:09:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.663+0000 7ff7277fe700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 0x7ff7100a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.664+0000 7ff7277fe700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 0x7ff7100a39d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff71800b2e0 tx=0x7ff718005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.664+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff71c0968e0 con 0x7ff728103960 2023-10-06T21:09:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.944+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff72819e890 con 0x7ff728103960 2023-10-06T21:09:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.944+0000 7ff724ff9700 1 -- 172.21.15.26:0/821955536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff71c099f00 con 0x7ff728103960 2023-10-06T21:09:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.946+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 msgr2=0x7ff7100a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.946+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 0x7ff7100a39d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff71800b2e0 tx=0x7ff718005960 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.946+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 msgr2=0x7ff72819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.946+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9a0 tx=0x7ff71c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 shutdown_connections 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff728105ef0 0x7ff7281a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff728068ee0 0x7ff72819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff7100a1510 0x7ff7100a39d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 --2- 172.21.15.26:0/821955536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff728103960 0x7ff72819da70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 >> 172.21.15.26:0/821955536 conn(0x7ff728075e00 msgr2=0x7ff7280fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 shutdown_connections 2023-10-06T21:09:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:04.947+0000 7ff72df20700 1 -- 172.21.15.26:0/821955536 wait complete. 2023-10-06T21:09:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.056+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f33d8097550 con 0x7f33e40695b0 2023-10-06T21:09:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.076+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc94810ed90 con 0x7fc948100820 2023-10-06T21:09:05.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.076+0000 7fc93e7fc700 1 -- 172.21.15.26:0/2500147630 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc938098e30 con 0x7fc948100820 2023-10-06T21:09:05.079 INFO:teuthology.orchestra.run.smithi026.stdout:197568495658 2023-10-06T21:09:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 msgr2=0x7fc9340a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 0x7fc9340a3a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c010 tx=0x7fc944019040 comp rx=0 tx=0).stop 2023-10-06T21:09:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 msgr2=0x7fc948199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 0x7fc948199830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc93800daa0 tx=0x7fc93800ddb0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 shutdown_connections 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc9340a1590 0x7fc9340a3a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc948105e70 0x7fc94819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9480ffe60 0x7fc9481992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 --2- 172.21.15.26:0/2500147630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc948100820 0x7fc948199830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.079+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 >> 172.21.15.26:0/2500147630 conn(0x7fc948075e00 msgr2=0x7fc9480fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.080+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 shutdown_connections 2023-10-06T21:09:05.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.080+0000 7fc94f2b7700 1 -- 172.21.15.26:0/2500147630 wait complete. 2023-10-06T21:09:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.264+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f703806c700 con 0x7f7038068ee0 2023-10-06T21:09:05.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.264+0000 7f702e7fc700 1 -- 172.21.15.26:0/3056644843 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7034099d30 con 0x7f7038068ee0 2023-10-06T21:09:05.267 INFO:teuthology.orchestra.run.smithi026.stdout:55834574915 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 msgr2=0x7f70240a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 0x7f70240a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f70280098e0 tx=0x7f7028005e80 comp rx=0 tx=0).stop 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 msgr2=0x7f703806b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f703400eb30 tx=0x7f703400ee40 comp rx=0 tx=0).stop 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 shutdown_connections 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f70240a1590 0x7f70240a3a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7038105ef0 0x7f70381b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7038068ee0 0x7f703806b3a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 --2- 172.21.15.26:0/3056644843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7038103960 0x7f703806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 >> 172.21.15.26:0/3056644843 conn(0x7f7038075e00 msgr2=0x7f70380fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 shutdown_connections 2023-10-06T21:09:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.267+0000 7f703f543700 1 -- 172.21.15.26:0/3056644843 wait complete. 2023-10-06T21:09:05.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.342+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f33e404ebb0 con 0x7f33e40695b0 2023-10-06T21:09:05.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.342+0000 7f33e0ff9700 1 -- 172.21.15.26:0/2325038021 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33d809ab70 con 0x7f33e40695b0 2023-10-06T21:09:05.345 INFO:teuthology.orchestra.run.smithi026.stdout:253403070498 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 msgr2=0x7f33cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 0x7f33cc0a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005fd0 tx=0x7f33d4009310 comp rx=0 tx=0).stop 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 msgr2=0x7f33e4100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33d8000c00 tx=0x7f33d800fa00 comp rx=0 tx=0).stop 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 shutdown_connections 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f33cc0a1510 0x7f33cc0a39d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33e41062b0 0x7f33e41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e4068c00 0x7f33e40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 --2- 172.21.15.26:0/2325038021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e40695b0 0x7f33e4100470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 >> 172.21.15.26:0/2325038021 conn(0x7f33e4075e00 msgr2=0x7f33e40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 shutdown_connections 2023-10-06T21:09:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:05.345+0000 7f33e9bcc700 1 -- 172.21.15.26:0/2325038021 wait complete. 2023-10-06T21:09:05.654 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:05 smithi026 ceph-mon[98490]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:09:05.655 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/821955536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:09:05.655 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2500147630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:09:05.655 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3056644843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:09:05.655 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2325038021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:09:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:05 smithi167 ceph-mon[134494]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:09:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/821955536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:09:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2500147630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:09:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3056644843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:09:05.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2325038021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:09:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:05 smithi078 ceph-mon[134896]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:09:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/821955536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-06T21:09:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2500147630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-06T21:09:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3056644843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-06T21:09:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2325038021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-06T21:09:05.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332133 got 219043332136 for osd.7 2023-10-06T21:09:05.898 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 -- 172.21.15.26:0/3753411309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 msgr2=0x7f0128107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 --2- 172.21.15.26:0/3753411309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f0128107370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0124009920 tx=0x7f0124009c30 comp rx=0 tx=0).stop 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 -- 172.21.15.26:0/3753411309 shutdown_connections 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 --2- 172.21.15.26:0/3753411309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 0x7f012810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 --2- 172.21.15.26:0/3753411309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f0128107370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 --2- 172.21.15.26:0/3753411309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.168+0000 7f012fb29700 1 -- 172.21.15.26:0/3753411309 >> 172.21.15.26:0/3753411309 conn(0x7f01280780a0 msgr2=0x7f01280784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.169+0000 7f012fb29700 1 -- 172.21.15.26:0/3753411309 shutdown_connections 2023-10-06T21:09:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.169+0000 7f012fb29700 1 -- 172.21.15.26:0/3753411309 wait complete. 2023-10-06T21:09:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 Processor -- start 2023-10-06T21:09:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 -- start start 2023-10-06T21:09:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 0x7f0128199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f012819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012819e1f0 con 0x7f0128101940 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012819e360 con 0x7f0128106f90 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012fb29700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012819e520 con 0x7f0128100f80 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012d8c5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:06.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012d0c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 0x7f0128199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:06.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012d8c5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41888/0 (socket says 172.21.15.26:41888) 2023-10-06T21:09:06.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.170+0000 7f012d8c5700 1 -- 172.21.15.26:0/430807679 learned_addr learned my addr 172.21.15.26:0/430807679 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:06.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012e0c6700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f012819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:06.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 -- 172.21.15.26:0/430807679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 msgr2=0x7f012819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:06.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f012819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 -- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 msgr2=0x7f0128199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 0x7f0128199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 -- 172.21.15.26:0/430807679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120009d00 con 0x7f0128100f80 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012d8c5700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f012400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012401c070 con 0x7f0128100f80 2023-10-06T21:09:06.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012400bbd0 con 0x7f0128100f80 2023-10-06T21:09:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124009520 con 0x7f0128100f80 2023-10-06T21:09:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.171+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240039b0 con 0x7f0128100f80 2023-10-06T21:09:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.172+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01281a5e90 con 0x7f0128100f80 2023-10-06T21:09:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.172+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012804ebb0 con 0x7f0128100f80 2023-10-06T21:09:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.175+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0124020030 con 0x7f0128100f80 2023-10-06T21:09:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.176+0000 7f011effd700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 0x7f01140a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.176+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01240cdf80 con 0x7f0128100f80 2023-10-06T21:09:06.527 INFO:teuthology.orchestra.run.smithi026.stdout:287762808861 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.176+0000 7f012d0c4700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 0x7f01140a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.177+0000 7f012d0c4700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 0x7f01140a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f012819a850 tx=0x7f0118009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.178+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f01240974d0 con 0x7f0128100f80 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.394+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f0128110e80 con 0x7f0128100f80 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.395+0000 7f011effd700 1 -- 172.21.15.26:0/430807679 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f012409aaf0 con 0x7f0128100f80 2023-10-06T21:09:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 msgr2=0x7f01140a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 0x7f01140a3a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f012819a850 tx=0x7f0118009260 comp rx=0 tx=0).stop 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 msgr2=0x7f0128199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0124009230 tx=0x7f012400fa00 comp rx=0 tx=0).stop 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 shutdown_connections 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128106f90 0x7f012819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0128100f80 0x7f0128199230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01140a1590 0x7f01140a3a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 --2- 172.21.15.26:0/430807679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0128101940 0x7f0128199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 >> 172.21.15.26:0/430807679 conn(0x7f01280780a0 msgr2=0x7f012810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 shutdown_connections 2023-10-06T21:09:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:06.398+0000 7f012fb29700 1 -- 172.21.15.26:0/430807679 wait complete. 2023-10-06T21:09:06.534 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:06 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/430807679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:09:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:06.605 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495656 got 197568495658 for osd.6 2023-10-06T21:09:06.605 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:06 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/430807679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:09:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:06 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/430807679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-06T21:09:07.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070496 got 253403070498 for osd.8 2023-10-06T21:09:07.109 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:07.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574915 for osd.1 2023-10-06T21:09:07.351 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:07.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:07 smithi026 ceph-mon[98490]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-06T21:09:07.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:07 smithi167 ceph-mon[134494]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-06T21:09:07.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:07 smithi078 ceph-mon[134896]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-10-06T21:09:08.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808859 got 287762808861 for osd.9 2023-10-06T21:09:08.090 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:09.004 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:08 smithi026 ceph-mon[98490]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-06T21:09:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:08 smithi078 ceph-mon[134896]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-06T21:09:09.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:08 smithi167 ceph-mon[134494]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-06T21:09:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 -- 172.21.15.26:0/3777267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec10b550 msgr2=0x7effec10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 --2- 172.21.15.26:0/3777267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec10b550 0x7effec10d9a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7effe400b0a0 tx=0x7effe400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 -- 172.21.15.26:0/3777267443 shutdown_connections 2023-10-06T21:09:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 --2- 172.21.15.26:0/3777267443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec10b550 0x7effec10d9a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 --2- 172.21.15.26:0/3777267443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec102170 0x7effec102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 --2- 172.21.15.26:0/3777267443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 0x7effec10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 -- 172.21.15.26:0/3777267443 >> 172.21.15.26:0/3777267443 conn(0x7effec0fb920 msgr2=0x7effec0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 -- 172.21.15.26:0/3777267443 shutdown_connections 2023-10-06T21:09:09.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.311+0000 7efff29da700 1 -- 172.21.15.26:0/3777267443 wait complete. 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.312+0000 7efff29da700 1 Processor -- start 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.312+0000 7efff29da700 1 -- start start 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 0x7effec197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec10b550 0x7effec19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec19c110 con 0x7effec102170 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec19c280 con 0x7effec10b550 2023-10-06T21:09:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff29da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec19c440 con 0x7effec102a90 2023-10-06T21:09:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7effebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7effebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41500/0 (socket says 172.21.15.26:41500) 2023-10-06T21:09:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7effebfff700 1 -- 172.21.15.26:0/4285490571 learned_addr learned my addr 172.21.15.26:0/4285490571 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7effeb7fe700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 0x7effec197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.313+0000 7efff0f77700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec10b550 0x7effec19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 -- 172.21.15.26:0/4285490571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 msgr2=0x7effec197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 0x7effec197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 -- 172.21.15.26:0/4285490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec10b550 msgr2=0x7effec19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec10b550 0x7effec19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 -- 172.21.15.26:0/4285490571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4009d00 con 0x7effec102170 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effebfff700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effdc00d980 tx=0x7effdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc004220 con 0x7effec102170 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec19c720 con 0x7effec102170 2023-10-06T21:09:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effdc007640 con 0x7effec102170 2023-10-06T21:09:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc005690 con 0x7effec102170 2023-10-06T21:09:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.314+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec1a3f80 con 0x7effec102170 2023-10-06T21:09:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.316+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec068b50 con 0x7effec102170 2023-10-06T21:09:09.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.317+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7effdc004750 con 0x7effec102170 2023-10-06T21:09:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.320+0000 7effe97fa700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 0x7effd40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.320+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7effdc0ccbd0 con 0x7effec102170 2023-10-06T21:09:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.320+0000 7effeb7fe700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 0x7effd40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.321+0000 7effeb7fe700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 0x7effd40a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effec198770 tx=0x7effe0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.324+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7effdc096090 con 0x7effec102170 2023-10-06T21:09:09.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 -- 172.21.15.26:0/389880032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 msgr2=0x7f53e40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/389880032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e40734c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009990 tx=0x7f53d4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 -- 172.21.15.26:0/389880032 shutdown_connections 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/389880032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e410d930 0x7f53e410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/389880032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e40734c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/389880032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e4073a00 0x7f53e410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.418+0000 7f53ea7ea700 1 -- 172.21.15.26:0/389880032 >> 172.21.15.26:0/389880032 conn(0x7f53e40780a0 msgr2=0x7f53e40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.419+0000 7f53ea7ea700 1 -- 172.21.15.26:0/389880032 shutdown_connections 2023-10-06T21:09:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.419+0000 7f53ea7ea700 1 -- 172.21.15.26:0/389880032 wait complete. 2023-10-06T21:09:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 Processor -- start 2023-10-06T21:09:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 -- start start 2023-10-06T21:09:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 0x7f53e419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e270 con 0x7f53e4073a00 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e3e0 con 0x7f53e410d930 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53ea7ea700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e5a0 con 0x7f53e40730e0 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.420+0000 7f53e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41520/0 (socket says 172.21.15.26:41520) 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 -- 172.21.15.26:0/3046727953 learned_addr learned my addr 172.21.15.26:0/3046727953 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e8d87700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 0x7f53e419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 -- 172.21.15.26:0/3046727953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 msgr2=0x7f53e41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 -- 172.21.15.26:0/3046727953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 msgr2=0x7f53e419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 0x7f53e419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 -- 172.21.15.26:0/3046727953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40095f0 con 0x7f53e4073a00 2023-10-06T21:09:09.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e8d87700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 0x7f53e419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.421+0000 7f53e37fe700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9a0 tx=0x7f53d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.422+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8009790 con 0x7f53e4073a00 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.422+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53d800cbd0 con 0x7f53e4073a00 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.422+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8018680 con 0x7f53e4073a00 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.422+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e419e880 con 0x7f53e4073a00 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.422+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e41a60e0 con 0x7f53e4073a00 2023-10-06T21:09:09.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.423+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e419eaa0 con 0x7f53e4073a00 2023-10-06T21:09:09.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.427+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f53d800cd40 con 0x7f53e4073a00 2023-10-06T21:09:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.428+0000 7f53e17fa700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 0x7f53cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.428+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f53d8014070 con 0x7f53e4073a00 2023-10-06T21:09:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.428+0000 7f53e3fff700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 0x7f53cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.429+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f53d80968a0 con 0x7f53e4073a00 2023-10-06T21:09:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.429+0000 7f53e3fff700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 0x7f53cc0a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b2e0 tx=0x7f53d4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.551+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7effec1984b0 con 0x7effec102170 2023-10-06T21:09:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.551+0000 7effe97fa700 1 -- 172.21.15.26:0/4285490571 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effdc016020 con 0x7effec102170 2023-10-06T21:09:09.554 INFO:teuthology.orchestra.run.smithi026.stdout:163208757297 2023-10-06T21:09:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.554+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 msgr2=0x7effd40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.554+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 0x7effd40a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7effec198770 tx=0x7effe0009260 comp rx=0 tx=0).stop 2023-10-06T21:09:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.554+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 msgr2=0x7effec197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.554+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effdc00d980 tx=0x7effdc00dc90 comp rx=0 tx=0).stop 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 shutdown_connections 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effec10b550 0x7effec19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7effd40a1590 0x7effd40a3a50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effec102170 0x7effec197150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 --2- 172.21.15.26:0/4285490571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effec102a90 0x7effec197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 >> 172.21.15.26:0/4285490571 conn(0x7effec0fb920 msgr2=0x7effec10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 shutdown_connections 2023-10-06T21:09:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.555+0000 7efff29da700 1 -- 172.21.15.26:0/4285490571 wait complete. 2023-10-06T21:09:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.682+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f53e419a610 con 0x7f53e4073a00 2023-10-06T21:09:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.682+0000 7f53e17fa700 1 -- 172.21.15.26:0/3046727953 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53e419a610 con 0x7f53e4073a00 2023-10-06T21:09:09.686 INFO:teuthology.orchestra.run.smithi026.stdout:107374182458 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 msgr2=0x7f53cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 0x7f53cc0a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b2e0 tx=0x7f53d4005fd0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 msgr2=0x7f53e41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e9a0 tx=0x7f53d800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 shutdown_connections 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53e410d930 0x7f53e419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53e40730e0 0x7f53e41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f53cc0a1510 0x7f53cc0a39d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 --2- 172.21.15.26:0/3046727953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53e4073a00 0x7f53e41997f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 >> 172.21.15.26:0/3046727953 conn(0x7f53e40780a0 msgr2=0x7f53e40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 shutdown_connections 2023-10-06T21:09:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.685+0000 7f53ea7ea700 1 -- 172.21.15.26:0/3046727953 wait complete. 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 -- 172.21.15.26:0/1134972497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c100f80 msgr2=0x7fe52c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 --2- 172.21.15.26:0/1134972497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c100f80 0x7fe52c101400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe520009990 tx=0x7fe520009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 -- 172.21.15.26:0/1134972497 shutdown_connections 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 --2- 172.21.15.26:0/1134972497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c101940 0x7fe52c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 --2- 172.21.15.26:0/1134972497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c106f90 0x7fe52c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 --2- 172.21.15.26:0/1134972497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c100f80 0x7fe52c101400 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 -- 172.21.15.26:0/1134972497 >> 172.21.15.26:0/1134972497 conn(0x7fe52c0780a0 msgr2=0x7fe52c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 -- 172.21.15.26:0/1134972497 shutdown_connections 2023-10-06T21:09:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.858+0000 7fe5318da700 1 -- 172.21.15.26:0/1134972497 wait complete. 2023-10-06T21:09:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.859+0000 7fe5318da700 1 Processor -- start 2023-10-06T21:09:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.859+0000 7fe5318da700 1 -- start start 2023-10-06T21:09:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 0x7fe52c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19e1b0 con 0x7fe52c106f90 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19e320 con 0x7fe52c100f80 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe5318da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c19e4e0 con 0x7fe52c101940 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 0x7fe52c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41950/0 (socket says 172.21.15.26:41950) 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39670/0 (socket says 172.21.15.26:39670) 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 -- 172.21.15.26:0/977786238 learned_addr learned my addr 172.21.15.26:0/977786238 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 -- 172.21.15.26:0/977786238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 msgr2=0x7fe52c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 -- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 msgr2=0x7fe52c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 0x7fe52c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52a7fc700 1 -- 172.21.15.26:0/977786238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5200095f0 con 0x7fe52c101940 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52affd700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.860+0000 7fe52b7fe700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 0x7fe52c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe52a7fc700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098e0 tx=0x7fe520005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52001d070 con 0x7fe52c101940 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe520022470 con 0x7fe52c101940 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520017650 con 0x7fe52c101940 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c19e760 con 0x7fe52c101940 2023-10-06T21:09:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.861+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c1a5e50 con 0x7fe52c101940 2023-10-06T21:09:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.862+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52c04ebb0 con 0x7fe52c101940 2023-10-06T21:09:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.864+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe5200225e0 con 0x7fe52c101940 2023-10-06T21:09:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.865+0000 7fe5308d8700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 0x7fe5140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.865+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5200ce120 con 0x7fe52c101940 2023-10-06T21:09:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.866+0000 7fe52affd700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 0x7fe5140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:09.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.866+0000 7fe52affd700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 0x7fe5140a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009fd0 tx=0x7fe51c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:09.868+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe520097670 con 0x7fe52c101940 2023-10-06T21:09:10.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.081+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe52c110f50 con 0x7fe52c101940 2023-10-06T21:09:10.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.082+0000 7fe5308d8700 1 -- 172.21.15.26:0/977786238 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe52009ac90 con 0x7fe52c101940 2023-10-06T21:09:10.085 INFO:teuthology.orchestra.run.smithi026.stdout:309237645338 2023-10-06T21:09:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 msgr2=0x7fe5140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 0x7fe5140a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009fd0 tx=0x7fe51c009260 comp rx=0 tx=0).stop 2023-10-06T21:09:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 msgr2=0x7fe52c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:10.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098e0 tx=0x7fe520005740 comp rx=0 tx=0).stop 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 shutdown_connections 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5140a1510 0x7fe5140a39d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52c106f90 0x7fe52c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.084+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe52c100f80 0x7fe52c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.085+0000 7fe5318da700 1 --2- 172.21.15.26:0/977786238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe52c101940 0x7fe52c199730 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.085+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 >> 172.21.15.26:0/977786238 conn(0x7fe52c0780a0 msgr2=0x7fe52c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.085+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 shutdown_connections 2023-10-06T21:09:10.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:10.085+0000 7fe5318da700 1 -- 172.21.15.26:0/977786238 wait complete. 2023-10-06T21:09:10.109 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4285490571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:09:10.109 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3046727953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:09:10.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4285490571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:09:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3046727953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:09:10.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4285490571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-06T21:09:10.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3046727953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-06T21:09:10.586 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182455 got 107374182458 for osd.3 2023-10-06T21:09:10.587 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:10.790 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757294 got 163208757297 for osd.5 2023-10-06T21:09:10.790 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:10.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645335 got 309237645338 for osd.10 2023-10-06T21:09:10.890 DEBUG:teuthology.parallel:result is None 2023-10-06T21:09:10.890 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-06T21:09:10.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T21:09:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/977786238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:09:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:10 smithi026 ceph-mon[98490]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:11.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/977786238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:09:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:10 smithi078 ceph-mon[134896]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:11.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/977786238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-06T21:09:11.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:10 smithi167 ceph-mon[134494]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:11.532 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 -- 172.21.15.26:0/1222547071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 msgr2=0x7f6628107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 --2- 172.21.15.26:0/1222547071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f6628107370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66240099c0 tx=0x7f6624009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 -- 172.21.15.26:0/1222547071 shutdown_connections 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 --2- 172.21.15.26:0/1222547071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628101940 0x7f662810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 --2- 172.21.15.26:0/1222547071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f6628107370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 --2- 172.21.15.26:0/1222547071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628100f80 0x7f6628101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 -- 172.21.15.26:0/1222547071 >> 172.21.15.26:0/1222547071 conn(0x7f66280780a0 msgr2=0x7f66280784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 -- 172.21.15.26:0/1222547071 shutdown_connections 2023-10-06T21:09:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.316+0000 7f66301ed700 1 -- 172.21.15.26:0/1222547071 wait complete. 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.317+0000 7f66301ed700 1 Processor -- start 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 -- start start 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628101940 0x7f6628199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f662819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819e110 con 0x7f6628106f90 2023-10-06T21:09:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819e280 con 0x7f6628101940 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f66301ed700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662819e440 con 0x7f6628100f80 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f662df89700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f662e78a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f662819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f662df89700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59704/0 (socket says 172.21.15.26:59704) 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f662df89700 1 -- 172.21.15.26:0/4231269758 learned_addr learned my addr 172.21.15.26:0/4231269758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.318+0000 7f662d788700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628101940 0x7f6628199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 -- 172.21.15.26:0/4231269758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 msgr2=0x7f6628199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 -- 172.21.15.26:0/4231269758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628101940 msgr2=0x7f6628199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628101940 0x7f6628199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 -- 172.21.15.26:0/4231269758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009520 con 0x7f6628106f90 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662df89700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f662e78a700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f662819db00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6620007f10 tx=0x7f662000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620004db0 con 0x7f6628106f90 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662000f040 con 0x7f6628106f90 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620005830 con 0x7f6628106f90 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66240095f0 con 0x7f6628106f90 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.319+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281a5db0 con 0x7f6628106f90 2023-10-06T21:09:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.321+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662804ebb0 con 0x7f6628106f90 2023-10-06T21:09:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.323+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f662000db20 con 0x7f6628106f90 2023-10-06T21:09:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.324+0000 7f661effd700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 0x7f66140a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.324+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66200cc690 con 0x7f6628106f90 2023-10-06T21:09:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.324+0000 7f662df89700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 0x7f66140a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.325+0000 7f662df89700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 0x7f66140a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6624004800 tx=0x7f662401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:13.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.328+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f662004f100 con 0x7f6628106f90 2023-10-06T21:09:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.536+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66281110a0 con 0x7f66140a1590 2023-10-06T21:09:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.548+0000 7f661effd700 1 -- 172.21.15.26:0/4231269758 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341240 (secure 0 0 0) 0x7f66281110a0 con 0x7f66140a1590 2023-10-06T21:09:13.559 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.554+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 msgr2=0x7f66140a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.554+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 0x7f66140a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6624004800 tx=0x7f662401a040 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.554+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 msgr2=0x7f662819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.554+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f662819db00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6620007f10 tx=0x7f662000d3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 shutdown_connections 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f66140a1590 0x7f66140a3a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6628106f90 0x7f662819db00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6628100f80 0x7f6628199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 --2- 172.21.15.26:0/4231269758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6628101940 0x7f6628199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 >> 172.21.15.26:0/4231269758 conn(0x7f66280780a0 msgr2=0x7f662810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 shutdown_connections 2023-10-06T21:09:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:13.555+0000 7f66301ed700 1 -- 172.21.15.26:0/4231269758 wait complete. 2023-10-06T21:09:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:09:13.713 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:13 smithi026 ceph-mon[98490]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:13.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:13 smithi167 ceph-mon[134494]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:13.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:13 smithi078 ceph-mon[134896]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:14.013 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-10-06T21:09:12.382431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654164,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":435,"num_read_kb":330,"num_write":333,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":3009370,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337500,"kb_used_data":13256,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604324,"statfs":{"total":1151940427776,"available":1151594827776,"internally_reserved":0,"allocated":13574144,"data_stored":7433933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440999+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:08:44.440999+0000","last_peered":"2023-10-06T21:08:44.440999+0000","last_clean":"2023-10-06T21:08:44.440999+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:08:44.440999+0000","last_undegraded":"2023-10-06T21:08:44.440999+0000","last_fullsized":"2023-10-06T21:08:44.440999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440516+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:08:44.440516+0000","last_peered":"2023-10-06T21:08:44.440516+0000","last_clean":"2023-10-06T21:08:44.440516+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:08:44.440516+0000","last_undegraded":"2023-10-06T21:08:44.440516+0000","last_fullsized":"2023-10-06T21:08:44.440516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440041+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:08:44.440041+0000","last_peered":"2023-10-06T21:08:44.440041+0000","last_clean":"2023-10-06T21:08:44.440041+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:08:44.440041+0000","last_undegraded":"2023-10-06T21:08:44.440041+0000","last_fullsized":"2023-10-06T21:08:44.440041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.451527+0000","last_change":"2023-10-06T21:08:45.451527+0000","last_active":"2023-10-06T21:08:45.451527+0000","last_peered":"2023-10-06T21:08:45.451527+0000","last_clean":"2023-10-06T21:08:45.451527+0000","last_became_active":"2023-10-06T21:08:45.451219+0000","last_became_peered":"2023-10-06T21:08:45.451219+0000","last_unstale":"2023-10-06T21:08:45.451527+0000","last_undegraded":"2023-10-06T21:08:45.451527+0000","last_fullsized":"2023-10-06T21:08:45.451527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:03:42.551771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440485+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:08:44.440485+0000","last_peered":"2023-10-06T21:08:44.440485+0000","last_clean":"2023-10-06T21:08:44.440485+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:08:44.440485+0000","last_undegraded":"2023-10-06T21:08:44.440485+0000","last_fullsized":"2023-10-06T21:08:44.440485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444016+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:08:44.444016+0000","last_peered":"2023-10-06T21:08:44.444016+0000","last_clean":"2023-10-06T21:08:44.444016+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:08:44.444016+0000","last_undegraded":"2023-10-06T21:08:44.444016+0000","last_fullsized":"2023-10-06T21:08:44.444016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441482+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:08:44.441482+0000","last_peered":"2023-10-06T21:08:44.441482+0000","last_clean":"2023-10-06T21:08:44.441482+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:08:44.441482+0000","last_undegraded":"2023-10-06T21:08:44.441482+0000","last_fullsized":"2023-10-06T21:08:44.441482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434436+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:08:44.434436+0000","last_peered":"2023-10-06T21:08:44.434436+0000","last_clean":"2023-10-06T21:08:44.434436+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:08:44.434436+0000","last_undegraded":"2023-10-06T21:08:44.434436+0000","last_fullsized":"2023-10-06T21:08:44.434436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.451859+0000","last_change":"2023-10-06T21:08:45.451859+0000","last_active":"2023-10-06T21:08:45.451859+0000","last_peered":"2023-10-06T21:08:45.451859+0000","last_clean":"2023-10-06T21:08:45.451859+0000","last_became_active":"2023-10-06T21:08:45.451525+0000","last_became_peered":"2023-10-06T21:08:45.451525+0000","last_unstale":"2023-10-06T21:08:45.451859+0000","last_undegraded":"2023-10-06T21:08:45.451859+0000","last_fullsized":"2023-10-06T21:08:45.451859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:52.350672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444757+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:08:44.444757+0000","last_peered":"2023-10-06T21:08:44.444757+0000","last_clean":"2023-10-06T21:08:44.444757+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:08:44.444757+0000","last_undegraded":"2023-10-06T21:08:44.444757+0000","last_fullsized":"2023-10-06T21:08:44.444757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441985+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:08:44.441985+0000","last_peered":"2023-10-06T21:08:44.441985+0000","last_clean":"2023-10-06T21:08:44.441985+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:08:44.441985+0000","last_undegraded":"2023-10-06T21:08:44.441985+0000","last_fullsized":"2023-10-06T21:08:44.441985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435762+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:08:44.435762+0000","last_peered":"2023-10-06T21:08:44.435762+0000","last_clean":"2023-10-06T21:08:44.435762+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:08:44.435762+0000","last_undegraded":"2023-10-06T21:08:44.435762+0000","last_fullsized":"2023-10-06T21:08:44.435762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437179+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:08:44.437179+0000","last_peered":"2023-10-06T21:08:44.437179+0000","last_clean":"2023-10-06T21:08:44.437179+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:08:44.437179+0000","last_undegraded":"2023-10-06T21:08:44.437179+0000","last_fullsized":"2023-10-06T21:08:44.437179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439448+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:08:44.439448+0000","last_peered":"2023-10-06T21:08:44.439448+0000","last_clean":"2023-10-06T21:08:44.439448+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:08:44.439448+0000","last_undegraded":"2023-10-06T21:08:44.439448+0000","last_fullsized":"2023-10-06T21:08:44.439448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439141+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:08:44.439141+0000","last_peered":"2023-10-06T21:08:44.439141+0000","last_clean":"2023-10-06T21:08:44.439141+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:08:44.439141+0000","last_undegraded":"2023-10-06T21:08:44.439141+0000","last_fullsized":"2023-10-06T21:08:44.439141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443021+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:08:44.443021+0000","last_peered":"2023-10-06T21:08:44.443021+0000","last_clean":"2023-10-06T21:08:44.443021+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:08:44.443021+0000","last_undegraded":"2023-10-06T21:08:44.443021+0000","last_fullsized":"2023-10-06T21:08:44.443021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440971+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:08:44.440971+0000","last_peered":"2023-10-06T21:08:44.440971+0000","last_clean":"2023-10-06T21:08:44.440971+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:08:44.440971+0000","last_undegraded":"2023-10-06T21:08:44.440971+0000","last_fullsized":"2023-10-06T21:08:44.440971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440030+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:08:44.440030+0000","last_peered":"2023-10-06T21:08:44.440030+0000","last_clean":"2023-10-06T21:08:44.440030+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:08:44.440030+0000","last_undegraded":"2023-10-06T21:08:44.440030+0000","last_fullsized":"2023-10-06T21:08:44.440030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443577+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:08:44.443577+0000","last_peered":"2023-10-06T21:08:44.443577+0000","last_clean":"2023-10-06T21:08:44.443577+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:08:44.443577+0000","last_undegraded":"2023-10-06T21:08:44.443577+0000","last_fullsized":"2023-10-06T21:08:44.443577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439306+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:44.439306+0000","last_peered":"2023-10-06T21:08:44.439306+0000","last_clean":"2023-10-06T21:08:44.439306+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:44.439306+0000","last_undegraded":"2023-10-06T21:08:44.439306+0000","last_fullsized":"2023-10-06T21:08:44.439306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435729+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:08:44.435729+0000","last_peered":"2023-10-06T21:08:44.435729+0000","last_clean":"2023-10-06T21:08:44.435729+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:08:44.435729+0000","last_undegraded":"2023-10-06T21:08:44.435729+0000","last_fullsized":"2023-10-06T21:08:44.435729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435884+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:44.435884+0000","last_peered":"2023-10-06T21:08:44.435884+0000","last_clean":"2023-10-06T21:08:44.435884+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:44.435884+0000","last_undegraded":"2023-10-06T21:08:44.435884+0000","last_fullsized":"2023-10-06T21:08:44.435884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440152+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:08:44.440152+0000","last_peered":"2023-10-06T21:08:44.440152+0000","last_clean":"2023-10-06T21:08:44.440152+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:08:44.440152+0000","last_undegraded":"2023-10-06T21:08:44.440152+0000","last_fullsized":"2023-10-06T21:08:44.440152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440673+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:44.440673+0000","last_peered":"2023-10-06T21:08:44.440673+0000","last_clean":"2023-10-06T21:08:44.440673+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:44.440673+0000","last_undegraded":"2023-10-06T21:08:44.440673+0000","last_fullsized":"2023-10-06T21:08:44.440673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443250+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:08:44.443250+0000","last_peered":"2023-10-06T21:08:44.443250+0000","last_clean":"2023-10-06T21:08:44.443250+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:08:44.443250+0000","last_undegraded":"2023-10-06T21:08:44.443250+0000","last_fullsized":"2023-10-06T21:08:44.443250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441604+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:44.441604+0000","last_peered":"2023-10-06T21:08:44.441604+0000","last_clean":"2023-10-06T21:08:44.441604+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:44.441604+0000","last_undegraded":"2023-10-06T21:08:44.441604+0000","last_fullsized":"2023-10-06T21:08:44.441604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438645+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:08:44.438645+0000","last_peered":"2023-10-06T21:08:44.438645+0000","last_clean":"2023-10-06T21:08:44.438645+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:08:44.438645+0000","last_undegraded":"2023-10-06T21:08:44.438645+0000","last_fullsized":"2023-10-06T21:08:44.438645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":75,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:10.938008+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:09:10.938008+0000","last_peered":"2023-10-06T21:09:10.938008+0000","last_clean":"2023-10-06T21:09:10.938008+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:09:10.938008+0000","last_undegraded":"2023-10-06T21:09:10.938008+0000","last_fullsized":"2023-10-06T21:09:10.938008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440639+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:08:44.440639+0000","last_peered":"2023-10-06T21:08:44.440639+0000","last_clean":"2023-10-06T21:08:44.440639+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:08:44.440639+0000","last_undegraded":"2023-10-06T21:08:44.440639+0000","last_fullsized":"2023-10-06T21:08:44.440639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443912+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:44.443912+0000","last_peered":"2023-10-06T21:08:44.443912+0000","last_clean":"2023-10-06T21:08:44.443912+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:44.443912+0000","last_undegraded":"2023-10-06T21:08:44.443912+0000","last_fullsized":"2023-10-06T21:08:44.443912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438326+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:08:44.438326+0000","last_peered":"2023-10-06T21:08:44.438326+0000","last_clean":"2023-10-06T21:08:44.438326+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:08:44.438326+0000","last_undegraded":"2023-10-06T21:08:44.438326+0000","last_fullsized":"2023-10-06T21:08:44.438326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436810+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:44.436810+0000","last_peered":"2023-10-06T21:08:44.436810+0000","last_clean":"2023-10-06T21:08:44.436810+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:44.436810+0000","last_undegraded":"2023-10-06T21:08:44.436810+0000","last_fullsized":"2023-10-06T21:08:44.436810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434382+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:08:44.434382+0000","last_peered":"2023-10-06T21:08:44.434382+0000","last_clean":"2023-10-06T21:08:44.434382+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:08:44.434382+0000","last_undegraded":"2023-10-06T21:08:44.434382+0000","last_fullsized":"2023-10-06T21:08:44.434382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438712+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:08:44.438712+0000","last_peered":"2023-10-06T21:08:44.438712+0000","last_clean":"2023-10-06T21:08:44.438712+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:08:44.438712+0000","last_undegraded":"2023-10-06T21:08:44.438712+0000","last_fullsized":"2023-10-06T21:08:44.438712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:05.929846+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:09:05.929846+0000","last_peered":"2023-10-06T21:09:05.929846+0000","last_clean":"2023-10-06T21:09:05.929846+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:09:05.929846+0000","last_undegraded":"2023-10-06T21:09:05.929846+0000","last_fullsized":"2023-10-06T21:09:05.929846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443619+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:08:44.443619+0000","last_peered":"2023-10-06T21:08:44.443619+0000","last_clean":"2023-10-06T21:08:44.443619+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:08:44.443619+0000","last_undegraded":"2023-10-06T21:08:44.443619+0000","last_fullsized":"2023-10-06T21:08:44.443619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441919+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:44.441919+0000","last_peered":"2023-10-06T21:08:44.441919+0000","last_clean":"2023-10-06T21:08:44.441919+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:44.441919+0000","last_undegraded":"2023-10-06T21:08:44.441919+0000","last_fullsized":"2023-10-06T21:08:44.441919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440080+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:08:44.440080+0000","last_peered":"2023-10-06T21:08:44.440080+0000","last_clean":"2023-10-06T21:08:44.440080+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:08:44.440080+0000","last_undegraded":"2023-10-06T21:08:44.440080+0000","last_fullsized":"2023-10-06T21:08:44.440080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438756+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:44.438756+0000","last_peered":"2023-10-06T21:08:44.438756+0000","last_clean":"2023-10-06T21:08:44.438756+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:44.438756+0000","last_undegraded":"2023-10-06T21:08:44.438756+0000","last_fullsized":"2023-10-06T21:08:44.438756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443447+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:08:44.443447+0000","last_peered":"2023-10-06T21:08:44.443447+0000","last_clean":"2023-10-06T21:08:44.443447+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:08:44.443447+0000","last_undegraded":"2023-10-06T21:08:44.443447+0000","last_fullsized":"2023-10-06T21:08:44.443447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":160,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:06.653839+0000","last_change":"2023-10-06T21:08:45.622755+0000","last_active":"2023-10-06T21:09:06.653839+0000","last_peered":"2023-10-06T21:09:06.653839+0000","last_clean":"2023-10-06T21:09:06.653839+0000","last_became_active":"2023-10-06T21:08:45.614102+0000","last_became_peered":"2023-10-06T21:08:45.614102+0000","last_unstale":"2023-10-06T21:09:06.653839+0000","last_undegraded":"2023-10-06T21:09:06.653839+0000","last_fullsized":"2023-10-06T21:09:06.653839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:17:14.262961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":98,"num_write":28,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57508,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440154+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:08:44.440154+0000","last_peered":"2023-10-06T21:08:44.440154+0000","last_clean":"2023-10-06T21:08:44.440154+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:08:44.440154+0000","last_undegraded":"2023-10-06T21:08:44.440154+0000","last_fullsized":"2023-10-06T21:08:44.440154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443552+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:44.443552+0000","last_peered":"2023-10-06T21:08:44.443552+0000","last_clean":"2023-10-06T21:08:44.443552+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:44.443552+0000","last_undegraded":"2023-10-06T21:08:44.443552+0000","last_fullsized":"2023-10-06T21:08:44.443552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436976+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:08:44.436976+0000","last_peered":"2023-10-06T21:08:44.436976+0000","last_clean":"2023-10-06T21:08:44.436976+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:08:44.436976+0000","last_undegraded":"2023-10-06T21:08:44.436976+0000","last_fullsized":"2023-10-06T21:08:44.436976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438078+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:44.438078+0000","last_peered":"2023-10-06T21:08:44.438078+0000","last_clean":"2023-10-06T21:08:44.438078+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:44.438078+0000","last_undegraded":"2023-10-06T21:08:44.438078+0000","last_fullsized":"2023-10-06T21:08:44.438078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441762+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:08:44.441762+0000","last_peered":"2023-10-06T21:08:44.441762+0000","last_clean":"2023-10-06T21:08:44.441762+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:08:44.441762+0000","last_undegraded":"2023-10-06T21:08:44.441762+0000","last_fullsized":"2023-10-06T21:08:44.441762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:05.929991+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:09:05.929991+0000","last_peered":"2023-10-06T21:09:05.929991+0000","last_clean":"2023-10-06T21:09:05.929991+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:09:05.929991+0000","last_undegraded":"2023-10-06T21:09:05.929991+0000","last_fullsized":"2023-10-06T21:09:05.929991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.447758+0000","last_change":"2023-10-06T21:08:45.447758+0000","last_active":"2023-10-06T21:08:45.447758+0000","last_peered":"2023-10-06T21:08:45.447758+0000","last_clean":"2023-10-06T21:08:45.447758+0000","last_became_active":"2023-10-06T21:08:45.447337+0000","last_became_peered":"2023-10-06T21:08:45.447337+0000","last_unstale":"2023-10-06T21:08:45.447758+0000","last_undegraded":"2023-10-06T21:08:45.447758+0000","last_fullsized":"2023-10-06T21:08:45.447758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:48:29.806108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439954+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:44.439954+0000","last_peered":"2023-10-06T21:08:44.439954+0000","last_clean":"2023-10-06T21:08:44.439954+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:44.439954+0000","last_undegraded":"2023-10-06T21:08:44.439954+0000","last_fullsized":"2023-10-06T21:08:44.439954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440701+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:44.440701+0000","last_peered":"2023-10-06T21:08:44.440701+0000","last_clean":"2023-10-06T21:08:44.440701+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:44.440701+0000","last_undegraded":"2023-10-06T21:08:44.440701+0000","last_fullsized":"2023-10-06T21:08:44.440701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440917+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:08:44.440917+0000","last_peered":"2023-10-06T21:08:44.440917+0000","last_clean":"2023-10-06T21:08:44.440917+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:08:44.440917+0000","last_undegraded":"2023-10-06T21:08:44.440917+0000","last_fullsized":"2023-10-06T21:08:44.440917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440260+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:44.440260+0000","last_peered":"2023-10-06T21:08:44.440260+0000","last_clean":"2023-10-06T21:08:44.440260+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:44.440260+0000","last_undegraded":"2023-10-06T21:08:44.440260+0000","last_fullsized":"2023-10-06T21:08:44.440260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444652+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:08:44.444652+0000","last_peered":"2023-10-06T21:08:44.444652+0000","last_clean":"2023-10-06T21:08:44.444652+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:08:44.444652+0000","last_undegraded":"2023-10-06T21:08:44.444652+0000","last_fullsized":"2023-10-06T21:08:44.444652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441637+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:44.441637+0000","last_peered":"2023-10-06T21:08:44.441637+0000","last_clean":"2023-10-06T21:08:44.441637+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:44.441637+0000","last_undegraded":"2023-10-06T21:08:44.441637+0000","last_fullsized":"2023-10-06T21:08:44.441637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444417+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:08:44.444417+0000","last_peered":"2023-10-06T21:08:44.444417+0000","last_clean":"2023-10-06T21:08:44.444417+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:08:44.444417+0000","last_undegraded":"2023-10-06T21:08:44.444417+0000","last_fullsized":"2023-10-06T21:08:44.444417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.453278+0000","last_change":"2023-10-06T21:08:45.453278+0000","last_active":"2023-10-06T21:08:45.453278+0000","last_peered":"2023-10-06T21:08:45.453278+0000","last_clean":"2023-10-06T21:08:45.453278+0000","last_became_active":"2023-10-06T21:08:45.452892+0000","last_became_peered":"2023-10-06T21:08:45.452892+0000","last_unstale":"2023-10-06T21:08:45.453278+0000","last_undegraded":"2023-10-06T21:08:45.453278+0000","last_fullsized":"2023-10-06T21:08:45.453278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:54.604308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:07.932254+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:09:07.932254+0000","last_peered":"2023-10-06T21:09:07.932254+0000","last_clean":"2023-10-06T21:09:07.932254+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:09:07.932254+0000","last_undegraded":"2023-10-06T21:09:07.932254+0000","last_fullsized":"2023-10-06T21:09:07.932254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434953+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:08:44.434953+0000","last_peered":"2023-10-06T21:08:44.434953+0000","last_clean":"2023-10-06T21:08:44.434953+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:08:44.434953+0000","last_undegraded":"2023-10-06T21:08:44.434953+0000","last_fullsized":"2023-10-06T21:08:44.434953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436314+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:44.436314+0000","last_peered":"2023-10-06T21:08:44.436314+0000","last_clean":"2023-10-06T21:08:44.436314+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:44.436314+0000","last_undegraded":"2023-10-06T21:08:44.436314+0000","last_fullsized":"2023-10-06T21:08:44.436314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442005+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:44.442005+0000","last_peered":"2023-10-06T21:08:44.442005+0000","last_clean":"2023-10-06T21:08:44.442005+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:44.442005+0000","last_undegraded":"2023-10-06T21:08:44.442005+0000","last_fullsized":"2023-10-06T21:08:44.442005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436613+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:08:44.436613+0000","last_peered":"2023-10-06T21:08:44.436613+0000","last_clean":"2023-10-06T21:08:44.436613+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:08:44.436613+0000","last_undegraded":"2023-10-06T21:08:44.436613+0000","last_fullsized":"2023-10-06T21:08:44.436613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442183+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:44.442183+0000","last_peered":"2023-10-06T21:08:44.442183+0000","last_clean":"2023-10-06T21:08:44.442183+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:44.442183+0000","last_undegraded":"2023-10-06T21:08:44.442183+0000","last_fullsized":"2023-10-06T21:08:44.442183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.454229+0000","last_change":"2023-10-06T21:08:45.453710+0000","last_active":"2023-10-06T21:08:45.454229+0000","last_peered":"2023-10-06T21:08:45.454229+0000","last_clean":"2023-10-06T21:08:45.454229+0000","last_became_active":"2023-10-06T21:08:45.447386+0000","last_became_peered":"2023-10-06T21:08:45.447386+0000","last_unstale":"2023-10-06T21:08:45.454229+0000","last_undegraded":"2023-10-06T21:08:45.454229+0000","last_fullsized":"2023-10-06T21:08:45.454229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:27.659601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437313+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:08:44.437313+0000","last_peered":"2023-10-06T21:08:44.437313+0000","last_clean":"2023-10-06T21:08:44.437313+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:08:44.437313+0000","last_undegraded":"2023-10-06T21:08:44.437313+0000","last_fullsized":"2023-10-06T21:08:44.437313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440135+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:44.440135+0000","last_peered":"2023-10-06T21:08:44.440135+0000","last_clean":"2023-10-06T21:08:44.440135+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:44.440135+0000","last_undegraded":"2023-10-06T21:08:44.440135+0000","last_fullsized":"2023-10-06T21:08:44.440135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435642+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:08:44.435642+0000","last_peered":"2023-10-06T21:08:44.435642+0000","last_clean":"2023-10-06T21:08:44.435642+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:08:44.435642+0000","last_undegraded":"2023-10-06T21:08:44.435642+0000","last_fullsized":"2023-10-06T21:08:44.435642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436021+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:44.436021+0000","last_peered":"2023-10-06T21:08:44.436021+0000","last_clean":"2023-10-06T21:08:44.436021+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:44.436021+0000","last_undegraded":"2023-10-06T21:08:44.436021+0000","last_fullsized":"2023-10-06T21:08:44.436021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.613497+0000","last_change":"2023-10-06T21:08:45.613497+0000","last_active":"2023-10-06T21:08:45.613497+0000","last_peered":"2023-10-06T21:08:45.613497+0000","last_clean":"2023-10-06T21:08:45.613497+0000","last_became_active":"2023-10-06T21:08:45.613012+0000","last_became_peered":"2023-10-06T21:08:45.613012+0000","last_unstale":"2023-10-06T21:08:45.613497+0000","last_undegraded":"2023-10-06T21:08:45.613497+0000","last_fullsized":"2023-10-06T21:08:45.613497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:04:42.351299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443069+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:44.443069+0000","last_peered":"2023-10-06T21:08:44.443069+0000","last_clean":"2023-10-06T21:08:44.443069+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:44.443069+0000","last_undegraded":"2023-10-06T21:08:44.443069+0000","last_fullsized":"2023-10-06T21:08:44.443069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439861+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:08:44.439861+0000","last_peered":"2023-10-06T21:08:44.439861+0000","last_clean":"2023-10-06T21:08:44.439861+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:08:44.439861+0000","last_undegraded":"2023-10-06T21:08:44.439861+0000","last_fullsized":"2023-10-06T21:08:44.439861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434716+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:44.434716+0000","last_peered":"2023-10-06T21:08:44.434716+0000","last_clean":"2023-10-06T21:08:44.434716+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:44.434716+0000","last_undegraded":"2023-10-06T21:08:44.434716+0000","last_fullsized":"2023-10-06T21:08:44.434716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439000+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:08:44.439000+0000","last_peered":"2023-10-06T21:08:44.439000+0000","last_clean":"2023-10-06T21:08:44.439000+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:08:44.439000+0000","last_undegraded":"2023-10-06T21:08:44.439000+0000","last_fullsized":"2023-10-06T21:08:44.439000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.457037+0000","last_change":"2023-10-06T21:08:45.456629+0000","last_active":"2023-10-06T21:08:45.457037+0000","last_peered":"2023-10-06T21:08:45.457037+0000","last_clean":"2023-10-06T21:08:45.457037+0000","last_became_active":"2023-10-06T21:08:45.449565+0000","last_became_peered":"2023-10-06T21:08:45.449565+0000","last_unstale":"2023-10-06T21:08:45.457037+0000","last_undegraded":"2023-10-06T21:08:45.457037+0000","last_fullsized":"2023-10-06T21:08:45.457037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:23:54.912161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435249+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:08:44.435249+0000","last_peered":"2023-10-06T21:08:44.435249+0000","last_clean":"2023-10-06T21:08:44.435249+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:08:44.435249+0000","last_undegraded":"2023-10-06T21:08:44.435249+0000","last_fullsized":"2023-10-06T21:08:44.435249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440584+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:44.440584+0000","last_peered":"2023-10-06T21:08:44.440584+0000","last_clean":"2023-10-06T21:08:44.440584+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:44.440584+0000","last_undegraded":"2023-10-06T21:08:44.440584+0000","last_fullsized":"2023-10-06T21:08:44.440584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441769+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:08:44.441769+0000","last_peered":"2023-10-06T21:08:44.441769+0000","last_clean":"2023-10-06T21:08:44.441769+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:08:44.441769+0000","last_undegraded":"2023-10-06T21:08:44.441769+0000","last_fullsized":"2023-10-06T21:08:44.441769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444225+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:44.444225+0000","last_peered":"2023-10-06T21:08:44.444225+0000","last_clean":"2023-10-06T21:08:44.444225+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:44.444225+0000","last_undegraded":"2023-10-06T21:08:44.444225+0000","last_fullsized":"2023-10-06T21:08:44.444225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442079+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:08:44.442079+0000","last_peered":"2023-10-06T21:08:44.442079+0000","last_clean":"2023-10-06T21:08:44.442079+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:08:44.442079+0000","last_undegraded":"2023-10-06T21:08:44.442079+0000","last_fullsized":"2023-10-06T21:08:44.442079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438547+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:44.438547+0000","last_peered":"2023-10-06T21:08:44.438547+0000","last_clean":"2023-10-06T21:08:44.438547+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:44.438547+0000","last_undegraded":"2023-10-06T21:08:44.438547+0000","last_fullsized":"2023-10-06T21:08:44.438547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441789+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:08:44.441789+0000","last_peered":"2023-10-06T21:08:44.441789+0000","last_clean":"2023-10-06T21:08:44.441789+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:08:44.441789+0000","last_undegraded":"2023-10-06T21:08:44.441789+0000","last_fullsized":"2023-10-06T21:08:44.441789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437500+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:44.437500+0000","last_peered":"2023-10-06T21:08:44.437500+0000","last_clean":"2023-10-06T21:08:44.437500+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:44.437500+0000","last_undegraded":"2023-10-06T21:08:44.437500+0000","last_fullsized":"2023-10-06T21:08:44.437500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442142+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:08:44.442142+0000","last_peered":"2023-10-06T21:08:44.442142+0000","last_clean":"2023-10-06T21:08:44.442142+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:08:44.442142+0000","last_undegraded":"2023-10-06T21:08:44.442142+0000","last_fullsized":"2023-10-06T21:08:44.442142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439097+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:08:44.439097+0000","last_peered":"2023-10-06T21:08:44.439097+0000","last_clean":"2023-10-06T21:08:44.439097+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:08:44.439097+0000","last_undegraded":"2023-10-06T21:08:44.439097+0000","last_fullsized":"2023-10-06T21:08:44.439097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443651+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:08:44.443651+0000","last_peered":"2023-10-06T21:08:44.443651+0000","last_clean":"2023-10-06T21:08:44.443651+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:08:44.443651+0000","last_undegraded":"2023-10-06T21:08:44.443651+0000","last_fullsized":"2023-10-06T21:08:44.443651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443339+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:08:44.443339+0000","last_peered":"2023-10-06T21:08:44.443339+0000","last_clean":"2023-10-06T21:08:44.443339+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:08:44.443339+0000","last_undegraded":"2023-10-06T21:08:44.443339+0000","last_fullsized":"2023-10-06T21:08:44.443339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.448913+0000","last_change":"2023-10-06T21:08:45.448913+0000","last_active":"2023-10-06T21:08:45.448913+0000","last_peered":"2023-10-06T21:08:45.448913+0000","last_clean":"2023-10-06T21:08:45.448913+0000","last_became_active":"2023-10-06T21:08:45.448485+0000","last_became_peered":"2023-10-06T21:08:45.448485+0000","last_unstale":"2023-10-06T21:08:45.448913+0000","last_undegraded":"2023-10-06T21:08:45.448913+0000","last_fullsized":"2023-10-06T21:08:45.448913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:42:07.732211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437384+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:08:44.437384+0000","last_peered":"2023-10-06T21:08:44.437384+0000","last_clean":"2023-10-06T21:08:44.437384+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:08:44.437384+0000","last_undegraded":"2023-10-06T21:08:44.437384+0000","last_fullsized":"2023-10-06T21:08:44.437384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441023+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:08:44.441023+0000","last_peered":"2023-10-06T21:08:44.441023+0000","last_clean":"2023-10-06T21:08:44.441023+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:08:44.441023+0000","last_undegraded":"2023-10-06T21:08:44.441023+0000","last_fullsized":"2023-10-06T21:08:44.441023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443842+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:08:44.443842+0000","last_peered":"2023-10-06T21:08:44.443842+0000","last_clean":"2023-10-06T21:08:44.443842+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:08:44.443842+0000","last_undegraded":"2023-10-06T21:08:44.443842+0000","last_fullsized":"2023-10-06T21:08:44.443842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435568+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:08:44.435568+0000","last_peered":"2023-10-06T21:08:44.435568+0000","last_clean":"2023-10-06T21:08:44.435568+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:08:44.435568+0000","last_undegraded":"2023-10-06T21:08:44.435568+0000","last_fullsized":"2023-10-06T21:08:44.435568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437894+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:08:44.437894+0000","last_peered":"2023-10-06T21:08:44.437894+0000","last_clean":"2023-10-06T21:08:44.437894+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:08:44.437894+0000","last_undegraded":"2023-10-06T21:08:44.437894+0000","last_fullsized":"2023-10-06T21:08:44.437894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434567+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:08:44.434567+0000","last_peered":"2023-10-06T21:08:44.434567+0000","last_clean":"2023-10-06T21:08:44.434567+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:08:44.434567+0000","last_undegraded":"2023-10-06T21:08:44.434567+0000","last_fullsized":"2023-10-06T21:08:44.434567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443365+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:08:44.443365+0000","last_peered":"2023-10-06T21:08:44.443365+0000","last_clean":"2023-10-06T21:08:44.443365+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:08:44.443365+0000","last_undegraded":"2023-10-06T21:08:44.443365+0000","last_fullsized":"2023-10-06T21:08:44.443365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437157+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:08:44.437157+0000","last_peered":"2023-10-06T21:08:44.437157+0000","last_clean":"2023-10-06T21:08:44.437157+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:08:44.437157+0000","last_undegraded":"2023-10-06T21:08:44.437157+0000","last_fullsized":"2023-10-06T21:08:44.437157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439923+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:08:44.439923+0000","last_peered":"2023-10-06T21:08:44.439923+0000","last_clean":"2023-10-06T21:08:44.439923+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:08:44.439923+0000","last_undegraded":"2023-10-06T21:08:44.439923+0000","last_fullsized":"2023-10-06T21:08:44.439923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63550,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":299,"num_read_kb":211,"num_write":150,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":57530,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383703,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":456585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.80100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.71499999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1107600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.752}]},{"osd":6,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.62}]},{"osd":9,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.76800000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.81799999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808863,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.83699999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.996,"5min":1.006,"15min":1.006},"last":0.91400000000000003}]},{"osd":3,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.03}]},{"osd":4,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0149999999999999,"5min":1.198,"15min":1.198},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83799999999999997,"15min":0.84599999999999997},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":1.085},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.399,"5min":1.474,"15min":1.474},"last":1.1160000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.173,"5min":1.226,"15min":1.226},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4770000000000001,"5min":1.542,"15min":1.542},"last":1.0720000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88100000000000001,"15min":0.879},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.878,"5min":0.95199999999999996,"15min":0.96399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2310000000000001,"5min":1.365,"15min":1.365},"last":1.056}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.95799999999999996,"15min":0.94999999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":1.097},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1359999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.96899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":8,"up_from":59,"seq":253403070500,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.5,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77200000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.754},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.87,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81799999999999995,"5min":1.087,"15min":1.087},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.038,"15min":1.038},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90800000000000003,"5min":1.155,"15min":1.155},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.83599999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.747},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.95999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1049044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.747},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.76500000000000001,"5min":1.002,"15min":1.002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81200000000000006,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97399999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.87}]},{"osd":9,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.008,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89200000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":456194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.57899999999999996,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":5.8620000000000001,"5min":5.8620000000000001,"15min":5.8620000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.67900000000000005},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.44800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73899999999999999,"15min":0.72199999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.753,"15min":0.748},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.9290000000000003,"5min":5.9290000000000003,"15min":5.9290000000000003},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.71899999999999997,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.0199999999999996,"5min":5.0199999999999996,"15min":5.0199999999999996},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.81799999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.92300000000000004,"15min":0.94399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83699999999999997,"15min":0.84099999999999997},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.61299999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.97299999999999998,"15min":0.98999999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.873},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.74399999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.88600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":515141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.60299999999999998,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8949999999999996,"5min":6.8949999999999996,"15min":6.8949999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503}]},{"osd":4,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.67600000000000005,"15min":0.64400000000000002},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.498},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.76600000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76700000000000002,"15min":0.755},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.83499999999999996,"15min":0.80500000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":457459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88900000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.55500000000000005},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.873,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.62},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.077,"5min":1.216,"15min":1.216},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.145},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.44}]},{"osd":6,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93999999999999995,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.139,"15min":1.139},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.621}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0649999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.629,"5min":0.442,"15min":0.442},"max":{"1min":0.996,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83799999999999997,"15min":0.83499999999999996},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.67000000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.76600000000000001}]},{"osd":11,"last update":"Fri Oct 6 21:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.752},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.78500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80700000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.745}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.501},{"interface":"front","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66600000000000004,"15min":0.64500000000000002},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63400000000000001}]},{"osd":9,"last update":"Fri Oct 6 21:08:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.751,"15min":0.745},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.58299999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.96299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":456511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.45700000000000002,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.44800000000000001,"15min":0.41199999999999998},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.44600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.872,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.51100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.626}]},{"osd":2,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69499999999999995,"15min":0.64900000000000002},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93600000000000005,"5min":1.103,"15min":1.103},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69999999999999996,"15min":0.67400000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.109,"5min":1.129,"15min":1.129},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.71299999999999997},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.53400000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.85099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.96699999999999997,"15min":0.97799999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.98099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.395,"15min":1.395},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.129,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.752}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":516175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.314,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.52700000000000002},"min":{"1min":0.47299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.624}]},{"osd":3,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83299999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.627,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89200000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.627},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84099999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49199999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78100000000000003,"5min":0.872,"15min":0.872},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.96899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.87}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1048007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.623},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":0.63900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.76400000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71899999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.84199999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:09:14.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-06T21:09:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:15.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:15 smithi026 ceph-mon[98490]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:15.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:15 smithi026 ceph-mon[98490]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:15.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:15 smithi167 ceph-mon[134494]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:15.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:15 smithi167 ceph-mon[134494]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:15 smithi078 ceph-mon[134896]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:15 smithi078 ceph-mon[134896]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 -- 172.21.15.26:0/1495266704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 msgr2=0x7fee9010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 --2- 172.21.15.26:0/1495266704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9010fd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b0a0 tx=0x7fee8800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 -- 172.21.15.26:0/1495266704 shutdown_connections 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 --2- 172.21.15.26:0/1495266704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9010fd20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 --2- 172.21.15.26:0/1495266704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee900734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 --2- 172.21.15.26:0/1495266704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 0x7fee9010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 -- 172.21.15.26:0/1495266704 >> 172.21.15.26:0/1495266704 conn(0x7fee900780a0 msgr2=0x7fee900784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 -- 172.21.15.26:0/1495266704 shutdown_connections 2023-10-06T21:09:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.274+0000 7fee98263700 1 -- 172.21.15.26:0/1495266704 wait complete. 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 Processor -- start 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 -- start start 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 0x7fee901998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019e350 con 0x7fee9010d930 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019e4c0 con 0x7fee900730e0 2023-10-06T21:09:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.275+0000 7fee98263700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9019e680 con 0x7fee90073a00 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee96800700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34630/0 (socket says 172.21.15.26:34630) 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 -- 172.21.15.26:0/808762564 learned_addr learned my addr 172.21.15.26:0/808762564 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee957fe700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 0x7fee901998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 -- 172.21.15.26:0/808762564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 msgr2=0x7fee901998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 0x7fee901998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 -- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 msgr2=0x7fee9019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.276+0000 7fee95fff700 1 -- 172.21.15.26:0/808762564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009d00 con 0x7fee900730e0 2023-10-06T21:09:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee95fff700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9a0 tx=0x7fee8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c009790 con 0x7fee900730e0 2023-10-06T21:09:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9019e960 con 0x7fee900730e0 2023-10-06T21:09:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8c010430 con 0x7fee900730e0 2023-10-06T21:09:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c018680 con 0x7fee900730e0 2023-10-06T21:09:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.277+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee901a61c0 con 0x7fee900730e0 2023-10-06T21:09:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.278+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9004ebb0 con 0x7fee900730e0 2023-10-06T21:09:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.280+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fee8c0105a0 con 0x7fee900730e0 2023-10-06T21:09:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.282+0000 7fee86ffd700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 0x7fee7c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.282+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fee8c014070 con 0x7fee900730e0 2023-10-06T21:09:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.282+0000 7fee957fe700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 0x7fee7c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.282+0000 7fee957fe700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 0x7fee7c0a39d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fee9019a9b0 tx=0x7fee80009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:16.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.283+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fee8c096770 con 0x7fee900730e0 2023-10-06T21:09:16.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.491+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee90113250 con 0x7fee7c0a1510 2023-10-06T21:09:16.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.502+0000 7fee86ffd700 1 -- 172.21.15.26:0/808762564 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341045 (secure 0 0 0) 0x7fee90113250 con 0x7fee7c0a1510 2023-10-06T21:09:16.509 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 msgr2=0x7fee7c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 0x7fee7c0a39d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fee9019a9b0 tx=0x7fee80009260 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 msgr2=0x7fee90199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e9a0 tx=0x7fee8c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 shutdown_connections 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fee7c0a1510 0x7fee7c0a39d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee9010d930 0x7fee9019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee900730e0 0x7fee90199390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 --2- 172.21.15.26:0/808762564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee90073a00 0x7fee901998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 >> 172.21.15.26:0/808762564 conn(0x7fee900780a0 msgr2=0x7fee900fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 shutdown_connections 2023-10-06T21:09:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:16.507+0000 7fee98263700 1 -- 172.21.15.26:0/808762564 wait complete. 2023-10-06T21:09:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:09:17.053 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-10-06T21:09:16.383692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654164,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":460,"num_read_kb":345,"num_write":333,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":3009370,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337500,"kb_used_data":13256,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604324,"statfs":{"total":1151940427776,"available":1151594827776,"internally_reserved":0,"allocated":13574144,"data_stored":7433933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003679"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440999+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:08:44.440999+0000","last_peered":"2023-10-06T21:08:44.440999+0000","last_clean":"2023-10-06T21:08:44.440999+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:08:44.440999+0000","last_undegraded":"2023-10-06T21:08:44.440999+0000","last_fullsized":"2023-10-06T21:08:44.440999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440516+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:08:44.440516+0000","last_peered":"2023-10-06T21:08:44.440516+0000","last_clean":"2023-10-06T21:08:44.440516+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:08:44.440516+0000","last_undegraded":"2023-10-06T21:08:44.440516+0000","last_fullsized":"2023-10-06T21:08:44.440516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440041+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:08:44.440041+0000","last_peered":"2023-10-06T21:08:44.440041+0000","last_clean":"2023-10-06T21:08:44.440041+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:08:44.440041+0000","last_undegraded":"2023-10-06T21:08:44.440041+0000","last_fullsized":"2023-10-06T21:08:44.440041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.451527+0000","last_change":"2023-10-06T21:08:45.451527+0000","last_active":"2023-10-06T21:08:45.451527+0000","last_peered":"2023-10-06T21:08:45.451527+0000","last_clean":"2023-10-06T21:08:45.451527+0000","last_became_active":"2023-10-06T21:08:45.451219+0000","last_became_peered":"2023-10-06T21:08:45.451219+0000","last_unstale":"2023-10-06T21:08:45.451527+0000","last_undegraded":"2023-10-06T21:08:45.451527+0000","last_fullsized":"2023-10-06T21:08:45.451527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:03:42.551771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440485+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:08:44.440485+0000","last_peered":"2023-10-06T21:08:44.440485+0000","last_clean":"2023-10-06T21:08:44.440485+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:08:44.440485+0000","last_undegraded":"2023-10-06T21:08:44.440485+0000","last_fullsized":"2023-10-06T21:08:44.440485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444016+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:08:44.444016+0000","last_peered":"2023-10-06T21:08:44.444016+0000","last_clean":"2023-10-06T21:08:44.444016+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:08:44.444016+0000","last_undegraded":"2023-10-06T21:08:44.444016+0000","last_fullsized":"2023-10-06T21:08:44.444016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441482+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:08:44.441482+0000","last_peered":"2023-10-06T21:08:44.441482+0000","last_clean":"2023-10-06T21:08:44.441482+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:08:44.441482+0000","last_undegraded":"2023-10-06T21:08:44.441482+0000","last_fullsized":"2023-10-06T21:08:44.441482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434436+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:08:44.434436+0000","last_peered":"2023-10-06T21:08:44.434436+0000","last_clean":"2023-10-06T21:08:44.434436+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:08:44.434436+0000","last_undegraded":"2023-10-06T21:08:44.434436+0000","last_fullsized":"2023-10-06T21:08:44.434436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.451859+0000","last_change":"2023-10-06T21:08:45.451859+0000","last_active":"2023-10-06T21:08:45.451859+0000","last_peered":"2023-10-06T21:08:45.451859+0000","last_clean":"2023-10-06T21:08:45.451859+0000","last_became_active":"2023-10-06T21:08:45.451525+0000","last_became_peered":"2023-10-06T21:08:45.451525+0000","last_unstale":"2023-10-06T21:08:45.451859+0000","last_undegraded":"2023-10-06T21:08:45.451859+0000","last_fullsized":"2023-10-06T21:08:45.451859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:52.350672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444757+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:08:44.444757+0000","last_peered":"2023-10-06T21:08:44.444757+0000","last_clean":"2023-10-06T21:08:44.444757+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:08:44.444757+0000","last_undegraded":"2023-10-06T21:08:44.444757+0000","last_fullsized":"2023-10-06T21:08:44.444757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441985+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:08:44.441985+0000","last_peered":"2023-10-06T21:08:44.441985+0000","last_clean":"2023-10-06T21:08:44.441985+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:08:44.441985+0000","last_undegraded":"2023-10-06T21:08:44.441985+0000","last_fullsized":"2023-10-06T21:08:44.441985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435762+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:08:44.435762+0000","last_peered":"2023-10-06T21:08:44.435762+0000","last_clean":"2023-10-06T21:08:44.435762+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:08:44.435762+0000","last_undegraded":"2023-10-06T21:08:44.435762+0000","last_fullsized":"2023-10-06T21:08:44.435762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437179+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:08:44.437179+0000","last_peered":"2023-10-06T21:08:44.437179+0000","last_clean":"2023-10-06T21:08:44.437179+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:08:44.437179+0000","last_undegraded":"2023-10-06T21:08:44.437179+0000","last_fullsized":"2023-10-06T21:08:44.437179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439448+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:08:44.439448+0000","last_peered":"2023-10-06T21:08:44.439448+0000","last_clean":"2023-10-06T21:08:44.439448+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:08:44.439448+0000","last_undegraded":"2023-10-06T21:08:44.439448+0000","last_fullsized":"2023-10-06T21:08:44.439448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439141+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:08:44.439141+0000","last_peered":"2023-10-06T21:08:44.439141+0000","last_clean":"2023-10-06T21:08:44.439141+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:08:44.439141+0000","last_undegraded":"2023-10-06T21:08:44.439141+0000","last_fullsized":"2023-10-06T21:08:44.439141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443021+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:08:44.443021+0000","last_peered":"2023-10-06T21:08:44.443021+0000","last_clean":"2023-10-06T21:08:44.443021+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:08:44.443021+0000","last_undegraded":"2023-10-06T21:08:44.443021+0000","last_fullsized":"2023-10-06T21:08:44.443021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440971+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:08:44.440971+0000","last_peered":"2023-10-06T21:08:44.440971+0000","last_clean":"2023-10-06T21:08:44.440971+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:08:44.440971+0000","last_undegraded":"2023-10-06T21:08:44.440971+0000","last_fullsized":"2023-10-06T21:08:44.440971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440030+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:08:44.440030+0000","last_peered":"2023-10-06T21:08:44.440030+0000","last_clean":"2023-10-06T21:08:44.440030+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:08:44.440030+0000","last_undegraded":"2023-10-06T21:08:44.440030+0000","last_fullsized":"2023-10-06T21:08:44.440030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443577+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:08:44.443577+0000","last_peered":"2023-10-06T21:08:44.443577+0000","last_clean":"2023-10-06T21:08:44.443577+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:08:44.443577+0000","last_undegraded":"2023-10-06T21:08:44.443577+0000","last_fullsized":"2023-10-06T21:08:44.443577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439306+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:44.439306+0000","last_peered":"2023-10-06T21:08:44.439306+0000","last_clean":"2023-10-06T21:08:44.439306+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:44.439306+0000","last_undegraded":"2023-10-06T21:08:44.439306+0000","last_fullsized":"2023-10-06T21:08:44.439306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435729+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:08:44.435729+0000","last_peered":"2023-10-06T21:08:44.435729+0000","last_clean":"2023-10-06T21:08:44.435729+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:08:44.435729+0000","last_undegraded":"2023-10-06T21:08:44.435729+0000","last_fullsized":"2023-10-06T21:08:44.435729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435884+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:44.435884+0000","last_peered":"2023-10-06T21:08:44.435884+0000","last_clean":"2023-10-06T21:08:44.435884+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:44.435884+0000","last_undegraded":"2023-10-06T21:08:44.435884+0000","last_fullsized":"2023-10-06T21:08:44.435884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440152+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:08:44.440152+0000","last_peered":"2023-10-06T21:08:44.440152+0000","last_clean":"2023-10-06T21:08:44.440152+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:08:44.440152+0000","last_undegraded":"2023-10-06T21:08:44.440152+0000","last_fullsized":"2023-10-06T21:08:44.440152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440673+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:44.440673+0000","last_peered":"2023-10-06T21:08:44.440673+0000","last_clean":"2023-10-06T21:08:44.440673+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:44.440673+0000","last_undegraded":"2023-10-06T21:08:44.440673+0000","last_fullsized":"2023-10-06T21:08:44.440673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443250+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:08:44.443250+0000","last_peered":"2023-10-06T21:08:44.443250+0000","last_clean":"2023-10-06T21:08:44.443250+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:08:44.443250+0000","last_undegraded":"2023-10-06T21:08:44.443250+0000","last_fullsized":"2023-10-06T21:08:44.443250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441604+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:44.441604+0000","last_peered":"2023-10-06T21:08:44.441604+0000","last_clean":"2023-10-06T21:08:44.441604+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:44.441604+0000","last_undegraded":"2023-10-06T21:08:44.441604+0000","last_fullsized":"2023-10-06T21:08:44.441604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438645+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:08:44.438645+0000","last_peered":"2023-10-06T21:08:44.438645+0000","last_clean":"2023-10-06T21:08:44.438645+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:08:44.438645+0000","last_undegraded":"2023-10-06T21:08:44.438645+0000","last_fullsized":"2023-10-06T21:08:44.438645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":75,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:10.938008+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:09:10.938008+0000","last_peered":"2023-10-06T21:09:10.938008+0000","last_clean":"2023-10-06T21:09:10.938008+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:09:10.938008+0000","last_undegraded":"2023-10-06T21:09:10.938008+0000","last_fullsized":"2023-10-06T21:09:10.938008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440639+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:08:44.440639+0000","last_peered":"2023-10-06T21:08:44.440639+0000","last_clean":"2023-10-06T21:08:44.440639+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:08:44.440639+0000","last_undegraded":"2023-10-06T21:08:44.440639+0000","last_fullsized":"2023-10-06T21:08:44.440639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443912+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:44.443912+0000","last_peered":"2023-10-06T21:08:44.443912+0000","last_clean":"2023-10-06T21:08:44.443912+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:44.443912+0000","last_undegraded":"2023-10-06T21:08:44.443912+0000","last_fullsized":"2023-10-06T21:08:44.443912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438326+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:08:44.438326+0000","last_peered":"2023-10-06T21:08:44.438326+0000","last_clean":"2023-10-06T21:08:44.438326+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:08:44.438326+0000","last_undegraded":"2023-10-06T21:08:44.438326+0000","last_fullsized":"2023-10-06T21:08:44.438326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436810+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:44.436810+0000","last_peered":"2023-10-06T21:08:44.436810+0000","last_clean":"2023-10-06T21:08:44.436810+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:44.436810+0000","last_undegraded":"2023-10-06T21:08:44.436810+0000","last_fullsized":"2023-10-06T21:08:44.436810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434382+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:08:44.434382+0000","last_peered":"2023-10-06T21:08:44.434382+0000","last_clean":"2023-10-06T21:08:44.434382+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:08:44.434382+0000","last_undegraded":"2023-10-06T21:08:44.434382+0000","last_fullsized":"2023-10-06T21:08:44.434382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438712+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:08:44.438712+0000","last_peered":"2023-10-06T21:08:44.438712+0000","last_clean":"2023-10-06T21:08:44.438712+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:08:44.438712+0000","last_undegraded":"2023-10-06T21:08:44.438712+0000","last_fullsized":"2023-10-06T21:08:44.438712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:10.937348+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:09:10.937348+0000","last_peered":"2023-10-06T21:09:10.937348+0000","last_clean":"2023-10-06T21:09:10.937348+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:09:10.937348+0000","last_undegraded":"2023-10-06T21:09:10.937348+0000","last_fullsized":"2023-10-06T21:09:10.937348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443619+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:08:44.443619+0000","last_peered":"2023-10-06T21:08:44.443619+0000","last_clean":"2023-10-06T21:08:44.443619+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:08:44.443619+0000","last_undegraded":"2023-10-06T21:08:44.443619+0000","last_fullsized":"2023-10-06T21:08:44.443619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441919+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:44.441919+0000","last_peered":"2023-10-06T21:08:44.441919+0000","last_clean":"2023-10-06T21:08:44.441919+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:44.441919+0000","last_undegraded":"2023-10-06T21:08:44.441919+0000","last_fullsized":"2023-10-06T21:08:44.441919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440080+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:08:44.440080+0000","last_peered":"2023-10-06T21:08:44.440080+0000","last_clean":"2023-10-06T21:08:44.440080+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:08:44.440080+0000","last_undegraded":"2023-10-06T21:08:44.440080+0000","last_fullsized":"2023-10-06T21:08:44.440080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438756+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:44.438756+0000","last_peered":"2023-10-06T21:08:44.438756+0000","last_clean":"2023-10-06T21:08:44.438756+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:44.438756+0000","last_undegraded":"2023-10-06T21:08:44.438756+0000","last_fullsized":"2023-10-06T21:08:44.438756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443447+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:08:44.443447+0000","last_peered":"2023-10-06T21:08:44.443447+0000","last_clean":"2023-10-06T21:08:44.443447+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:08:44.443447+0000","last_undegraded":"2023-10-06T21:08:44.443447+0000","last_fullsized":"2023-10-06T21:08:44.443447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'27","reported_seq":170,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:11.661790+0000","last_change":"2023-10-06T21:08:45.622755+0000","last_active":"2023-10-06T21:09:11.661790+0000","last_peered":"2023-10-06T21:09:11.661790+0000","last_clean":"2023-10-06T21:09:11.661790+0000","last_became_active":"2023-10-06T21:08:45.614102+0000","last_became_peered":"2023-10-06T21:08:45.614102+0000","last_unstale":"2023-10-06T21:09:11.661790+0000","last_undegraded":"2023-10-06T21:09:11.661790+0000","last_fullsized":"2023-10-06T21:09:11.661790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:17:14.262961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":99,"num_read_kb":103,"num_write":28,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57508,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440154+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:08:44.440154+0000","last_peered":"2023-10-06T21:08:44.440154+0000","last_clean":"2023-10-06T21:08:44.440154+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:08:44.440154+0000","last_undegraded":"2023-10-06T21:08:44.440154+0000","last_fullsized":"2023-10-06T21:08:44.440154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443552+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:44.443552+0000","last_peered":"2023-10-06T21:08:44.443552+0000","last_clean":"2023-10-06T21:08:44.443552+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:44.443552+0000","last_undegraded":"2023-10-06T21:08:44.443552+0000","last_fullsized":"2023-10-06T21:08:44.443552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436976+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:08:44.436976+0000","last_peered":"2023-10-06T21:08:44.436976+0000","last_clean":"2023-10-06T21:08:44.436976+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:08:44.436976+0000","last_undegraded":"2023-10-06T21:08:44.436976+0000","last_fullsized":"2023-10-06T21:08:44.436976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438078+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:44.438078+0000","last_peered":"2023-10-06T21:08:44.438078+0000","last_clean":"2023-10-06T21:08:44.438078+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:44.438078+0000","last_undegraded":"2023-10-06T21:08:44.438078+0000","last_fullsized":"2023-10-06T21:08:44.438078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441762+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:08:44.441762+0000","last_peered":"2023-10-06T21:08:44.441762+0000","last_clean":"2023-10-06T21:08:44.441762+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:08:44.441762+0000","last_undegraded":"2023-10-06T21:08:44.441762+0000","last_fullsized":"2023-10-06T21:08:44.441762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:10.937407+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:09:10.937407+0000","last_peered":"2023-10-06T21:09:10.937407+0000","last_clean":"2023-10-06T21:09:10.937407+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:09:10.937407+0000","last_undegraded":"2023-10-06T21:09:10.937407+0000","last_fullsized":"2023-10-06T21:09:10.937407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.447758+0000","last_change":"2023-10-06T21:08:45.447758+0000","last_active":"2023-10-06T21:08:45.447758+0000","last_peered":"2023-10-06T21:08:45.447758+0000","last_clean":"2023-10-06T21:08:45.447758+0000","last_became_active":"2023-10-06T21:08:45.447337+0000","last_became_peered":"2023-10-06T21:08:45.447337+0000","last_unstale":"2023-10-06T21:08:45.447758+0000","last_undegraded":"2023-10-06T21:08:45.447758+0000","last_fullsized":"2023-10-06T21:08:45.447758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:48:29.806108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439954+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:44.439954+0000","last_peered":"2023-10-06T21:08:44.439954+0000","last_clean":"2023-10-06T21:08:44.439954+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:44.439954+0000","last_undegraded":"2023-10-06T21:08:44.439954+0000","last_fullsized":"2023-10-06T21:08:44.439954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440701+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:44.440701+0000","last_peered":"2023-10-06T21:08:44.440701+0000","last_clean":"2023-10-06T21:08:44.440701+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:44.440701+0000","last_undegraded":"2023-10-06T21:08:44.440701+0000","last_fullsized":"2023-10-06T21:08:44.440701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440917+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:08:44.440917+0000","last_peered":"2023-10-06T21:08:44.440917+0000","last_clean":"2023-10-06T21:08:44.440917+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:08:44.440917+0000","last_undegraded":"2023-10-06T21:08:44.440917+0000","last_fullsized":"2023-10-06T21:08:44.440917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440260+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:44.440260+0000","last_peered":"2023-10-06T21:08:44.440260+0000","last_clean":"2023-10-06T21:08:44.440260+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:44.440260+0000","last_undegraded":"2023-10-06T21:08:44.440260+0000","last_fullsized":"2023-10-06T21:08:44.440260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444652+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:08:44.444652+0000","last_peered":"2023-10-06T21:08:44.444652+0000","last_clean":"2023-10-06T21:08:44.444652+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:08:44.444652+0000","last_undegraded":"2023-10-06T21:08:44.444652+0000","last_fullsized":"2023-10-06T21:08:44.444652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441637+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:44.441637+0000","last_peered":"2023-10-06T21:08:44.441637+0000","last_clean":"2023-10-06T21:08:44.441637+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:44.441637+0000","last_undegraded":"2023-10-06T21:08:44.441637+0000","last_fullsized":"2023-10-06T21:08:44.441637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444417+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:08:44.444417+0000","last_peered":"2023-10-06T21:08:44.444417+0000","last_clean":"2023-10-06T21:08:44.444417+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:08:44.444417+0000","last_undegraded":"2023-10-06T21:08:44.444417+0000","last_fullsized":"2023-10-06T21:08:44.444417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.453278+0000","last_change":"2023-10-06T21:08:45.453278+0000","last_active":"2023-10-06T21:08:45.453278+0000","last_peered":"2023-10-06T21:08:45.453278+0000","last_clean":"2023-10-06T21:08:45.453278+0000","last_became_active":"2023-10-06T21:08:45.452892+0000","last_became_peered":"2023-10-06T21:08:45.452892+0000","last_unstale":"2023-10-06T21:08:45.453278+0000","last_undegraded":"2023-10-06T21:08:45.453278+0000","last_fullsized":"2023-10-06T21:08:45.453278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:54.604308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:12.939602+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:09:12.939602+0000","last_peered":"2023-10-06T21:09:12.939602+0000","last_clean":"2023-10-06T21:09:12.939602+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:09:12.939602+0000","last_undegraded":"2023-10-06T21:09:12.939602+0000","last_fullsized":"2023-10-06T21:09:12.939602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434953+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:08:44.434953+0000","last_peered":"2023-10-06T21:08:44.434953+0000","last_clean":"2023-10-06T21:08:44.434953+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:08:44.434953+0000","last_undegraded":"2023-10-06T21:08:44.434953+0000","last_fullsized":"2023-10-06T21:08:44.434953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436314+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:44.436314+0000","last_peered":"2023-10-06T21:08:44.436314+0000","last_clean":"2023-10-06T21:08:44.436314+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:44.436314+0000","last_undegraded":"2023-10-06T21:08:44.436314+0000","last_fullsized":"2023-10-06T21:08:44.436314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442005+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:44.442005+0000","last_peered":"2023-10-06T21:08:44.442005+0000","last_clean":"2023-10-06T21:08:44.442005+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:44.442005+0000","last_undegraded":"2023-10-06T21:08:44.442005+0000","last_fullsized":"2023-10-06T21:08:44.442005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436613+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:08:44.436613+0000","last_peered":"2023-10-06T21:08:44.436613+0000","last_clean":"2023-10-06T21:08:44.436613+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:08:44.436613+0000","last_undegraded":"2023-10-06T21:08:44.436613+0000","last_fullsized":"2023-10-06T21:08:44.436613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442183+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:44.442183+0000","last_peered":"2023-10-06T21:08:44.442183+0000","last_clean":"2023-10-06T21:08:44.442183+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:44.442183+0000","last_undegraded":"2023-10-06T21:08:44.442183+0000","last_fullsized":"2023-10-06T21:08:44.442183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.454229+0000","last_change":"2023-10-06T21:08:45.453710+0000","last_active":"2023-10-06T21:08:45.454229+0000","last_peered":"2023-10-06T21:08:45.454229+0000","last_clean":"2023-10-06T21:08:45.454229+0000","last_became_active":"2023-10-06T21:08:45.447386+0000","last_became_peered":"2023-10-06T21:08:45.447386+0000","last_unstale":"2023-10-06T21:08:45.454229+0000","last_undegraded":"2023-10-06T21:08:45.454229+0000","last_fullsized":"2023-10-06T21:08:45.454229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:27.659601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437313+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:08:44.437313+0000","last_peered":"2023-10-06T21:08:44.437313+0000","last_clean":"2023-10-06T21:08:44.437313+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:08:44.437313+0000","last_undegraded":"2023-10-06T21:08:44.437313+0000","last_fullsized":"2023-10-06T21:08:44.437313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440135+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:44.440135+0000","last_peered":"2023-10-06T21:08:44.440135+0000","last_clean":"2023-10-06T21:08:44.440135+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:44.440135+0000","last_undegraded":"2023-10-06T21:08:44.440135+0000","last_fullsized":"2023-10-06T21:08:44.440135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435642+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:08:44.435642+0000","last_peered":"2023-10-06T21:08:44.435642+0000","last_clean":"2023-10-06T21:08:44.435642+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:08:44.435642+0000","last_undegraded":"2023-10-06T21:08:44.435642+0000","last_fullsized":"2023-10-06T21:08:44.435642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436021+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:44.436021+0000","last_peered":"2023-10-06T21:08:44.436021+0000","last_clean":"2023-10-06T21:08:44.436021+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:44.436021+0000","last_undegraded":"2023-10-06T21:08:44.436021+0000","last_fullsized":"2023-10-06T21:08:44.436021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.613497+0000","last_change":"2023-10-06T21:08:45.613497+0000","last_active":"2023-10-06T21:08:45.613497+0000","last_peered":"2023-10-06T21:08:45.613497+0000","last_clean":"2023-10-06T21:08:45.613497+0000","last_became_active":"2023-10-06T21:08:45.613012+0000","last_became_peered":"2023-10-06T21:08:45.613012+0000","last_unstale":"2023-10-06T21:08:45.613497+0000","last_undegraded":"2023-10-06T21:08:45.613497+0000","last_fullsized":"2023-10-06T21:08:45.613497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:04:42.351299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443069+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:44.443069+0000","last_peered":"2023-10-06T21:08:44.443069+0000","last_clean":"2023-10-06T21:08:44.443069+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:44.443069+0000","last_undegraded":"2023-10-06T21:08:44.443069+0000","last_fullsized":"2023-10-06T21:08:44.443069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439861+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:08:44.439861+0000","last_peered":"2023-10-06T21:08:44.439861+0000","last_clean":"2023-10-06T21:08:44.439861+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:08:44.439861+0000","last_undegraded":"2023-10-06T21:08:44.439861+0000","last_fullsized":"2023-10-06T21:08:44.439861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434716+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:44.434716+0000","last_peered":"2023-10-06T21:08:44.434716+0000","last_clean":"2023-10-06T21:08:44.434716+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:44.434716+0000","last_undegraded":"2023-10-06T21:08:44.434716+0000","last_fullsized":"2023-10-06T21:08:44.434716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439000+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:08:44.439000+0000","last_peered":"2023-10-06T21:08:44.439000+0000","last_clean":"2023-10-06T21:08:44.439000+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:08:44.439000+0000","last_undegraded":"2023-10-06T21:08:44.439000+0000","last_fullsized":"2023-10-06T21:08:44.439000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.457037+0000","last_change":"2023-10-06T21:08:45.456629+0000","last_active":"2023-10-06T21:08:45.457037+0000","last_peered":"2023-10-06T21:08:45.457037+0000","last_clean":"2023-10-06T21:08:45.457037+0000","last_became_active":"2023-10-06T21:08:45.449565+0000","last_became_peered":"2023-10-06T21:08:45.449565+0000","last_unstale":"2023-10-06T21:08:45.457037+0000","last_undegraded":"2023-10-06T21:08:45.457037+0000","last_fullsized":"2023-10-06T21:08:45.457037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:23:54.912161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435249+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:08:44.435249+0000","last_peered":"2023-10-06T21:08:44.435249+0000","last_clean":"2023-10-06T21:08:44.435249+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:08:44.435249+0000","last_undegraded":"2023-10-06T21:08:44.435249+0000","last_fullsized":"2023-10-06T21:08:44.435249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440584+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:44.440584+0000","last_peered":"2023-10-06T21:08:44.440584+0000","last_clean":"2023-10-06T21:08:44.440584+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:44.440584+0000","last_undegraded":"2023-10-06T21:08:44.440584+0000","last_fullsized":"2023-10-06T21:08:44.440584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441769+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:08:44.441769+0000","last_peered":"2023-10-06T21:08:44.441769+0000","last_clean":"2023-10-06T21:08:44.441769+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:08:44.441769+0000","last_undegraded":"2023-10-06T21:08:44.441769+0000","last_fullsized":"2023-10-06T21:08:44.441769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444225+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:44.444225+0000","last_peered":"2023-10-06T21:08:44.444225+0000","last_clean":"2023-10-06T21:08:44.444225+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:44.444225+0000","last_undegraded":"2023-10-06T21:08:44.444225+0000","last_fullsized":"2023-10-06T21:08:44.444225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442079+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:08:44.442079+0000","last_peered":"2023-10-06T21:08:44.442079+0000","last_clean":"2023-10-06T21:08:44.442079+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:08:44.442079+0000","last_undegraded":"2023-10-06T21:08:44.442079+0000","last_fullsized":"2023-10-06T21:08:44.442079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438547+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:44.438547+0000","last_peered":"2023-10-06T21:08:44.438547+0000","last_clean":"2023-10-06T21:08:44.438547+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:44.438547+0000","last_undegraded":"2023-10-06T21:08:44.438547+0000","last_fullsized":"2023-10-06T21:08:44.438547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441789+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:08:44.441789+0000","last_peered":"2023-10-06T21:08:44.441789+0000","last_clean":"2023-10-06T21:08:44.441789+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:08:44.441789+0000","last_undegraded":"2023-10-06T21:08:44.441789+0000","last_fullsized":"2023-10-06T21:08:44.441789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437500+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:44.437500+0000","last_peered":"2023-10-06T21:08:44.437500+0000","last_clean":"2023-10-06T21:08:44.437500+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:44.437500+0000","last_undegraded":"2023-10-06T21:08:44.437500+0000","last_fullsized":"2023-10-06T21:08:44.437500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442142+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:08:44.442142+0000","last_peered":"2023-10-06T21:08:44.442142+0000","last_clean":"2023-10-06T21:08:44.442142+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:08:44.442142+0000","last_undegraded":"2023-10-06T21:08:44.442142+0000","last_fullsized":"2023-10-06T21:08:44.442142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439097+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:08:44.439097+0000","last_peered":"2023-10-06T21:08:44.439097+0000","last_clean":"2023-10-06T21:08:44.439097+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:08:44.439097+0000","last_undegraded":"2023-10-06T21:08:44.439097+0000","last_fullsized":"2023-10-06T21:08:44.439097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443651+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:08:44.443651+0000","last_peered":"2023-10-06T21:08:44.443651+0000","last_clean":"2023-10-06T21:08:44.443651+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:08:44.443651+0000","last_undegraded":"2023-10-06T21:08:44.443651+0000","last_fullsized":"2023-10-06T21:08:44.443651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443339+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:08:44.443339+0000","last_peered":"2023-10-06T21:08:44.443339+0000","last_clean":"2023-10-06T21:08:44.443339+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:08:44.443339+0000","last_undegraded":"2023-10-06T21:08:44.443339+0000","last_fullsized":"2023-10-06T21:08:44.443339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.448913+0000","last_change":"2023-10-06T21:08:45.448913+0000","last_active":"2023-10-06T21:08:45.448913+0000","last_peered":"2023-10-06T21:08:45.448913+0000","last_clean":"2023-10-06T21:08:45.448913+0000","last_became_active":"2023-10-06T21:08:45.448485+0000","last_became_peered":"2023-10-06T21:08:45.448485+0000","last_unstale":"2023-10-06T21:08:45.448913+0000","last_undegraded":"2023-10-06T21:08:45.448913+0000","last_fullsized":"2023-10-06T21:08:45.448913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:42:07.732211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437384+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:08:44.437384+0000","last_peered":"2023-10-06T21:08:44.437384+0000","last_clean":"2023-10-06T21:08:44.437384+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:08:44.437384+0000","last_undegraded":"2023-10-06T21:08:44.437384+0000","last_fullsized":"2023-10-06T21:08:44.437384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441023+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:08:44.441023+0000","last_peered":"2023-10-06T21:08:44.441023+0000","last_clean":"2023-10-06T21:08:44.441023+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:08:44.441023+0000","last_undegraded":"2023-10-06T21:08:44.441023+0000","last_fullsized":"2023-10-06T21:08:44.441023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443842+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:08:44.443842+0000","last_peered":"2023-10-06T21:08:44.443842+0000","last_clean":"2023-10-06T21:08:44.443842+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:08:44.443842+0000","last_undegraded":"2023-10-06T21:08:44.443842+0000","last_fullsized":"2023-10-06T21:08:44.443842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435568+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:08:44.435568+0000","last_peered":"2023-10-06T21:08:44.435568+0000","last_clean":"2023-10-06T21:08:44.435568+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:08:44.435568+0000","last_undegraded":"2023-10-06T21:08:44.435568+0000","last_fullsized":"2023-10-06T21:08:44.435568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437894+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:08:44.437894+0000","last_peered":"2023-10-06T21:08:44.437894+0000","last_clean":"2023-10-06T21:08:44.437894+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:08:44.437894+0000","last_undegraded":"2023-10-06T21:08:44.437894+0000","last_fullsized":"2023-10-06T21:08:44.437894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434567+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:08:44.434567+0000","last_peered":"2023-10-06T21:08:44.434567+0000","last_clean":"2023-10-06T21:08:44.434567+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:08:44.434567+0000","last_undegraded":"2023-10-06T21:08:44.434567+0000","last_fullsized":"2023-10-06T21:08:44.434567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443365+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:08:44.443365+0000","last_peered":"2023-10-06T21:08:44.443365+0000","last_clean":"2023-10-06T21:08:44.443365+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:08:44.443365+0000","last_undegraded":"2023-10-06T21:08:44.443365+0000","last_fullsized":"2023-10-06T21:08:44.443365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437157+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:08:44.437157+0000","last_peered":"2023-10-06T21:08:44.437157+0000","last_clean":"2023-10-06T21:08:44.437157+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:08:44.437157+0000","last_undegraded":"2023-10-06T21:08:44.437157+0000","last_fullsized":"2023-10-06T21:08:44.437157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439923+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:08:44.439923+0000","last_peered":"2023-10-06T21:08:44.439923+0000","last_clean":"2023-10-06T21:08:44.439923+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:08:44.439923+0000","last_undegraded":"2023-10-06T21:08:44.439923+0000","last_fullsized":"2023-10-06T21:08:44.439923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63550,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":324,"num_read_kb":226,"num_write":150,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":57530,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":190650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":456585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.621}]},{"osd":2,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.76100000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.91900000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645339,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1107600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":1.089,"15min":1.089},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.82099999999999995,"5min":1.165,"15min":1.165},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.873,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.85199999999999998,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84899999999999998,"5min":1.26,"15min":1.26},"last":0.58099999999999996}]},{"osd":11,"last update":"Fri Oct 6 21:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.81000000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.76500000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.501},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.83699999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70799999999999996,"15min":0.70999999999999996},"min":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.996,"5min":1.006,"15min":1.006},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.79000000000000004,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91400000000000003,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.62},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0149999999999999,"5min":1.198,"15min":1.198},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83799999999999997,"15min":0.84599999999999997},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1719999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.399,"5min":1.474,"15min":1.474},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.173,"5min":1.226,"15min":1.226},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":1.0049999999999999,"15min":1.024},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4770000000000001,"5min":1.542,"15min":1.542},"last":0.79300000000000004}]},{"osd":8,"last update":"Fri Oct 6 21:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.88100000000000001,"15min":0.879},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.9169999999999998,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.876},{"interface":"front","average":{"1min":0.878,"5min":0.95199999999999996,"15min":0.96399999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2310000000000001,"5min":1.365,"15min":1.365},"last":0.91200000000000003}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.95799999999999996,"15min":0.94999999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1359999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.5,"15min":0.505},"min":{"1min":0.32500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72699999999999998,"5min":0.748,"15min":0.748},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59999999999999998,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77200000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.87,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54400000000000004,"15min":0.55200000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83299999999999996,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81799999999999995,"5min":1.087,"15min":1.087},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95399999999999996,"5min":1.038,"15min":1.038},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90800000000000003,"5min":1.155,"15min":1.155},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":1.01}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.747},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.032}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1049044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.747},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.76500000000000001,"5min":1.002,"15min":1.002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81200000000000006,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97399999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.87}]},{"osd":9,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.008,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89200000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":456194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.57899999999999996,"15min":0.53900000000000003},"min":{"1min":0.32000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":5.8620000000000001,"5min":5.8620000000000001,"15min":5.8620000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.67900000000000005},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.44800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":4.9809999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73899999999999999,"15min":0.72199999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.375},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.753,"15min":0.748},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.9290000000000003,"5min":5.9290000000000003,"15min":5.9290000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.71899999999999997,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.8369999999999997,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.0199999999999996,"5min":5.0199999999999996,"15min":5.0199999999999996},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.70599999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.92300000000000004,"15min":0.94399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83699999999999997,"15min":0.84099999999999997},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.1000000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.753}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.97299999999999998,"15min":0.98999999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.872,"15min":0.873},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86899999999999999,"15min":0.83399999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.8979999999999997,"5min":4.8979999999999997,"15min":4.8979999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":515141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.60299999999999998,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.8949999999999996,"5min":6.8949999999999996,"15min":6.8949999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.51300000000000001},"min":{"1min":0.375,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.36099999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63400000000000001,"15min":0.60699999999999998},"min":{"1min":0.47899999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.67600000000000005,"15min":0.64400000000000002},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":0.748}]},{"osd":7,"last update":"Fri Oct 6 21:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.498},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.627},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96099999999999997,"5min":1.022,"15min":1.022},"last":0.64400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76700000000000002,"15min":0.755},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.83499999999999996,"15min":0.80500000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.497},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.45100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":457459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88900000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60799999999999998,"15min":0.55500000000000005},"min":{"1min":0.47299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.873,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.80500000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97399999999999998,"5min":1.216,"15min":1.216},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.145},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:08:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93999999999999995,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.441}]},{"osd":7,"last update":"Fri Oct 6 21:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.74099999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0449999999999999,"5min":1.139,"15min":1.139},"last":0.747},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84199999999999997,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0960000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.70999999999999996}]},{"osd":8,"last update":"Fri Oct 6 21:08:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0649999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.84899999999999998,"15min":0.85599999999999998},"min":{"1min":0.629,"5min":0.442,"15min":0.442},"max":{"1min":0.996,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83799999999999997,"15min":0.83499999999999996},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.68100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.90900000000000003}]},{"osd":11,"last update":"Fri Oct 6 21:08:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.876}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80700000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77400000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.5},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.442,"15min":0.44500000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33900000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66600000000000004,"15min":0.64500000000000002},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.745}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:08:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42099999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:08:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.751,"15min":0.745},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0369999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.87},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82399999999999995}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.58299999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.75900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":456511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.45700000000000002,"15min":0.42599999999999999},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.44800000000000001,"15min":0.41199999999999998},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40100000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.44600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.872,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.51100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Oct 6 21:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.89400000000000002,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69499999999999995,"15min":0.64900000000000002},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93600000000000005,"5min":1.103,"15min":1.103},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72099999999999997,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69999999999999996,"15min":0.67400000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.109,"5min":1.129,"15min":1.129},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.71299999999999997},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:08:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83899999999999997,"15min":0.85099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.96699999999999997,"15min":0.97799999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.98099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.395,"15min":1.395},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.129,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":516175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.33800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.36199999999999999},"min":{"1min":0.314,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.52700000000000002},"min":{"1min":0.47299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.55500000000000005},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83299999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.44500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.627,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89200000000000002,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.66100000000000003},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89600000000000002,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.627},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84099999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Oct 6 21:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.441},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72299999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:08:29 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.49199999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78100000000000003,"5min":0.872,"15min":0.872},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.68600000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:08:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.96899999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.751}]},{"osd":10,"last update":"Fri Oct 6 21:08:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1568768,"data_stored":1048007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.623},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71099999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.38400000000000001},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58299999999999996,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.40100000000000002},"min":{"1min":0.251,"5min":0.25,"15min":0.25},"max":{"1min":0.63900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Oct 6 21:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64900000000000002,"15min":0.64100000000000001},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.752},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.76400000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71899999999999997}]},{"osd":9,"last update":"Fri Oct 6 21:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.626,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.84199999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:09:17.063 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-06T21:09:17.063 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T21:09:17.064 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-06T21:09:17.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-06T21:09:17.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:17 smithi167 ceph-mon[134494]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:17.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:17 smithi167 ceph-mon[134494]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:17 smithi026 ceph-mon[98490]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:17 smithi026 ceph-mon[98490]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:17.833 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:17 smithi078 ceph-mon[134896]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:17.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:17 smithi078 ceph-mon[134896]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:09:19.036 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:18 smithi026 ceph-mon[98490]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:19.036 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:18 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:09:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:18 smithi078 ceph-mon[134896]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:18 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:09:19.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:18 smithi167 ceph-mon[134494]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:18 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 -- 172.21.15.26:0/2080043015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 msgr2=0x7f06e0107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 --2- 172.21.15.26:0/2080043015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0107370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0099c0 tx=0x7f06dc009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 -- 172.21.15.26:0/2080043015 shutdown_connections 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 --2- 172.21.15.26:0/2080043015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 --2- 172.21.15.26:0/2080043015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0107370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 --2- 172.21.15.26:0/2080043015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 -- 172.21.15.26:0/2080043015 >> 172.21.15.26:0/2080043015 conn(0x7f06e00780a0 msgr2=0x7f06e00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 -- 172.21.15.26:0/2080043015 shutdown_connections 2023-10-06T21:09:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.292+0000 7f06e8389700 1 -- 172.21.15.26:0/2080043015 wait complete. 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.293+0000 7f06e8389700 1 Processor -- start 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.293+0000 7f06e8389700 1 -- start start 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e00736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e01a5400 con 0x7f06e0106f90 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e01a5570 con 0x7f06e0101940 2023-10-06T21:09:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e8389700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e01a5730 con 0x7f06e0100f80 2023-10-06T21:09:19.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e6125700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:19.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e6125700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59748/0 (socket says 172.21.15.26:59748) 2023-10-06T21:09:19.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e6125700 1 -- 172.21.15.26:0/2580690266 learned_addr learned my addr 172.21.15.26:0/2580690266 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:19.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e6926700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:19.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e5924700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e00736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:19.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e5924700 1 -- 172.21.15.26:0/2580690266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 msgr2=0x7f06e00731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:19.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.294+0000 7f06e5924700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e5924700 1 -- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 msgr2=0x7f06e0073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e5924700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e5924700 1 -- 172.21.15.26:0/2580690266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d8009c40 con 0x7f06e0101940 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e6125700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e6926700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e5924700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e00736f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d750 tx=0x7f06d000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:19.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004020 con 0x7f06e0101940 2023-10-06T21:09:19.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc0095f0 con 0x7f06e0101940 2023-10-06T21:09:19.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06d0004830 con 0x7f06e0101940 2023-10-06T21:09:19.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00054c0 con 0x7f06e0101940 2023-10-06T21:09:19.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.295+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e01a5d80 con 0x7f06e0101940 2023-10-06T21:09:19.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.296+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e004ebb0 con 0x7f06e0101940 2023-10-06T21:09:19.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.298+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f06d00049a0 con 0x7f06e0101940 2023-10-06T21:09:19.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.299+0000 7f06d77fe700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 0x7f06cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:19.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.300+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06d00cc820 con 0x7f06e0101940 2023-10-06T21:09:19.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.300+0000 7f06e6125700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 0x7f06cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:19.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.300+0000 7f06e6125700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 0x7f06cc0a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0036c0 tx=0x7f06dc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:19.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.303+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f06d0095ce0 con 0x7f06e0101940 2023-10-06T21:09:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.585+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f06e0110fe0 con 0x7f06e0101940 2023-10-06T21:09:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.585+0000 7f06d77fe700 1 -- 172.21.15.26:0/2580690266 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f06d0099300 con 0x7f06e0101940 2023-10-06T21:09:19.588 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:19.588 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T21:09:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 msgr2=0x7f06cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 0x7f06cc0a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0036c0 tx=0x7f06dc005fb0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 msgr2=0x7f06e00736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e00736f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d750 tx=0x7f06d000da60 comp rx=0 tx=0).stop 2023-10-06T21:09:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 shutdown_connections 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f06cc0a1510 0x7f06cc0a39d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0106f90 0x7f06e0073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06e0100f80 0x7f06e00731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 --2- 172.21.15.26:0/2580690266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06e0101940 0x7f06e00736f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.588+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 >> 172.21.15.26:0/2580690266 conn(0x7f06e00780a0 msgr2=0x7f06e00ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.589+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 shutdown_connections 2023-10-06T21:09:19.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:19.589+0000 7f06e8389700 1 -- 172.21.15.26:0/2580690266 wait complete. 2023-10-06T21:09:20.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:19 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2580690266' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:09:20.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:19 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2580690266' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:09:20.127 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-06T21:09:20.128 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:20.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:19 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2580690266' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 -- 172.21.15.26:0/3878022593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4073910 msgr2=0x7f01d4073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3878022593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4073910 0x7f01d4073d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098f0 tx=0x7f01c8009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 -- 172.21.15.26:0/3878022593 shutdown_connections 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3878022593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 0x7f01d410b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3878022593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4074ef0 0x7f01d4073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3878022593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4073910 0x7f01d4073d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 -- 172.21.15.26:0/3878022593 >> 172.21.15.26:0/3878022593 conn(0x7f01d40fc420 msgr2=0x7f01d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 -- 172.21.15.26:0/3878022593 shutdown_connections 2023-10-06T21:09:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.357+0000 7f01d9dff700 1 -- 172.21.15.26:0/3878022593 wait complete. 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.358+0000 7f01d9dff700 1 Processor -- start 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.358+0000 7f01d9dff700 1 -- start start 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 0x7f01d4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 0x7f01d41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41a09d0 con 0x7f01d4073910 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41a0b10 con 0x7f01d4108db0 2023-10-06T21:09:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d9dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41a0ca0 con 0x7f01d4074ef0 2023-10-06T21:09:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 0x7f01d4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59772/0 (socket says 172.21.15.26:59772) 2023-10-06T21:09:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 0x7f01d4198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40892/0 (socket says 172.21.15.26:40892) 2023-10-06T21:09:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 -- 172.21.15.26:0/3380452706 learned_addr learned my addr 172.21.15.26:0/3380452706 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d3fff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 0x7f01d41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 -- 172.21.15.26:0/3380452706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 msgr2=0x7f01d41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 0x7f01d41a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 -- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 msgr2=0x7f01d4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 0x7f01d4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.359+0000 7f01d2ffd700 1 -- 172.21.15.26:0/3380452706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8009520 con 0x7f01d4074ef0 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d2ffd700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009fd0 tx=0x7f01c800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c801c070 con 0x7f01d4074ef0 2023-10-06T21:09:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01c800f7d0 con 0x7f01d4074ef0 2023-10-06T21:09:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d41a0f20 con 0x7f01d4074ef0 2023-10-06T21:09:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003a20 con 0x7f01d4074ef0 2023-10-06T21:09:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.360+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41a12e0 con 0x7f01d4074ef0 2023-10-06T21:09:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.362+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d4107460 con 0x7f01d4074ef0 2023-10-06T21:09:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.363+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f01c8020030 con 0x7f01d4074ef0 2023-10-06T21:09:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.365+0000 7f01d0ff9700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 0x7f01bc0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.365+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01c80cddd0 con 0x7f01d4074ef0 2023-10-06T21:09:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.365+0000 7f01d37fe700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 0x7f01bc0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.366+0000 7f01d37fe700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 0x7f01bc0a3a90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01c4005fd0 tx=0x7f01c4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.368+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f01c8097320 con 0x7f01d4074ef0 2023-10-06T21:09:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.598+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01d410eae0 con 0x7f01d4074ef0 2023-10-06T21:09:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.600+0000 7f01d0ff9700 1 -- 172.21.15.26:0/3380452706 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f01c809a940 con 0x7f01d4074ef0 2023-10-06T21:09:20.603 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:20.603 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 msgr2=0x7f01bc0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 0x7f01bc0a3a90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01c4005fd0 tx=0x7f01c4009310 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 msgr2=0x7f01d4199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009fd0 tx=0x7f01c800b790 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 shutdown_connections 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d4108db0 0x7f01d41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f01bc0a15d0 0x7f01bc0a3a90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4073910 0x7f01d4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 --2- 172.21.15.26:0/3380452706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01d4074ef0 0x7f01d4199390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 >> 172.21.15.26:0/3380452706 conn(0x7f01d40fc420 msgr2=0x7f01d40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 shutdown_connections 2023-10-06T21:09:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.602+0000 7f01d9dff700 1 -- 172.21.15.26:0/3380452706 wait complete. 2023-10-06T21:09:20.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:09:20.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:09:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 -- 172.21.15.26:0/3921781904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102070 msgr2=0x7f99a81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/3921781904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102070 0x7f99a81024f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f999c0098f0 tx=0x7f999c009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:20.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 -- 172.21.15.26:0/3921781904 shutdown_connections 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/3921781904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102a30 0x7f99a8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/3921781904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/3921781904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102070 0x7f99a81024f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 -- 172.21.15.26:0/3921781904 >> 172.21.15.26:0/3921781904 conn(0x7f99a80fc430 msgr2=0x7f99a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 -- 172.21.15.26:0/3921781904 shutdown_connections 2023-10-06T21:09:20.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.844+0000 7f99ad5ff700 1 -- 172.21.15.26:0/3921781904 wait complete. 2023-10-06T21:09:20.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.846+0000 7f99ad5ff700 1 Processor -- start 2023-10-06T21:09:20.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.846+0000 7f99ad5ff700 1 -- start start 2023-10-06T21:09:20.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102070 0x7f99a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102a30 0x7f99a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8073f80 con 0x7f99a8102a30 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80740f0 con 0x7f99a8100e90 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99ad5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81a0c80 con 0x7f99a8102070 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39936/0 (socket says 172.21.15.26:39936) 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 -- 172.21.15.26:0/2326384079 learned_addr learned my addr 172.21.15.26:0/2326384079 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a67fc700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102070 0x7f99a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a77fe700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102a30 0x7f99a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 -- 172.21.15.26:0/2326384079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102070 msgr2=0x7f99a80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102070 0x7f99a80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 -- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102a30 msgr2=0x7f99a80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:20.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102a30 0x7f99a80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:20.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 -- 172.21.15.26:0/2326384079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980095f0 con 0x7f99a8100e90 2023-10-06T21:09:20.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.847+0000 7f99a6ffd700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f999800ebf0 tx=0x7f999800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:20.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.849+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998009a10 con 0x7f99a8100e90 2023-10-06T21:09:20.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.849+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999800cd40 con 0x7f99a8100e90 2023-10-06T21:09:20.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.849+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c009520 con 0x7f99a8100e90 2023-10-06T21:09:20.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.849+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998018820 con 0x7f99a8100e90 2023-10-06T21:09:20.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.849+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a81a1250 con 0x7f99a8100e90 2023-10-06T21:09:20.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.850+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a8107ef0 con 0x7f99a8100e90 2023-10-06T21:09:20.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.853+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9998010430 con 0x7f99a8100e90 2023-10-06T21:09:20.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.854+0000 7f998ffff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 0x7f99900a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:20.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.854+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9998014070 con 0x7f99a8100e90 2023-10-06T21:09:20.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.855+0000 7f99a67fc700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 0x7f99900a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:20.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.855+0000 7f99a67fc700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 0x7f99900a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f999c0098c0 tx=0x7f999c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:20.858+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f99980969a0 con 0x7f99a8100e90 2023-10-06T21:09:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:20 smithi026 ceph-mon[98490]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:20 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3380452706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.109+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99a802d020 con 0x7f99a8100e90 2023-10-06T21:09:21.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.110+0000 7f998ffff700 1 -- 172.21.15.26:0/2326384079 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9998099fc0 con 0x7f99a8100e90 2023-10-06T21:09:21.113 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:09:21.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 msgr2=0x7f99900a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 0x7f99900a3a50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f999c0098c0 tx=0x7f999c019040 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 msgr2=0x7f99a8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f999800ebf0 tx=0x7f999800ef00 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 shutdown_connections 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99900a1590 0x7f99900a3a50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a8102a30 0x7f99a80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99a8100e90 0x7f99a8074c60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 --2- 172.21.15.26:0/2326384079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99a8102070 0x7f99a80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 >> 172.21.15.26:0/2326384079 conn(0x7f99a80fc430 msgr2=0x7f99a80fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.112+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 shutdown_connections 2023-10-06T21:09:21.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.113+0000 7f99ad5ff700 1 -- 172.21.15.26:0/2326384079 wait complete. 2023-10-06T21:09:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:20 smithi078 ceph-mon[134896]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:20 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3380452706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:21.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:09:21.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:20 smithi167 ceph-mon[134494]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:21.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:20 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3380452706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2150498351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 msgr2=0x7fc174109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2150498351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc174109350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00b0a0 tx=0x7fc16c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2150498351 shutdown_connections 2023-10-06T21:09:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2150498351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc174109350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2150498351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 0x7fc1741012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2150498351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc1741024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2150498351 >> 172.21.15.26:0/2150498351 conn(0x7fc1740fc430 msgr2=0x7fc1740fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2150498351 shutdown_connections 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.353+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2150498351 wait complete. 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.355+0000 7fc17a1ff700 1 Processor -- start 2023-10-06T21:09:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.355+0000 7fc17a1ff700 1 -- start start 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.355+0000 7fc17a1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 0x7fc174196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc17a1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc174197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc17a1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc17419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc17a1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17419e860 con 0x7fc174100e90 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc17a1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17419e9d0 con 0x7fc174102a30 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc17a1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17419eb90 con 0x7fc174102070 2023-10-06T21:09:21.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc1737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 0x7fc174196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc172ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc174197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc172ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc174197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36042/0 (socket says 172.21.15.26:36042) 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc172ffd700 1 -- 172.21.15.26:0/2261649950 learned_addr learned my addr 172.21.15.26:0/2261649950 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc17419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 -- 172.21.15.26:0/2261649950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 msgr2=0x7fc174197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc174197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 -- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 msgr2=0x7fc174196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 0x7fc174196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.356+0000 7fc173fff700 1 -- 172.21.15.26:0/2261649950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009d00 con 0x7fc174102a30 2023-10-06T21:09:21.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc173fff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc17419e290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009fd0 tx=0x7fc16c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:21.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00e040 con 0x7fc174102a30 2023-10-06T21:09:21.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc16c0094c0 con 0x7fc174102a30 2023-10-06T21:09:21.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0052a0 con 0x7fc174102a30 2023-10-06T21:09:21.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17419ee10 con 0x7fc174102a30 2023-10-06T21:09:21.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.357+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17419f180 con 0x7fc174102a30 2023-10-06T21:09:21.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.358+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17410c750 con 0x7fc174102a30 2023-10-06T21:09:21.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.360+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc16c01d470 con 0x7fc174102a30 2023-10-06T21:09:21.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.363+0000 7fc170ff9700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 0x7fc15c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.363+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc16c0ce360 con 0x7fc174102a30 2023-10-06T21:09:21.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.363+0000 7fc1737fe700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 0x7fc15c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.363+0000 7fc1737fe700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 0x7fc15c0a3a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1741981f0 tx=0x7fc164007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:21.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.365+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc16c0978b0 con 0x7fc174102a30 2023-10-06T21:09:21.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.584+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc17419f5f0 con 0x7fc174102a30 2023-10-06T21:09:21.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.585+0000 7fc170ff9700 1 -- 172.21.15.26:0/2261649950 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc16c09aed0 con 0x7fc174102a30 2023-10-06T21:09:21.588 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 msgr2=0x7fc15c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 0x7fc15c0a3a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1741981f0 tx=0x7fc164007400 comp rx=0 tx=0).stop 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 msgr2=0x7fc17419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc17419e290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009fd0 tx=0x7fc16c00bce0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 shutdown_connections 2023-10-06T21:09:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc174102a30 0x7fc17419e290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc15c0a1590 0x7fc15c0a3a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc174100e90 0x7fc174196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 --2- 172.21.15.26:0/2261649950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc174102070 0x7fc174197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 >> 172.21.15.26:0/2261649950 conn(0x7fc1740fc430 msgr2=0x7fc1740fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 shutdown_connections 2023-10-06T21:09:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.588+0000 7fc17a1ff700 1 -- 172.21.15.26:0/2261649950 wait complete. 2023-10-06T21:09:21.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:09:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 -- 172.21.15.26:0/2641743307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 msgr2=0x7f05381066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 --2- 172.21.15.26:0/2641743307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05381066a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f052c0098f0 tx=0x7f052c009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 -- 172.21.15.26:0/2641743307 shutdown_connections 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 --2- 172.21.15.26:0/2641743307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 0x7f0538109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 --2- 172.21.15.26:0/2641743307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f0538103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 --2- 172.21.15.26:0/2641743307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05381066a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.829+0000 7f053e7ff700 1 -- 172.21.15.26:0/2641743307 >> 172.21.15.26:0/2641743307 conn(0x7f05380fb370 msgr2=0x7f05380fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.830+0000 7f053e7ff700 1 -- 172.21.15.26:0/2641743307 shutdown_connections 2023-10-06T21:09:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.830+0000 7f053e7ff700 1 -- 172.21.15.26:0/2641743307 wait complete. 2023-10-06T21:09:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.831+0000 7f053e7ff700 1 Processor -- start 2023-10-06T21:09:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.831+0000 7f053e7ff700 1 -- start start 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f05381012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05380ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 0x7f05380ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538100610 con 0x7f0538106be0 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538100780 con 0x7f05381042b0 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053e7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05381a0ca0 con 0x7f0538101980 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f0537fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f05381012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f0537fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f05381012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36066/0 (socket says 172.21.15.26:36066) 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f0537fff700 1 -- 172.21.15.26:0/758212746 learned_addr learned my addr 172.21.15.26:0/758212746 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f05377fe700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05380ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.832+0000 7f053cd9c700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 0x7f05380ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 -- 172.21.15.26:0/758212746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 msgr2=0x7f05381012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f05381012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 -- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 msgr2=0x7f05380ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 0x7f05380ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 -- 172.21.15.26:0/758212746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c009520 con 0x7f05381042b0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05377fe700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05380ff930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c01c070 con 0x7f05381042b0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05381a0f20 con 0x7f05381042b0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c00f7d0 con 0x7f05381042b0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.833+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c003aa0 con 0x7f05381042b0 2023-10-06T21:09:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.834+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05381a1340 con 0x7f05381042b0 2023-10-06T21:09:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.835+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05380fcef0 con 0x7f05381042b0 2023-10-06T21:09:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.838+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f052c020030 con 0x7f05381042b0 2023-10-06T21:09:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.839+0000 7f05357fa700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 0x7f05200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.840+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f052c0cdd40 con 0x7f05381042b0 2023-10-06T21:09:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.840+0000 7f0537fff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 0x7f05200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.840+0000 7f0537fff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 0x7f05200a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0528009cb0 tx=0x7f0528009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:21.843+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f052c097290 con 0x7f05381042b0 2023-10-06T21:09:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2326384079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:21 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2261649950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.072+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f053802cf70 con 0x7f05381042b0 2023-10-06T21:09:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.073+0000 7f05357fa700 1 -- 172.21.15.26:0/758212746 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f052c09a8b0 con 0x7f05381042b0 2023-10-06T21:09:22.075 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:09:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 msgr2=0x7f05200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 0x7f05200a3a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0528009cb0 tx=0x7f0528009260 comp rx=0 tx=0).stop 2023-10-06T21:09:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 msgr2=0x7f05380ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05380ff930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f052c009230 tx=0x7f052c00b790 comp rx=0 tx=0).stop 2023-10-06T21:09:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 shutdown_connections 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f05200a1590 0x7f05200a3a50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538106be0 0x7f05380ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0538101980 0x7f05381012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 --2- 172.21.15.26:0/758212746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05381042b0 0x7f05380ff930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 >> 172.21.15.26:0/758212746 conn(0x7f05380fb370 msgr2=0x7f05381025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 shutdown_connections 2023-10-06T21:09:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.075+0000 7f053e7ff700 1 -- 172.21.15.26:0/758212746 wait complete. 2023-10-06T21:09:22.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:09:22.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2326384079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:21 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2261649950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2326384079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:22.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:21 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2261649950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 -- 172.21.15.26:0/3309437674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0101950 msgr2=0x7fe5c0103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/3309437674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0101950 0x7fe5c0103d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009990 tx=0x7fe5b0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 -- 172.21.15.26:0/3309437674 shutdown_connections 2023-10-06T21:09:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/3309437674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c0109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/3309437674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0101950 0x7fe5c0103d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/3309437674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0104280 0x7fe5c0106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 -- 172.21.15.26:0/3309437674 >> 172.21.15.26:0/3309437674 conn(0x7fe5c00fb340 msgr2=0x7fe5c00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 -- 172.21.15.26:0/3309437674 shutdown_connections 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.304+0000 7fe5c61ff700 1 -- 172.21.15.26:0/3309437674 wait complete. 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.306+0000 7fe5c61ff700 1 Processor -- start 2023-10-06T21:09:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.306+0000 7fe5c61ff700 1 -- start start 2023-10-06T21:09:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.306+0000 7fe5c61ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.306+0000 7fe5c61ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0104280 0x7fe5c00ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.306+0000 7fe5c61ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c00ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5c61ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01005e0 con 0x7fe5c0106bb0 2023-10-06T21:09:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5c61ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0100750 con 0x7fe5c0104280 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5c61ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01a0b70 con 0x7fe5c0101950 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c00ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36078/0 (socket says 172.21.15.26:36078) 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 -- 172.21.15.26:0/1937693322 learned_addr learned my addr 172.21.15.26:0/1937693322 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:22.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5beffd700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0104280 0x7fe5c00ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 -- 172.21.15.26:0/1937693322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0104280 msgr2=0x7fe5c00ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0104280 0x7fe5c00ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 -- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 msgr2=0x7fe5c00ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c00ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 -- 172.21.15.26:0/1937693322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00095f0 con 0x7fe5c0101950 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bffff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c00ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.307+0000 7fe5bf7fe700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00059d0 tx=0x7fe5b0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.308+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001d070 con 0x7fe5c0101950 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.308+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b000fb90 con 0x7fe5c0101950 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.308+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c01a0df0 con 0x7fe5c0101950 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.308+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00173f0 con 0x7fe5c0101950 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.308+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c01a11c0 con 0x7fe5c0101950 2023-10-06T21:09:22.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.309+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c010c6b0 con 0x7fe5c0101950 2023-10-06T21:09:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.311+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe5b000fd00 con 0x7fe5c0101950 2023-10-06T21:09:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.312+0000 7fe5bcff9700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 0x7fe5a80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.312+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5b00ce2b0 con 0x7fe5c0101950 2023-10-06T21:09:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.312+0000 7fe5beffd700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 0x7fe5a80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.313+0000 7fe5beffd700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 0x7fe5a80a3a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009c60 tx=0x7fe5b4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.314+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe5b0097800 con 0x7fe5c0101950 2023-10-06T21:09:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.578+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5c01a1350 con 0x7fe5c0101950 2023-10-06T21:09:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.579+0000 7fe5bcff9700 1 -- 172.21.15.26:0/1937693322 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7fe5b009ae20 con 0x7fe5c0101950 2023-10-06T21:09:22.582 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:22.583 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:09:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 msgr2=0x7fe5a80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 0x7fe5a80a3a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009c60 tx=0x7fe5b4009260 comp rx=0 tx=0).stop 2023-10-06T21:09:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 msgr2=0x7fe5c01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00059d0 tx=0x7fe5b0004730 comp rx=0 tx=0).stop 2023-10-06T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 shutdown_connections 2023-10-06T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5a80a1550 0x7fe5a80a3a10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5c0106bb0 0x7fe5c00ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5c0101950 0x7fe5c01012c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 --2- 172.21.15.26:0/1937693322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5c0104280 0x7fe5c00ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 >> 172.21.15.26:0/1937693322 conn(0x7fe5c00fb340 msgr2=0x7fe5c0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 shutdown_connections 2023-10-06T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.582+0000 7fe5c61ff700 1 -- 172.21.15.26:0/1937693322 wait complete. 2023-10-06T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:22.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 -- 172.21.15.26:0/3074599146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728073910 msgr2=0x7f3728073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 --2- 172.21.15.26:0/3074599146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728073910 0x7f3728073d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f37180098f0 tx=0x7f3718009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 -- 172.21.15.26:0/3074599146 shutdown_connections 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 --2- 172.21.15.26:0/3074599146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728108da0 0x7f372810b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 --2- 172.21.15.26:0/3074599146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728074ef0 0x7f3728073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 --2- 172.21.15.26:0/3074599146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728073910 0x7f3728073d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 -- 172.21.15.26:0/3074599146 >> 172.21.15.26:0/3074599146 conn(0x7f37280fc430 msgr2=0x7f37280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 -- 172.21.15.26:0/3074599146 shutdown_connections 2023-10-06T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.813+0000 7f37301ff700 1 -- 172.21.15.26:0/3074599146 wait complete. 2023-10-06T21:09:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.814+0000 7f37301ff700 1 Processor -- start 2023-10-06T21:09:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 -- start start 2023-10-06T21:09:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728074ef0 0x7f3728199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 0x7f37281a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281a09c0 con 0x7f3728073910 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281a0b00 con 0x7f3728074ef0 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f37301ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281a0c90 con 0x7f3728108da0 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39430/0 (socket says 172.21.15.26:39430) 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372df9b700 1 -- 172.21.15.26:0/572605349 learned_addr learned my addr 172.21.15.26:0/572605349 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372e79c700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 0x7f37281a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372d79a700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728074ef0 0x7f3728199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.815+0000 7f372df9b700 1 -- 172.21.15.26:0/572605349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 msgr2=0x7f37281a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372df9b700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 0x7f37281a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372df9b700 1 -- 172.21.15.26:0/572605349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728074ef0 msgr2=0x7f3728199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372df9b700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728074ef0 0x7f3728199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372df9b700 1 -- 172.21.15.26:0/572605349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240095f0 con 0x7f3728073910 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372e79c700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 0x7f37281a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f372df9b700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f372400ec10 tx=0x7f372400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400cc80 con 0x7f3728073910 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3720009c40 con 0x7f3728073910 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37240044e0 con 0x7f3728073910 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724010430 con 0x7f3728073910 2023-10-06T21:09:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.816+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3718009520 con 0x7f3728073910 2023-10-06T21:09:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.817+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3728107450 con 0x7f3728073910 2023-10-06T21:09:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.819+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3724003f90 con 0x7f3728073910 2023-10-06T21:09:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.820+0000 7f371effd700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 0x7f37140a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.821+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3724014070 con 0x7f3728073910 2023-10-06T21:09:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.821+0000 7f372d79a700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 0x7f37140a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:22.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.821+0000 7f372d79a700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 0x7f37140a3a90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3718009fd0 tx=0x7f3718019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:22.825+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f37240967c0 con 0x7f3728073910 2023-10-06T21:09:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.042+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f372810e920 con 0x7f3728073910 2023-10-06T21:09:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.044+0000 7f371effd700 1 -- 172.21.15.26:0/572605349 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f3724099de0 con 0x7f3728073910 2023-10-06T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 msgr2=0x7f37140a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 0x7f37140a3a90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3718009fd0 tx=0x7f3718019040 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 msgr2=0x7f3728198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f372400ec10 tx=0x7f372400ef20 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 shutdown_connections 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3728108da0 0x7f37281a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f37140a15d0 0x7f37140a3a90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3728073910 0x7f3728198e40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 --2- 172.21.15.26:0/572605349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728074ef0 0x7f3728199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 >> 172.21.15.26:0/572605349 conn(0x7f37280fc430 msgr2=0x7f37280fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 shutdown_connections 2023-10-06T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.047+0000 7f37301ff700 1 -- 172.21.15.26:0/572605349 wait complete. 2023-10-06T21:09:23.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:09:23.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:09:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:22 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/758212746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:22 smithi078 ceph-mon[134896]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:22 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:22 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1937693322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:23.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:22 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/758212746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:23.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:22 smithi026 ceph-mon[98490]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:23.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.134 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:22 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.135 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:22 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1937693322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:22 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/758212746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:22 smithi167 ceph-mon[134494]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:22 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:22 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1937693322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 -- 172.21.15.26:0/3954542941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 msgr2=0x7f99581024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 --2- 172.21.15.26:0/3954542941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f99581024f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9940009990 tx=0x7f9940009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 -- 172.21.15.26:0/3954542941 shutdown_connections 2023-10-06T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 --2- 172.21.15.26:0/3954542941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958102a30 0x7f9958109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 --2- 172.21.15.26:0/3954542941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958100e90 0x7f99581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 --2- 172.21.15.26:0/3954542941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f99581024f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 -- 172.21.15.26:0/3954542941 >> 172.21.15.26:0/3954542941 conn(0x7f99580fc430 msgr2=0x7f99580fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 -- 172.21.15.26:0/3954542941 shutdown_connections 2023-10-06T21:09:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.297+0000 7f995cbff700 1 -- 172.21.15.26:0/3954542941 wait complete. 2023-10-06T21:09:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.298+0000 7f995cbff700 1 Processor -- start 2023-10-06T21:09:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.298+0000 7f995cbff700 1 -- start start 2023-10-06T21:09:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 0x7f9958198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958102a30 0x7f99581a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99581a09c0 con 0x7f9958102070 2023-10-06T21:09:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99581a0b00 con 0x7f9958100e90 2023-10-06T21:09:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995cbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99581a0c90 con 0x7f9958102a30 2023-10-06T21:09:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f994e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 0x7f9958198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39456/0 (socket says 172.21.15.26:39456) 2023-10-06T21:09:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f995659c700 1 -- 172.21.15.26:0/3772571997 learned_addr learned my addr 172.21.15.26:0/3772571997 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f994e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 0x7f9958198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40036/0 (socket says 172.21.15.26:40036) 2023-10-06T21:09:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.299+0000 7f9956d9d700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958102a30 0x7f99581a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 -- 172.21.15.26:0/3772571997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958102a30 msgr2=0x7f99581a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958102a30 0x7f99581a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 -- 172.21.15.26:0/3772571997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 msgr2=0x7f9958198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 0x7f9958198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 -- 172.21.15.26:0/3772571997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9948009520 con 0x7f9958102070 2023-10-06T21:09:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995659c700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9940005720 tx=0x7f9940009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994001d070 con 0x7f9958102070 2023-10-06T21:09:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9940021410 con 0x7f9958102070 2023-10-06T21:09:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400095f0 con 0x7f9958102070 2023-10-06T21:09:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940017650 con 0x7f9958102070 2023-10-06T21:09:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.300+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99581a11a0 con 0x7f9958102070 2023-10-06T21:09:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.301+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995810c750 con 0x7f9958102070 2023-10-06T21:09:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.304+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9940021580 con 0x7f9958102070 2023-10-06T21:09:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.307+0000 7f994ffff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 0x7f99440a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.307+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99400ce4c0 con 0x7f9958102070 2023-10-06T21:09:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.307+0000 7f994e59c700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 0x7f99440a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.308+0000 7f994e59c700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 0x7f99440a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9958101be0 tx=0x7f9948009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.308+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9940097a10 con 0x7f9958102070 2023-10-06T21:09:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.521+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99581a1330 con 0x7f9958102070 2023-10-06T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.521+0000 7f994ffff700 1 -- 172.21.15.26:0/3772571997 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f994009b030 con 0x7f9958102070 2023-10-06T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 msgr2=0x7f99440a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 0x7f99440a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9958101be0 tx=0x7f9948009260 comp rx=0 tx=0).stop 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 msgr2=0x7f9958199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9940005720 tx=0x7f9940009580 comp rx=0 tx=0).stop 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 shutdown_connections 2023-10-06T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958102a30 0x7f99581a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958100e90 0x7f9958198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f99440a1510 0x7f99440a39d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 --2- 172.21.15.26:0/3772571997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958102070 0x7f9958199380 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.523+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 >> 172.21.15.26:0/3772571997 conn(0x7f99580fc430 msgr2=0x7f99580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.524+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 shutdown_connections 2023-10-06T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.524+0000 7f995cbff700 1 -- 172.21.15.26:0/3772571997 wait complete. 2023-10-06T21:09:23.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.750+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/2098369697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 msgr2=0x7f5e741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.750+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/2098369697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e741012b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009990 tx=0x7f5e64009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/2098369697 shutdown_connections 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/2098369697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 0x7f5e74109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/2098369697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e741012b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/2098369697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e741024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/2098369697 >> 172.21.15.26:0/2098369697 conn(0x7f5e740fc430 msgr2=0x7f5e740fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/2098369697 shutdown_connections 2023-10-06T21:09:23.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.751+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/2098369697 wait complete. 2023-10-06T21:09:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 Processor -- start 2023-10-06T21:09:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 -- start start 2023-10-06T21:09:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e74197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 0x7f5e7419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419e790 con 0x7f5e74100e90 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419e900 con 0x7f5e74102070 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e7a7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7419eac0 con 0x7f5e74102a30 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39474/0 (socket says 172.21.15.26:39474) 2023-10-06T21:09:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.752+0000 7f5e6bfff700 1 -- 172.21.15.26:0/1399943347 learned_addr learned my addr 172.21.15.26:0/1399943347 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e78d9c700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 0x7f5e7419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e73fff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e74197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 -- 172.21.15.26:0/1399943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 msgr2=0x7f5e7419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 0x7f5e7419e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 -- 172.21.15.26:0/1399943347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 msgr2=0x7f5e74197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e74197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 -- 172.21.15.26:0/1399943347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640095f0 con 0x7f5e74100e90 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e73fff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e74197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e6bfff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e64004710 tx=0x7f5e64009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6401d070 con 0x7f5e74100e90 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e64021410 con 0x7f5e74100e90 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64017650 con 0x7f5e74100e90 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.753+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7419ed40 con 0x7f5e74100e90 2023-10-06T21:09:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.754+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7419f110 con 0x7f5e74100e90 2023-10-06T21:09:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.755+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74107e90 con 0x7f5e74100e90 2023-10-06T21:09:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.757+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5e64021580 con 0x7f5e74100e90 2023-10-06T21:09:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.758+0000 7f5e71ffb700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 0x7f5e600a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.758+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e640ce290 con 0x7f5e74100e90 2023-10-06T21:09:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.758+0000 7f5e73fff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 0x7f5e600a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.759+0000 7f5e73fff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 0x7f5e600a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e74100bb0 tx=0x7f5e5c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.762+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5e640977e0 con 0x7f5e74100e90 2023-10-06T21:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.975+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e7410c750 con 0x7f5e74100e90 2023-10-06T21:09:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.975+0000 7f5e71ffb700 1 -- 172.21.15.26:0/1399943347 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e7410c750 con 0x7f5e74100e90 2023-10-06T21:09:23.978 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 msgr2=0x7f5e600a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 0x7f5e600a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e74100bb0 tx=0x7f5e5c009260 comp rx=0 tx=0).stop 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 msgr2=0x7f5e74196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5e64004710 tx=0x7f5e64009580 comp rx=0 tx=0).stop 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 shutdown_connections 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e74102a30 0x7f5e7419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f5e600a1590 0x7f5e600a3a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e74100e90 0x7f5e74196c50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 --2- 172.21.15.26:0/1399943347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e74102070 0x7f5e74197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 >> 172.21.15.26:0/1399943347 conn(0x7f5e740fc430 msgr2=0x7f5e740fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 shutdown_connections 2023-10-06T21:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:23.978+0000 7f5e7a7ff700 1 -- 172.21.15.26:0/1399943347 wait complete. 2023-10-06T21:09:23.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:09:24.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:23 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/572605349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:24.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:09:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:09:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:23 smithi078 ceph-mon[134896]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:24.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:23 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3772571997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 -- 172.21.15.26:0/201426377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4100ea0 msgr2=0x7f58f41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 --2- 172.21.15.26:0/201426377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4100ea0 0x7f58f41012c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009990 tx=0x7f58f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 -- 172.21.15.26:0/201426377 shutdown_connections 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 --2- 172.21.15.26:0/201426377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 --2- 172.21.15.26:0/201426377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4100ea0 0x7f58f41012c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 --2- 172.21.15.26:0/201426377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4102080 0x7f58f4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 -- 172.21.15.26:0/201426377 >> 172.21.15.26:0/201426377 conn(0x7f58f40fc460 msgr2=0x7f58f40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 -- 172.21.15.26:0/201426377 shutdown_connections 2023-10-06T21:09:24.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.203+0000 7f58fbbff700 1 -- 172.21.15.26:0/201426377 wait complete. 2023-10-06T21:09:24.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.204+0000 7f58fbbff700 1 Processor -- start 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.204+0000 7f58fbbff700 1 -- start start 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.204+0000 7f58fbbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4100ea0 0x7f58f4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.204+0000 7f58fbbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fbbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f41a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fbbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41a09c0 con 0x7f58f4102a40 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fbbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41a0b00 con 0x7f58f4102080 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fbbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41a0c90 con 0x7f58f4100ea0 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fa19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f41a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f999b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4100ea0 0x7f58f4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40056/0 (socket says 172.21.15.26:40056) 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 -- 172.21.15.26:0/1580672324 learned_addr learned my addr 172.21.15.26:0/1580672324 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 -- 172.21.15.26:0/1580672324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4100ea0 msgr2=0x7f58f4198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4100ea0 0x7f58f4198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 -- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 msgr2=0x7f58f41a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f41a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58f919a700 1 -- 172.21.15.26:0/1580672324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec009c40 con 0x7f58f4102080 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.205+0000 7f58fa19c700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58f919a700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58e400e970 tx=0x7f58e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4009790 con 0x7f58f4102080 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e400c7c0 con 0x7f58f4102080 2023-10-06T21:09:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00095f0 con 0x7f58f4102080 2023-10-06T21:09:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4003c20 con 0x7f58f4102080 2023-10-06T21:09:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.206+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f41a1260 con 0x7f58f4102080 2023-10-06T21:09:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.207+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4193550 con 0x7f58f4102080 2023-10-06T21:09:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.209+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f58e400c930 con 0x7f58f4102080 2023-10-06T21:09:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.210+0000 7f58eaffd700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 0x7f58e00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.210+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58e4014070 con 0x7f58f4102080 2023-10-06T21:09:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.211+0000 7f58f999b700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 0x7f58e00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.211+0000 7f58f999b700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 0x7f58e00a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b2e0 tx=0x7f58f0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:24.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.212+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58e4096540 con 0x7f58f4102080 2023-10-06T21:09:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:23 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/572605349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:09:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:09:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:23 smithi167 ceph-mon[134494]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:24.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:23 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3772571997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:23 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/572605349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:09:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:09:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:23 smithi026 ceph-mon[98490]: from='mgr.14148 172.21.15.26:0/3829480380' entity='mgr.x' 2023-10-06T21:09:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:23 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3772571997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:24.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.423+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58f410ca20 con 0x7f58f4102080 2023-10-06T21:09:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.424+0000 7f58eaffd700 1 -- 172.21.15.26:0/1580672324 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58e4099b60 con 0x7f58f4102080 2023-10-06T21:09:24.427 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:09:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 msgr2=0x7f58e00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 0x7f58e00a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58f000b2e0 tx=0x7f58f0005fb0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 msgr2=0x7f58f4199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58e400e970 tx=0x7f58e400ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:24.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 shutdown_connections 2023-10-06T21:09:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58e00a1510 0x7f58e00a39d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58f4102a40 0x7f58f41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58f4100ea0 0x7f58f4198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 --2- 172.21.15.26:0/1580672324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f4102080 0x7f58f4199380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 >> 172.21.15.26:0/1580672324 conn(0x7f58f40fc460 msgr2=0x7f58f4107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 shutdown_connections 2023-10-06T21:09:24.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.427+0000 7f58fbbff700 1 -- 172.21.15.26:0/1580672324 wait complete. 2023-10-06T21:09:24.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:09:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 -- 172.21.15.26:0/2748043038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 msgr2=0x7fe528077cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 --2- 172.21.15.26:0/2748043038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe528077cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b0a0 tx=0x7fe52000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 -- 172.21.15.26:0/2748043038 shutdown_connections 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 --2- 172.21.15.26:0/2748043038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe528077cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 --2- 172.21.15.26:0/2748043038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528075870 0x7fe528075c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 --2- 172.21.15.26:0/2748043038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528076ab0 0x7fe528076f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 -- 172.21.15.26:0/2748043038 >> 172.21.15.26:0/2748043038 conn(0x7fe5280fe660 msgr2=0x7fe528100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 -- 172.21.15.26:0/2748043038 shutdown_connections 2023-10-06T21:09:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.649+0000 7fe52ebff700 1 -- 172.21.15.26:0/2748043038 wait complete. 2023-10-06T21:09:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.650+0000 7fe52ebff700 1 Processor -- start 2023-10-06T21:09:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 -- start start 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528075870 0x7fe52819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528076ab0 0x7fe52819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a4df0 con 0x7fe528077470 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a4f30 con 0x7fe528075870 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52ebff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a50c0 con 0x7fe528076ab0 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52d19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52d19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39508/0 (socket says 172.21.15.26:39508) 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52d19c700 1 -- 172.21.15.26:0/1861851084 learned_addr learned my addr 172.21.15.26:0/1861851084 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe52c99b700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528075870 0x7fe52819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.651+0000 7fe527fff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528076ab0 0x7fe52819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 -- 172.21.15.26:0/1861851084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528076ab0 msgr2=0x7fe52819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528076ab0 0x7fe52819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 -- 172.21.15.26:0/1861851084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528075870 msgr2=0x7fe52819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528075870 0x7fe52819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 -- 172.21.15.26:0/1861851084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009d00 con 0x7fe528077470 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52d19c700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe520009fd0 tx=0x7fe52000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000e040 con 0x7fe528077470 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5200094a0 con 0x7fe528077470 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281a5340 con 0x7fe528077470 2023-10-06T21:09:24.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52001b6c0 con 0x7fe528077470 2023-10-06T21:09:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.652+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5281a57c0 con 0x7fe528077470 2023-10-06T21:09:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.653+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52810fb10 con 0x7fe528077470 2023-10-06T21:09:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.655+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe52001d470 con 0x7fe528077470 2023-10-06T21:09:24.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.658+0000 7fe525ffb700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 0x7fe5180a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.658+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5200cddc0 con 0x7fe528077470 2023-10-06T21:09:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.658+0000 7fe52c99b700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 0x7fe5180a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:24.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.658+0000 7fe52c99b700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 0x7fe5180a3a90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009c60 tx=0x7fe51c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:24.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.662+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe520097310 con 0x7fe528077470 2023-10-06T21:09:24.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.920+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe528112e00 con 0x7fe528077470 2023-10-06T21:09:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.920+0000 7fe525ffb700 1 -- 172.21.15.26:0/1861851084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7fe52009fe60 con 0x7fe528077470 2023-10-06T21:09:24.923 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:24.924 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 msgr2=0x7fe5180a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 0x7fe5180a3a90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009c60 tx=0x7fe51c009260 comp rx=0 tx=0).stop 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 msgr2=0x7fe5281a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe520009fd0 tx=0x7fe52000bce0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 shutdown_connections 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe5180a15d0 0x7fe5180a3a90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe528077470 0x7fe5281a4820 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528075870 0x7fe52819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 --2- 172.21.15.26:0/1861851084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe528076ab0 0x7fe52819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 >> 172.21.15.26:0/1861851084 conn(0x7fe5280fe660 msgr2=0x7fe5281001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 shutdown_connections 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:24.923+0000 7fe52ebff700 1 -- 172.21.15.26:0/1861851084 wait complete. 2023-10-06T21:09:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:24.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.146+0000 7f9bb81ff700 1 -- 172.21.15.26:0/460326168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0102a40 msgr2=0x7f9bb0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.146+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/460326168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0102a40 0x7f9bb0109360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800b0a0 tx=0x7f9ba800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 -- 172.21.15.26:0/460326168 shutdown_connections 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/460326168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0102a40 0x7f9bb0109360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/460326168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0100ea0 0x7f9bb01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/460326168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 -- 172.21.15.26:0/460326168 >> 172.21.15.26:0/460326168 conn(0x7f9bb00fc420 msgr2=0x7f9bb00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 -- 172.21.15.26:0/460326168 shutdown_connections 2023-10-06T21:09:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.147+0000 7f9bb81ff700 1 -- 172.21.15.26:0/460326168 wait complete. 2023-10-06T21:09:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 Processor -- start 2023-10-06T21:09:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 -- start start 2023-10-06T21:09:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0102a40 0x7f9bb01a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb01a0a60 con 0x7f9bb0100ea0 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb01a0ba0 con 0x7f9bb0102080 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb81ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb01a0d30 con 0x7f9bb0102a40 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb5f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb5f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39524/0 (socket says 172.21.15.26:39524) 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb5f9b700 1 -- 172.21.15.26:0/4241783477 learned_addr learned my addr 172.21.15.26:0/4241783477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.148+0000 7f9bb579a700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb679c700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0102a40 0x7f9bb01a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 -- 172.21.15.26:0/4241783477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0102a40 msgr2=0x7f9bb01a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0102a40 0x7f9bb01a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 -- 172.21.15.26:0/4241783477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 msgr2=0x7f9bb0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 -- 172.21.15.26:0/4241783477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8009d00 con 0x7f9bb0100ea0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb579a700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb5f9b700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d980 tx=0x7f9bac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac004020 con 0x7f9bb0100ea0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bac009730 con 0x7f9bb0100ea0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb01a0fb0 con 0x7f9bb0100ea0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.149+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003a30 con 0x7f9bb0100ea0 2023-10-06T21:09:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.150+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb01a1430 con 0x7f9bb0100ea0 2023-10-06T21:09:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.151+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb010c760 con 0x7f9bb0100ea0 2023-10-06T21:09:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.153+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9bac0098a0 con 0x7f9bb0100ea0 2023-10-06T21:09:25.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.154+0000 7f9ba6ffd700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 0x7f9b9c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.154+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bac0ccb90 con 0x7f9bb0100ea0 2023-10-06T21:09:25.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.154+0000 7f9bb579a700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 0x7f9b9c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.155+0000 7f9bb579a700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 0x7f9b9c0a3a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0100bc0 tx=0x7f9ba0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.158+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9bac096050 con 0x7f9bb0100ea0 2023-10-06T21:09:25.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:24 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1399943347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:24 smithi167 ceph-mon[134494]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:25.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:24 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1580672324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:24 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1399943347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:24 smithi026 ceph-mon[98490]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:25.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:24 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1580672324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:24 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1399943347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:24 smithi078 ceph-mon[134896]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:25.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:24 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1580672324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.370+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bb010c9a0 con 0x7f9bb0100ea0 2023-10-06T21:09:25.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.372+0000 7f9ba6ffd700 1 -- 172.21.15.26:0/4241783477 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f9bac099670 con 0x7f9bb0100ea0 2023-10-06T21:09:25.375 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:25.375 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 msgr2=0x7f9b9c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 0x7f9b9c0a3a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0100bc0 tx=0x7f9ba0009260 comp rx=0 tx=0).stop 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 msgr2=0x7f9bb0198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d980 tx=0x7f9bac00dc90 comp rx=0 tx=0).stop 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 shutdown_connections 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb0102a40 0x7f9bb01a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f9b9c0a1550 0x7f9b9c0a3a10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0100ea0 0x7f9bb0198ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 --2- 172.21.15.26:0/4241783477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bb0102080 0x7f9bb0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 >> 172.21.15.26:0/4241783477 conn(0x7f9bb00fc420 msgr2=0x7f9bb01079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 shutdown_connections 2023-10-06T21:09:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.374+0000 7f9bb81ff700 1 -- 172.21.15.26:0/4241783477 wait complete. 2023-10-06T21:09:25.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:09:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 -- 172.21.15.26:0/1059512182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8102a40 msgr2=0x7f2ab8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/1059512182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8102a40 0x7f2ab8109360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009f90 tx=0x7f2ab000b330 comp rx=0 tx=0).stop 2023-10-06T21:09:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 -- 172.21.15.26:0/1059512182 shutdown_connections 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/1059512182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8102a40 0x7f2ab8109360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/1059512182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8100ea0 0x7f2ab81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/1059512182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102080 0x7f2ab8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 -- 172.21.15.26:0/1059512182 >> 172.21.15.26:0/1059512182 conn(0x7f2ab80fc420 msgr2=0x7f2ab80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 -- 172.21.15.26:0/1059512182 shutdown_connections 2023-10-06T21:09:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.598+0000 7f2abe7ff700 1 -- 172.21.15.26:0/1059512182 wait complete. 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.599+0000 7f2abe7ff700 1 Processor -- start 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.599+0000 7f2abe7ff700 1 -- start start 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 0x7f2ab8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8102080 0x7f2ab8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81a09b0 con 0x7f2ab8102a40 2023-10-06T21:09:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81a0af0 con 0x7f2ab8100ea0 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abe7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81a0c80 con 0x7f2ab8102080 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39530/0 (socket says 172.21.15.26:39530) 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 -- 172.21.15.26:0/2825214136 learned_addr learned my addr 172.21.15.26:0/2825214136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2ab77fe700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8102080 0x7f2ab8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2ab7fff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 0x7f2ab8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 -- 172.21.15.26:0/2825214136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8102080 msgr2=0x7f2ab8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8102080 0x7f2ab8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 -- 172.21.15.26:0/2825214136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 msgr2=0x7f2ab8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 0x7f2ab8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2abcd9c700 1 -- 172.21.15.26:0/2825214136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0095f0 con 0x7f2ab8102a40 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.600+0000 7f2ab7fff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 0x7f2ab8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2abcd9c700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009f60 tx=0x7f2ab0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000e040 con 0x7f2ab8102a40 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab00093a0 con 0x7f2ab8102a40 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab0009c40 con 0x7f2ab8102a40 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00052a0 con 0x7f2ab8102a40 2023-10-06T21:09:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.601+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab81a11a0 con 0x7f2ab8102a40 2023-10-06T21:09:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.602+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab810c710 con 0x7f2ab8102a40 2023-10-06T21:09:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.604+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2ab001d470 con 0x7f2ab8102a40 2023-10-06T21:09:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.605+0000 7f2ab57fa700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 0x7f2aa00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.606+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ab00ceac0 con 0x7f2ab8102a40 2023-10-06T21:09:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.606+0000 7f2ab7fff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 0x7f2aa00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.606+0000 7f2ab7fff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 0x7f2aa00a3a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8101bf0 tx=0x7f2aa8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.607+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2ab0098010 con 0x7f2ab8102a40 2023-10-06T21:09:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.868+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab810c900 con 0x7f2ab8102a40 2023-10-06T21:09:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.869+0000 7f2ab57fa700 1 -- 172.21.15.26:0/2825214136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f2ab009b630 con 0x7f2ab8102a40 2023-10-06T21:09:25.872 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:25.872 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:09:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 msgr2=0x7f2aa00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 0x7f2aa00a3a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8101bf0 tx=0x7f2aa8009260 comp rx=0 tx=0).stop 2023-10-06T21:09:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 msgr2=0x7f2ab81a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009f60 tx=0x7f2ab0004ce0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 shutdown_connections 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f2aa00a1550 0x7f2aa00a3a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ab8102a40 0x7f2ab81a03e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ab8100ea0 0x7f2ab8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 --2- 172.21.15.26:0/2825214136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ab8102080 0x7f2ab8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.871+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 >> 172.21.15.26:0/2825214136 conn(0x7f2ab80fc420 msgr2=0x7f2ab80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.872+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 shutdown_connections 2023-10-06T21:09:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:25.872+0000 7f2abe7ff700 1 -- 172.21.15.26:0/2825214136 wait complete. 2023-10-06T21:09:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:25.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T21:08:16.065653+0000', 'modified': '2023-10-06T21:08:21.849944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24559, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6835/4281425672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 4281425672}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 4281425672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2946984451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2946984451}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2946984451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/1710830195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 1710830195}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 1710830195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.78:6837/1931966907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 1931966907}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 1931966907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6841/2891556508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2891556508}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2891556508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6839/3790769886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3790769886}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3790769886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T21:09:25.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T21:09:25.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-06T21:09:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.096+0000 7f9596dff700 1 -- 172.21.15.26:0/2631107837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102a30 msgr2=0x7f9590109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.096+0000 7f9596dff700 1 --2- 172.21.15.26:0/2631107837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102a30 0x7f9590109350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f958800b0a0 tx=0x7f958800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 -- 172.21.15.26:0/2631107837 shutdown_connections 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 --2- 172.21.15.26:0/2631107837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102a30 0x7f9590109350 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 --2- 172.21.15.26:0/2631107837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590100e90 0x7f95901012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 --2- 172.21.15.26:0/2631107837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590102070 0x7f95901024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 -- 172.21.15.26:0/2631107837 >> 172.21.15.26:0/2631107837 conn(0x7f95900fc430 msgr2=0x7f95900fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 -- 172.21.15.26:0/2631107837 shutdown_connections 2023-10-06T21:09:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.097+0000 7f9596dff700 1 -- 172.21.15.26:0/2631107837 wait complete. 2023-10-06T21:09:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 Processor -- start 2023-10-06T21:09:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 -- start start 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590100e90 0x7f9590194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590102a30 0x7f959019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959019c650 con 0x7f9590102070 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959019c790 con 0x7f9590102a30 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f9596dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959019c920 con 0x7f9590100e90 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f958ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f958ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39546/0 (socket says 172.21.15.26:39546) 2023-10-06T21:09:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.098+0000 7f958ffff700 1 -- 172.21.15.26:0/4128810847 learned_addr learned my addr 172.21.15.26:0/4128810847 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f9594b9b700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590100e90 0x7f9590194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f959539c700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590102a30 0x7f959019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 -- 172.21.15.26:0/4128810847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590100e90 msgr2=0x7f9590194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590100e90 0x7f9590194ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 -- 172.21.15.26:0/4128810847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590102a30 msgr2=0x7f959019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590102a30 0x7f959019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 -- 172.21.15.26:0/4128810847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584009520 con 0x7f9590102070 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958ffff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f957c012f90 tx=0x7f957c0100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958dffb700 1 -- 172.21.15.26:0/4128810847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c01c070 con 0x7f9590102070 2023-10-06T21:09:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9588009d00 con 0x7f9590102070 2023-10-06T21:09:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958dffb700 1 -- 172.21.15.26:0/4128810847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957c013040 con 0x7f9590102070 2023-10-06T21:09:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.099+0000 7f958dffb700 1 -- 172.21.15.26:0/4128810847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c010cf0 con 0x7f9590102070 2023-10-06T21:09:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.100+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959019cf20 con 0x7f9590102070 2023-10-06T21:09:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.103+0000 7f958dffb700 1 -- 172.21.15.26:0/4128810847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f957c00b5d0 con 0x7f9590102070 2023-10-06T21:09:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 Processor -- start 2023-10-06T21:09:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 -- start start 2023-10-06T21:09:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf534b410 con 0x560bf53e3b70 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf533cbd0 con 0x560bf53e3210 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.104+0000 7f95aa783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf533a4f0 con 0x560bf53e4a10 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f958dffb700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 0x7f95800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f958dffb700 1 -- 172.21.15.26:0/4128810847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f957c0d0590 con 0x7f9590102070 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f959539c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39560/0 (socket says 172.21.15.26:39560) 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 -- 172.21.15.26:0/663039005 learned_addr learned my addr 172.21.15.26:0/663039005 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f959539c700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 0x7f95800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f958ffff700 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 -- 172.21.15.26:0/663039005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 msgr2=0x560bf53e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 -- 172.21.15.26:0/663039005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 msgr2=0x560bf53e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.105+0000 7f9594b9b700 1 -- 172.21.15.26:0/663039005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bf5160d90 con 0x560bf53e3b70 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f9594b9b700 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9584009eb0 tx=0x7f9584011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bd5086e8f123a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f959539c700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 0x7f95800a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f958800b780 tx=0x7f9588006230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f957affd700 1 -- 172.21.15.26:0/663039005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584019630 con 0x560bf53e3b70 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f957affd700 1 -- 172.21.15.26:0/663039005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9584019c50 con 0x560bf53e3b70 2023-10-06T21:09:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f957affd700 1 -- 172.21.15.26:0/663039005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400ed70 con 0x560bf53e3b70 2023-10-06T21:09:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f95aa783b80 1 -- 172.21.15.26:0/663039005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 msgr2=0x560bf53e4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.106+0000 7f95aa783b80 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9584009eb0 tx=0x7f9584011720 comp rx=0 tx=0).stop 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 -- 172.21.15.26:0/663039005 shutdown_connections 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 --2- 172.21.15.26:0/663039005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e4430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 -- 172.21.15.26:0/663039005 >> 172.21.15.26:0/663039005 conn(0x560bf5357210 msgr2=0x560bf53de0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 -- 172.21.15.26:0/663039005 shutdown_connections 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 -- 172.21.15.26:0/663039005 wait complete. 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 -- start start 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.107+0000 7f95aa783b80 1 Processor -- start 2023-10-06T21:09:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf5335de0 con 0x560bf53e4a10 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf52d26c0 con 0x560bf53e3210 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf52d2360 con 0x560bf53e3b70 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f95aa783b80 10 client.0 _ll_register_callbacks cb 0x560bf512f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f958ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f958ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36218/0 (socket says 172.21.15.26:36218) 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f958ffff700 1 -- 172.21.15.26:0/2674248341 learned_addr learned my addr 172.21.15.26:0/2674248341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f959539c700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f9594b9b700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f958ffff700 1 -- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 msgr2=0x560bf53ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.108+0000 7f958ffff700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f958ffff700 1 -- 172.21.15.26:0/2674248341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 msgr2=0x560bf53f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f958ffff700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f958ffff700 1 -- 172.21.15.26:0/2674248341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9584009ae0 con 0x560bf53e3b70 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f958ffff700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f957c099ce0 tx=0x7f957c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6634af8f01129d6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f956ffff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c09dee0 con 0x560bf53e3b70 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f957c09d7e0 con 0x560bf53e3b70 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bf53d5ec0 con 0x560bf53e3b70 2023-10-06T21:09:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f957c0a1930 con 0x560bf53e3b70 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f95aa783b80 10 client.24967 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560bf5149c10 con 0x560bf53e3b70 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.109+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560bf53d5ec0 con 0x560bf53e3b70 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f957c0b8020 con 0x560bf53e3b70 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 10 client.24967 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 10 client.24967 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 20 client.24967 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf53f8af0 0x560bf53f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 4 client.24967 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bf532edc0 con 0x560bf53f8af0 2023-10-06T21:09:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.110+0000 7f9594b9b700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf53f8af0 0x560bf53f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.111+0000 7f9594b9b700 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf53f8af0 0x560bf53f75f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=718a6fb5a92fe7ac server_cookie=33075b6c83e9aa81 in_seq=0 out_seq=0 2023-10-06T21:09:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.111+0000 7f956ffff700 10 client.24967 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f956ffff700 1 -- 172.21.15.26:0/2674248341 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560bf532edc0 con 0x560bf53f8af0 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f956ffff700 10 client.24967 handle_command_reply: tid=0 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 client.24967 shutdown 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 msgr2=0x560bf53ed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f957c099ce0 tx=0x7f957c020000 comp rx=0 tx=0).stop 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 shutdown_connections 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf53f8af0 0x560bf53f75f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 --2- 172.21.15.26:0/2674248341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ed4e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f956ffff700 0 client.24967 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 >> 172.21.15.26:0/2674248341 conn(0x560bf5357210 msgr2=0x560bf53ddf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 shutdown_connections 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 1 -- 172.21.15.26:0/2674248341 wait complete. 2023-10-06T21:09:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.112+0000 7f95aa783b80 20 client.24967 trim_cache size 0 max 16384 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 Processor -- start 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 -- start start 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf51b68d0 con 0x560bf53e3b70 2023-10-06T21:09:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf53d82b0 con 0x560bf53e3210 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.130+0000 7f95aa783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf5331ed0 con 0x560bf53e4a10 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39586/0 (socket says 172.21.15.26:39586) 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 -- 172.21.15.26:0/3907620182 learned_addr learned my addr 172.21.15.26:0/3907620182 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f9594b9b700 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f959539c700 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 -- 172.21.15.26:0/3907620182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 msgr2=0x560bf53f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 -- 172.21.15.26:0/3907620182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 msgr2=0x560bf53e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 -- 172.21.15.26:0/3907620182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bf5187200 con 0x560bf53e3b70 2023-10-06T21:09:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.131+0000 7f958ffff700 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f957c005950 tx=0x7f957c09bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1475103cd1263f45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f957b7fe700 1 -- 172.21.15.26:0/3907620182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c010e70 con 0x560bf53e3b70 2023-10-06T21:09:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f957b7fe700 1 -- 172.21.15.26:0/3907620182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957c0b6020 con 0x560bf53e3b70 2023-10-06T21:09:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f957b7fe700 1 -- 172.21.15.26:0/3907620182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c0a58b0 con 0x560bf53e3b70 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 -- 172.21.15.26:0/3907620182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 msgr2=0x560bf53e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f957c005950 tx=0x7f957c09bd90 comp rx=0 tx=0).stop 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 -- 172.21.15.26:0/3907620182 shutdown_connections 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e4a10 0x560bf53f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3907620182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e3b70 0x560bf53e7e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 -- 172.21.15.26:0/3907620182 >> 172.21.15.26:0/3907620182 conn(0x560bf5357210 msgr2=0x560bf53eada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 -- 172.21.15.26:0/3907620182 shutdown_connections 2023-10-06T21:09:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.132+0000 7f95aa783b80 1 -- 172.21.15.26:0/3907620182 wait complete. 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 -- start start 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 Processor -- start 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf53f4720 con 0x560bf53e4a10 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf53f4890 con 0x560bf53e3210 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf544c4e0 con 0x560bf53e3b70 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.133+0000 7f95aa783b80 10 client.0 _ll_register_callbacks cb 0x560bf512f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f958ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f958ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36226/0 (socket says 172.21.15.26:36226) 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f958ffff700 1 -- 172.21.15.26:0/3201484173 learned_addr learned my addr 172.21.15.26:0/3201484173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f9594b9b700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 -- 172.21.15.26:0/3201484173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 msgr2=0x560bf53ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 -- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 msgr2=0x560bf53eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 -- 172.21.15.26:0/3201484173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bf5149c10 con 0x560bf53e4a10 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f959539c700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f41e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9588009fd0 tx=0x7f958800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5c736cdfcd9a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f956ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.134+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800e040 con 0x560bf53e4a10 2023-10-06T21:09:26.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9584019c50 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9588020650 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9588020810 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 10 client.14934 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560bf515f260 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9584019c50 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f958802c040 con 0x560bf53e4a10 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 10 client.14934 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 10 client.14934 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 20 client.14934 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf54517b0 0x560bf544dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 4 client.14934 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.135+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560bf532eae0 con 0x560bf54517b0 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f9594b9b700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf54517b0 0x560bf544dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f956ffff700 10 client.14934 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f9594b9b700 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf54517b0 0x560bf544dc30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7c13c267fa6e6e6 server_cookie=47f4fd072a50a09 in_seq=0 out_seq=0 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f956ffff700 1 -- 172.21.15.26:0/3201484173 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560bf532eae0 con 0x560bf54517b0 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f956ffff700 10 client.14934 handle_command_reply: tid=0 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.136+0000 7f95aa783b80 1 client.14934 shutdown 2023-10-06T21:09:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 msgr2=0x560bf53f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f41e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9588009fd0 tx=0x7f958800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 shutdown_connections 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560bf54517b0 0x560bf544dc30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bf53e4a10 0x560bf53f41e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f956ffff700 0 client.14934 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bf53e3210 0x560bf53eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 --2- 172.21.15.26:0/3201484173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bf53e3b70 0x560bf53ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 >> 172.21.15.26:0/3201484173 conn(0x560bf5357210 msgr2=0x560bf53e78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 shutdown_connections 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 1 -- 172.21.15.26:0/3201484173 wait complete. 2023-10-06T21:09:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f95aa783b80 20 client.14934 trim_cache size 0 max 16384 2023-10-06T21:09:26.146 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:26.146 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:26.146 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2023-10-06T21:09:26.148 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24539, 2023-10-06T21:09:26.148 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:26.148 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:26.148 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 65.291378567999999, 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 86.749905667999997 2023-10-06T21:09:26.149 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 msgr2=0x7f95800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 0x7f95800a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f958800b780 tx=0x7f9588006230 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 msgr2=0x7f9590195010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.137+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f957c012f90 tx=0x7f957c0100c0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 shutdown_connections 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590102a30 0x7f959019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9590100e90 0x7f9590194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f95800a1510 0x7f95800a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 --2- 172.21.15.26:0/4128810847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9590102070 0x7f9590195010 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 >> 172.21.15.26:0/4128810847 conn(0x7f95900fc430 msgr2=0x7f95900fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 shutdown_connections 2023-10-06T21:09:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.138+0000 7f9596dff700 1 -- 172.21.15.26:0/4128810847 wait complete. 2023-10-06T21:09:26.157 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:25 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1861851084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.157 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:25 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4241783477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T21:09:26.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:25 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1861851084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:25 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4241783477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 -- 172.21.15.26:0/4031592248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110102070 msgr2=0x7ff1101024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 --2- 172.21.15.26:0/4031592248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110102070 0x7ff1101024f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009990 tx=0x7ff0f8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 -- 172.21.15.26:0/4031592248 shutdown_connections 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 --2- 172.21.15.26:0/4031592248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 0x7ff110109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 --2- 172.21.15.26:0/4031592248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110100e90 0x7ff1101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 --2- 172.21.15.26:0/4031592248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110102070 0x7ff1101024f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 -- 172.21.15.26:0/4031592248 >> 172.21.15.26:0/4031592248 conn(0x7ff1100fc430 msgr2=0x7ff1100fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.362+0000 7ff1157ff700 1 -- 172.21.15.26:0/4031592248 shutdown_connections 2023-10-06T21:09:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.363+0000 7ff1157ff700 1 -- 172.21.15.26:0/4031592248 wait complete. 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.363+0000 7ff1157ff700 1 Processor -- start 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 -- start start 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110102070 0x7ff110199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 0x7ff1101a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101a09c0 con 0x7ff110100e90 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101a0b00 con 0x7ff110102a30 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff1157ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101a0c90 con 0x7ff110102070 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39606/0 (socket says 172.21.15.26:39606) 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 -- 172.21.15.26:0/520168519 learned_addr learned my addr 172.21.15.26:0/520168519 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff10effd700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110102070 0x7ff110199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 0x7ff1101a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 -- 172.21.15.26:0/520168519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110102070 msgr2=0x7ff110199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110102070 0x7ff110199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 -- 172.21.15.26:0/520168519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 msgr2=0x7ff1101a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.364+0000 7ff106ffd700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 0x7ff1101a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff106ffd700 1 -- 172.21.15.26:0/520168519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80095f0 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff106ffd700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff10000e970 tx=0x7ff10000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff10cff9700 1 -- 172.21.15.26:0/520168519 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100009790 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff10cff9700 1 -- 172.21.15.26:0/520168519 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1000044e0 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1101a0f70 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff10cff9700 1 -- 172.21.15.26:0/520168519 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100003e30 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.365+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1101a1390 con 0x7ff110100e90 2023-10-06T21:09:26.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:25 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1861851084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:25 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4241783477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:26.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.368+0000 7ff10cff9700 1 -- 172.21.15.26:0/520168519 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff100004000 con 0x7ff110100e90 2023-10-06T21:09:26.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.370+0000 7ff129041b80 1 Processor -- start 2023-10-06T21:09:26.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.370+0000 7ff129041b80 1 -- start start 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586807f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x5586807f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868075e410 con 0x5586807f6210 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868074fbd0 con 0x5586807f6b70 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff129041b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868074d4f0 con 0x5586807f7a10 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10cff9700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 0x7ff0fc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10cff9700 1 -- 172.21.15.26:0/520168519 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff100014070 con 0x7ff110100e90 2023-10-06T21:09:26.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x5586807f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39612/0 (socket says 172.21.15.26:39612) 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10f7fe700 1 -- 172.21.15.26:0/2544128736 learned_addr learned my addr 172.21.15.26:0/2544128736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586807f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.371+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 0x7ff0fc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 -- 172.21.15.26:0/2544128736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 msgr2=0x5586807f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586807f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 -- 172.21.15.26:0/2544128736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 msgr2=0x5586807f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 -- 172.21.15.26:0/2544128736 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558680573d90 con 0x5586807f7a10 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff106ffd700 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586807f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:26.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10effd700 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x5586807f7e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80098e0 tx=0x7ff0f8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddea0518e64adff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 0x7ff0fc0a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff108012fd0 tx=0x7ff10800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff1057fa700 1 -- 172.21.15.26:0/2544128736 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800beb0 con 0x5586807f7a10 2023-10-06T21:09:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff1057fa700 1 -- 172.21.15.26:0/2544128736 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f801e040 con 0x5586807f7a10 2023-10-06T21:09:26.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.372+0000 7ff1057fa700 1 -- 172.21.15.26:0/2544128736 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004b70 con 0x5586807f7a10 2023-10-06T21:09:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 -- 172.21.15.26:0/2544128736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 msgr2=0x5586807f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x5586807f7e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80098e0 tx=0x7ff0f8005e80 comp rx=0 tx=0).stop 2023-10-06T21:09:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 -- 172.21.15.26:0/2544128736 shutdown_connections 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x5586807f7e90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6210 0x5586807f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 --2- 172.21.15.26:0/2544128736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586807f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 -- 172.21.15.26:0/2544128736 >> 172.21.15.26:0/2544128736 conn(0x55868076a210 msgr2=0x5586807f10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 -- 172.21.15.26:0/2544128736 shutdown_connections 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.373+0000 7ff129041b80 1 -- 172.21.15.26:0/2544128736 wait complete. 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.374+0000 7ff129041b80 1 -- start start 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.374+0000 7ff129041b80 1 Processor -- start 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.374+0000 7ff129041b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.374+0000 7ff129041b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.374+0000 7ff129041b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f7a10 0x5586808074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff129041b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680748de0 con 0x5586807f6b70 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff129041b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586806e56c0 con 0x5586807f7a10 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff129041b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586806e5360 con 0x5586807f6210 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff129041b80 10 client.0 _ll_register_callbacks cb 0x5586805427b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f7a10 0x5586808074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39616/0 (socket says 172.21.15.26:39616) 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 -- 172.21.15.26:0/1506099599 learned_addr learned my addr 172.21.15.26:0/1506099599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff106ffd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 -- 172.21.15.26:0/1506099599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 msgr2=0x5586807fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 -- 172.21.15.26:0/1506099599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f7a10 msgr2=0x5586808074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f7a10 0x5586808074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 -- 172.21.15.26:0/1506099599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff108012cc0 con 0x5586807f6b70 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff10effd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ce0 tx=0x7ff0f8004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b997eb12633e22f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.375+0000 7ff0f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8005240 con 0x5586807f6b70 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586807e8ec0 con 0x5586807f6b70 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0f80053a0 con 0x5586807f6b70 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f8004570 con 0x5586807f6b70 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff129041b80 10 client.14946 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff1101a1390 con 0x5586807f6b70 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586807e8ec0 con 0x5586807f6b70 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff0f802b430 con 0x5586807f6b70 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.376+0000 7ff129041b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff129041b80 10 client.14946 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff129041b80 20 client.14946 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55868080b6b0 0x55868080a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff129041b80 4 client.14946 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558680741dc0 con 0x55868080b6b0 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff106ffd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55868080b6b0 0x55868080a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff106ffd700 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55868080b6b0 0x55868080a420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf5f810e50a91ee5 server_cookie=d2a0d8e731805050 in_seq=0 out_seq=0 2023-10-06T21:09:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.377+0000 7ff0f67fc700 10 client.14946 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1506099599 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558680741dc0 con 0x55868080b6b0 2023-10-06T21:09:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff0f67fc700 10 client.14946 handle_command_reply: tid=0 2023-10-06T21:09:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 client.14946 shutdown 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 msgr2=0x5586808004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ce0 tx=0x7ff0f8004b20 comp rx=0 tx=0).stop 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 shutdown_connections 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f7a10 0x5586808074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55868080b6b0 0x55868080a420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff0f67fc700 0 client.14946 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 --2- 172.21.15.26:0/1506099599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x5586808004b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 >> 172.21.15.26:0/1506099599 conn(0x55868076a210 msgr2=0x5586807f0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.378+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 shutdown_connections 2023-10-06T21:09:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.379+0000 7ff129041b80 1 -- 172.21.15.26:0/1506099599 wait complete. 2023-10-06T21:09:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.379+0000 7ff129041b80 20 client.14946 trim_cache size 0 max 16384 2023-10-06T21:09:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.397+0000 7ff129041b80 1 Processor -- start 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.397+0000 7ff129041b80 1 -- start start 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586808074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586805c98d0 con 0x5586807f7a10 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680744ed0 con 0x5586807f6b70 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff129041b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586805d5de0 con 0x5586807f6210 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39628/0 (socket says 172.21.15.26:39628) 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 -- 172.21.15.26:0/2157362119 learned_addr learned my addr 172.21.15.26:0/2157362119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10effd700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586808074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff106ffd700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 -- 172.21.15.26:0/2157362119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 msgr2=0x5586807fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 -- 172.21.15.26:0/2157362119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 msgr2=0x5586808074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586808074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10f7fe700 1 -- 172.21.15.26:0/2157362119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110109c80 con 0x5586807f7a10 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.398+0000 7ff10effd700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586808074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.399+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff108019ea0 tx=0x7ff108009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d275f9a13501ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.399+0000 7ff105ffb700 1 -- 172.21.15.26:0/2157362119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10802a070 con 0x5586807f7a10 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.399+0000 7ff105ffb700 1 -- 172.21.15.26:0/2157362119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff108020040 con 0x5586807f7a10 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.399+0000 7ff105ffb700 1 -- 172.21.15.26:0/2157362119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10800e040 con 0x5586807f7a10 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- 172.21.15.26:0/2157362119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 msgr2=0x5586807efb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff108019ea0 tx=0x7ff108009920 comp rx=0 tx=0).stop 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- 172.21.15.26:0/2157362119 shutdown_connections 2023-10-06T21:09:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f7a10 0x5586807efb70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f6210 0x5586807fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 --2- 172.21.15.26:0/2157362119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6b70 0x5586808074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- 172.21.15.26:0/2157362119 >> 172.21.15.26:0/2157362119 conn(0x55868076a210 msgr2=0x558680814010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- 172.21.15.26:0/2157362119 shutdown_connections 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- 172.21.15.26:0/2157362119 wait complete. 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 -- start start 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.400+0000 7ff129041b80 1 Processor -- start 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6210 0x558680801750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x558680806cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680807200 con 0x5586807f6b70 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680807370 con 0x5586807f6210 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680807530 con 0x5586807f7a10 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff129041b80 10 client.0 _ll_register_callbacks cb 0x7ff108012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6210 0x558680801750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39630/0 (socket says 172.21.15.26:39630) 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 -- 172.21.15.26:0/1089083391 learned_addr learned my addr 172.21.15.26:0/1089083391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10f7fe700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x558680806cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 -- 172.21.15.26:0/1089083391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 msgr2=0x558680806cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x558680806cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 -- 172.21.15.26:0/1089083391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6210 msgr2=0x558680801750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6210 0x558680801750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.401+0000 7ff10effd700 1 -- 172.21.15.26:0/1089083391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1101a1390 con 0x5586807f6b70 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff10effd700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001eb0 tx=0x7ff0f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c615e8967870531 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff0f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8005160 con 0x5586807f6b70 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f801e040 con 0x5586807f6b70 2023-10-06T21:09:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0f8004b90 con 0x5586807f6b70 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f802bb30 con 0x5586807f6b70 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff129041b80 10 client.14952 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff108019be0 con 0x5586807f6b70 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.402+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f801e040 con 0x5586807f6b70 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff0f8037a30 con 0x5586807f6b70 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 10 client.14952 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 10 client.14952 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T21:09:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 20 client.14952 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x5586808639d0 0x55868085fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 4 client.14952 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558680741ae0 con 0x5586808639d0 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff106ffd700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x5586808639d0 0x55868085fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff106ffd700 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x5586808639d0 0x55868085fe50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee12032e08d93ba2 server_cookie=cbf1511af045a47d in_seq=0 out_seq=0 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.403+0000 7ff0f67fc700 10 client.14952 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff0f67fc700 1 -- 172.21.15.26:0/1089083391 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558680741ae0 con 0x5586808639d0 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff0f67fc700 10 client.14952 handle_command_reply: tid=0 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 client.14952 shutdown 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 msgr2=0x558680801c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001eb0 tx=0x7ff0f8004730 comp rx=0 tx=0).stop 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 shutdown_connections 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x5586808639d0 0x55868085fe50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586807f7a10 0x558680806cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586807f6210 0x558680801750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff0f67fc700 0 client.14952 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 --2- 172.21.15.26:0/1089083391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586807f6b70 0x558680801c90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 >> 172.21.15.26:0/1089083391 conn(0x55868076a210 msgr2=0x5586807fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 shutdown_connections 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 1 -- 172.21.15.26:0/1089083391 wait complete. 2023-10-06T21:09:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.404+0000 7ff129041b80 20 client.14952 trim_cache size 0 max 16384 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24559, 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 66.568168948999997, 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:26.413 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:26.414 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 78.620419656999999 2023-10-06T21:09:26.414 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 msgr2=0x7ff0fc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 0x7ff0fc0a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff108012fd0 tx=0x7ff10800b040 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 msgr2=0x7ff110198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff10000e970 tx=0x7ff10000ed30 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 shutdown_connections 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff110102a30 0x7ff1101a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7ff0fc0a1510 0x7ff0fc0a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff110100e90 0x7ff110198e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 --2- 172.21.15.26:0/520168519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110102070 0x7ff110199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 >> 172.21.15.26:0/520168519 conn(0x7ff1100fc430 msgr2=0x7ff1100fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 shutdown_connections 2023-10-06T21:09:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.405+0000 7ff1157ff700 1 -- 172.21.15.26:0/520168519 wait complete. 2023-10-06T21:09:26.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T21:09:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 -- 172.21.15.26:0/1189018777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102070 msgr2=0x7f37081024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 --2- 172.21.15.26:0/1189018777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102070 0x7f37081024f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009990 tx=0x7f36f8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 -- 172.21.15.26:0/1189018777 shutdown_connections 2023-10-06T21:09:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 --2- 172.21.15.26:0/1189018777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102a30 0x7f3708109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 --2- 172.21.15.26:0/1189018777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f37081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 --2- 172.21.15.26:0/1189018777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102070 0x7f37081024f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 -- 172.21.15.26:0/1189018777 >> 172.21.15.26:0/1189018777 conn(0x7f37080fc430 msgr2=0x7f37080fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 -- 172.21.15.26:0/1189018777 shutdown_connections 2023-10-06T21:09:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.629+0000 7f37105ff700 1 -- 172.21.15.26:0/1189018777 wait complete. 2023-10-06T21:09:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 Processor -- start 2023-10-06T21:09:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 -- start start 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102070 0x7f3708199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 0x7f37081a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37081a09b0 con 0x7f3708102a30 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37081a0af0 con 0x7f3708102070 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.631+0000 7f37105ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37081a0c80 con 0x7f3708100e90 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 0x7f37081a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36280/0 (socket says 172.21.15.26:36280) 2023-10-06T21:09:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 -- 172.21.15.26:0/1721835447 learned_addr learned my addr 172.21.15.26:0/1721835447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370db9a700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102070 0x7f3708199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 -- 172.21.15.26:0/1721835447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102070 msgr2=0x7f3708199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102070 0x7f3708199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 -- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 msgr2=0x7f37081a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 0x7f37081a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 -- 172.21.15.26:0/1721835447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009520 con 0x7f3708100e90 2023-10-06T21:09:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370eb9c700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 0x7f37081a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.632+0000 7f370e39b700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f370400ea50 tx=0x7f370400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.633+0000 7f36ff7fe700 1 -- 172.21.15.26:0/1721835447 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400cbd0 con 0x7f3708100e90 2023-10-06T21:09:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.633+0000 7f36ff7fe700 1 -- 172.21.15.26:0/1721835447 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370400cd30 con 0x7f3708100e90 2023-10-06T21:09:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.633+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f80095f0 con 0x7f3708100e90 2023-10-06T21:09:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.633+0000 7f36ff7fe700 1 -- 172.21.15.26:0/1721835447 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704018680 con 0x7f3708100e90 2023-10-06T21:09:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.633+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37081a1220 con 0x7f3708100e90 2023-10-06T21:09:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.636+0000 7f36ff7fe700 1 -- 172.21.15.26:0/1721835447 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3704010430 con 0x7f3708100e90 2023-10-06T21:09:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.637+0000 7f3723e2bb80 1 Processor -- start 2023-10-06T21:09:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 -- start start 2023-10-06T21:09:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bdcf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bdd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdd0a10 0x56485bdd0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bd37410 con 0x56485bdcfb70 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bd28bd0 con 0x56485bdcf210 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bd264f0 con 0x56485bdd0a10 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f370e39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bdd0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f370e39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bdd0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39660/0 (socket says 172.21.15.26:39660) 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f370e39b700 1 -- 172.21.15.26:0/2347060583 learned_addr learned my addr 172.21.15.26:0/2347060583 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f370eb9c700 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bdcf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.638+0000 7f370db9a700 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdd0a10 0x56485bdd0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f36ff7fe700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 0x7f36f40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 -- 172.21.15.26:0/2347060583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdd0a10 msgr2=0x56485bdd0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdd0a10 0x56485bdd0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 -- 172.21.15.26:0/2347060583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 msgr2=0x56485bdd0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bdd0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 -- 172.21.15.26:0/2347060583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56485bb4cd90 con 0x56485bdcf210 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f36ff7fe700 1 -- 172.21.15.26:0/1721835447 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3704014070 con 0x7f3708100e90 2023-10-06T21:09:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 0x7f36f40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.639+0000 7f370eb9c700 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bdcf630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3700007630 tx=0x7f3700007940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c42fa797189f8db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f36fcbfe700 1 -- 172.21.15.26:0/2347060583 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700012710 con 0x56485bdcf210 2023-10-06T21:09:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f36fcbfe700 1 -- 172.21.15.26:0/2347060583 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370000f040 con 0x56485bdcf210 2023-10-06T21:09:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f36fcbfe700 1 -- 172.21.15.26:0/2347060583 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700021840 con 0x56485bdcf210 2023-10-06T21:09:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f370eb9c700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 0x7f36f40a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f370001f000 tx=0x7f3700020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f3723e2bb80 1 -- 172.21.15.26:0/2347060583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 msgr2=0x56485bdcf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.640+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bdcf630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3700007630 tx=0x7f3700007940 comp rx=0 tx=0).stop 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 -- 172.21.15.26:0/2347060583 shutdown_connections 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdd0a10 0x56485bdd0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bdcf630 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/2347060583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bdd0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 -- 172.21.15.26:0/2347060583 >> 172.21.15.26:0/2347060583 conn(0x56485bd43210 msgr2=0x56485bdca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 -- 172.21.15.26:0/2347060583 shutdown_connections 2023-10-06T21:09:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 -- 172.21.15.26:0/2347060583 wait complete. 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 -- start start 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.641+0000 7f3723e2bb80 1 Processor -- start 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bdd9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485bde0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bd21de0 con 0x56485bdcf210 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bcbe6c0 con 0x56485bdd0a10 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bcbe360 con 0x56485bdcfb70 2023-10-06T21:09:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f3723e2bb80 10 client.0 _ll_register_callbacks cb 0x56485bb1b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39672/0 (socket says 172.21.15.26:39672) 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 -- 172.21.15.26:0/967447859 learned_addr learned my addr 172.21.15.26:0/967447859 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370eb9c700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485bde0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370db9a700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bdd9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 -- 172.21.15.26:0/967447859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 msgr2=0x56485bdd9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bdd9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 -- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 msgr2=0x56485bde0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485bde0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.642+0000 7f370e39b700 1 -- 172.21.15.26:0/967447859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3700009d90 con 0x56485bdcf210 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f370e39b700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37040a2380 tx=0x7f37040cb240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80048d70d5b3c2e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704096040 con 0x56485bdcf210 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f370409ac20 con 0x56485bdcf210 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56485bdc1ec0 con 0x56485bdcf210 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f370409a2f0 con 0x56485bdcf210 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f3723e2bb80 10 client.14964 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56485bb35c10 con 0x56485bdcf210 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56485bdc1ec0 con 0x56485bdcf210 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.643+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37040a2420 con 0x56485bdcf210 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 10 client.14964 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 20 client.14964 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485bde4b00 0x56485bde36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 4 client.14964 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56485bd1adc0 con 0x56485bde4b00 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f370db9a700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485bde4b00 0x56485bde36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f370db9a700 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485bde4b00 0x56485bde36b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe5a32f1ae39123 server_cookie=f805f4ce3e023fbd in_seq=0 out_seq=0 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.644+0000 7f36f1ffb700 10 client.14964 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.645+0000 7f36f1ffb700 1 -- 172.21.15.26:0/967447859 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56485bd1adc0 con 0x56485bde4b00 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.645+0000 7f36f1ffb700 10 client.14964 handle_command_reply: tid=0 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.645+0000 7f3723e2bb80 1 client.14964 shutdown 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.645+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 msgr2=0x56485bdd8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.645+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37040a2380 tx=0x7f37040cb240 comp rx=0 tx=0).stop 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 shutdown_connections 2023-10-06T21:09:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485bde4b00 0x56485bde36b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485bde0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcf210 0x56485bdd8fd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f36f1ffb700 0 client.14964 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/967447859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bdd9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 >> 172.21.15.26:0/967447859 conn(0x56485bd43210 msgr2=0x56485bdc9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 shutdown_connections 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 1 -- 172.21.15.26:0/967447859 wait complete. 2023-10-06T21:09:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.646+0000 7f3723e2bb80 20 client.14964 trim_cache size 0 max 16384 2023-10-06T21:09:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 Processor -- start 2023-10-06T21:09:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 -- start start 2023-10-06T21:09:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bde36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bde0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bba28d0 con 0x56485bdd0a10 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bdc42b0 con 0x56485bdcf210 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.664+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bd1adc0 con 0x56485bdcfb70 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39680/0 (socket says 172.21.15.26:39680) 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 -- 172.21.15.26:0/3115966047 learned_addr learned my addr 172.21.15.26:0/3115966047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370e39b700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bde36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370db9a700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bde0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 -- 172.21.15.26:0/3115966047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 msgr2=0x56485bde0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bde0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 -- 172.21.15.26:0/3115966047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 msgr2=0x56485bde36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bde36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 -- 172.21.15.26:0/3115966047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56485bb73200 con 0x56485bdd0a10 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370e39b700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bde36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.665+0000 7f370eb9c700 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37000094f0 tx=0x7f3700012710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06b044ec2277495 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f36fd3ff700 1 -- 172.21.15.26:0/3115966047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700021ba0 con 0x56485bdd0a10 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f36fd3ff700 1 -- 172.21.15.26:0/3115966047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3700021d00 con 0x56485bdd0a10 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f36fd3ff700 1 -- 172.21.15.26:0/3115966047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700005130 con 0x56485bdd0a10 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 -- 172.21.15.26:0/3115966047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 msgr2=0x56485bde3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37000094f0 tx=0x7f3700012710 comp rx=0 tx=0).stop 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 -- 172.21.15.26:0/3115966047 shutdown_connections 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdd0a10 0x56485bde3bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdcf210 0x56485bde36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/3115966047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcfb70 0x56485bde0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 -- 172.21.15.26:0/3115966047 >> 172.21.15.26:0/3115966047 conn(0x56485bd43210 msgr2=0x56485bdd3980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 -- 172.21.15.26:0/3115966047 shutdown_connections 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.666+0000 7f3723e2bb80 1 -- 172.21.15.26:0/3115966047 wait complete. 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.667+0000 7f3723e2bb80 1 -- start start 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.667+0000 7f3723e2bb80 1 Processor -- start 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.667+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.667+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bddd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.667+0000 7f3723e2bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485be384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bde0650 con 0x56485bdcfb70 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bde07c0 con 0x56485bdd0a10 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f3723e2bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485bde0980 con 0x56485bdcf210 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f3723e2bb80 10 client.0 _ll_register_callbacks cb 0x56485bb1b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36302/0 (socket says 172.21.15.26:36302) 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 -- 172.21.15.26:0/1723428416 learned_addr learned my addr 172.21.15.26:0/1723428416 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370eb9c700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485be384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370db9a700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bddd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 -- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 msgr2=0x56485be384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485be384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 -- 172.21.15.26:0/1723428416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 msgr2=0x56485bddd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bddd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 -- 172.21.15.26:0/1723428416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56485bb35c10 con 0x56485bdcf210 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f370e39b700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f370409a4b0 tx=0x7f37040850b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3b984e7e2ae1d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.668+0000 7f36f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:09:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704096030 con 0x56485bdcf210 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f370000f040 con 0x56485bdcf210 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f370409ac20 con 0x56485bdcf210 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f370409a2f0 con 0x56485bdcf210 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f3723e2bb80 10 client.25009 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f370001a040 con 0x56485bdcf210 2023-10-06T21:09:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.669+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370000f040 con 0x56485bdcf210 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3704096310 con 0x56485bdcf210 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 10 client.25009 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 10 client.25009 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 20 client.25009 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485be3d690 0x56485be39b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 4 client.25009 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56485bd1aae0 con 0x56485be3d690 2023-10-06T21:09:26.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f370db9a700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485be3d690 0x56485be39b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f36f1ffb700 10 client.25009 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.670+0000 7f370db9a700 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485be3d690 0x56485be39b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3ff45604886754 server_cookie=bc7f56bf2635e066 in_seq=0 out_seq=0 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f36f1ffb700 1 -- 172.21.15.26:0/1723428416 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56485bd1aae0 con 0x56485be3d690 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f36f1ffb700 10 client.25009 handle_command_reply: tid=0 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 client.25009 shutdown 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 msgr2=0x56485bddd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f370409a4b0 tx=0x7f37040850b0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 shutdown_connections 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56485be3d690 0x56485be39b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56485bdd0a10 0x56485be384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56485bdcf210 0x56485bddd0f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f36f1ffb700 0 client.25009 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 --2- 172.21.15.26:0/1723428416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56485bdcfb70 0x56485bddd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 >> 172.21.15.26:0/1723428416 conn(0x56485bd43210 msgr2=0x56485bde36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 shutdown_connections 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 1 -- 172.21.15.26:0/1723428416 wait complete. 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.671+0000 7f3723e2bb80 20 client.25009 trim_cache size 0 max 16384 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-10-06T21:09:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24587, 2023-10-06T21:09:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 70.141325703999996, 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 76.543854369000002 2023-10-06T21:09:26.682 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 msgr2=0x7f36f40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 0x7f36f40a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f370001f000 tx=0x7f3700020000 comp rx=0 tx=0).stop 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 msgr2=0x7f3708198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f370400ea50 tx=0x7f370400ed60 comp rx=0 tx=0).stop 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 shutdown_connections 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f36f40a1510 0x7f36f40a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3708102a30 0x7f37081a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3708100e90 0x7f3708198e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 --2- 172.21.15.26:0/1721835447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3708102070 0x7f3708199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 >> 172.21.15.26:0/1721835447 conn(0x7f37080fc430 msgr2=0x7f37080fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 shutdown_connections 2023-10-06T21:09:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:26.672+0000 7f37105ff700 1 -- 172.21.15.26:0/1721835447 wait complete. 2023-10-06T21:09:26.693 INFO:tasks.cephadm:Setup complete, yielding 2023-10-06T21:09:26.693 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T21:09:26.703 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:09:26.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-06T21:09:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:26 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2825214136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:27.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:26 smithi026 ceph-mon[98490]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:26 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2825214136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:26 smithi167 ceph-mon[134494]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:27.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:26 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2825214136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:27.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:26 smithi078 ceph-mon[134896]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:27.435 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:28.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 -- 172.21.15.26:0/3139796881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 msgr2=0x7fdc641093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:28.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/3139796881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc641093c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00b0d0 tx=0x7fdc5c00b3e0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 -- 172.21.15.26:0/3139796881 shutdown_connections 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/3139796881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc641093c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/3139796881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 0x7fdc64101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/3139796881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc641025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 -- 172.21.15.26:0/3139796881 >> 172.21.15.26:0/3139796881 conn(0x7fdc640fc4d0 msgr2=0x7fdc640fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 -- 172.21.15.26:0/3139796881 shutdown_connections 2023-10-06T21:09:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.958+0000 7fdc6acb4700 1 -- 172.21.15.26:0/3139796881 wait complete. 2023-10-06T21:09:28.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.959+0000 7fdc6acb4700 1 Processor -- start 2023-10-06T21:09:28.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 -- start start 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 0x7fdc64074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc640732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc64073f60 con 0x7fdc64102b10 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640740d0 con 0x7fdc64100f70 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc6acb4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc641a0dc0 con 0x7fdc64102170 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc69251700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc69251700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39726/0 (socket says 172.21.15.26:39726) 2023-10-06T21:09:28.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc69251700 1 -- 172.21.15.26:0/2143281153 learned_addr learned my addr 172.21.15.26:0/2143281153 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.960+0000 7fdc68a50700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 0x7fdc64074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc63fff700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc640732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 -- 172.21.15.26:0/2143281153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 msgr2=0x7fdc640732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc640732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 -- 172.21.15.26:0/2143281153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 msgr2=0x7fdc64074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 0x7fdc64074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 -- 172.21.15.26:0/2143281153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c009d00 con 0x7fdc64102b10 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc63fff700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc640732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.961+0000 7fdc69251700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c007990 tx=0x7fdc5c008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.962+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00e040 con 0x7fdc64102b10 2023-10-06T21:09:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.962+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5c00bdb0 con 0x7fdc64102b10 2023-10-06T21:09:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.962+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc641a1040 con 0x7fdc64102b10 2023-10-06T21:09:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.962+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01d470 con 0x7fdc64102b10 2023-10-06T21:09:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.962+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc641a14b0 con 0x7fdc64102b10 2023-10-06T21:09:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.963+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6410c7d0 con 0x7fdc64102b10 2023-10-06T21:09:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.965+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fdc5c0050b0 con 0x7fdc64102b10 2023-10-06T21:09:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.966+0000 7fdc61ffb700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 0x7fdc540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.966+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc5c0cdec0 con 0x7fdc64102b10 2023-10-06T21:09:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.966+0000 7fdc68a50700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 0x7fdc540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.967+0000 7fdc68a50700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 0x7fdc540a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdc64101fd0 tx=0x7fdc58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:28.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:28.970+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdc5c097410 con 0x7fdc64102b10 2023-10-06T21:09:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:28 smithi026 ceph-mon[98490]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:28 smithi078 ceph-mon[134896]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.200+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc640612f0 con 0x7fdc540a1510 2023-10-06T21:09:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.201+0000 7fdc61ffb700 1 -- 172.21.15.26:0/2143281153 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdc640612f0 con 0x7fdc540a1510 2023-10-06T21:09:29.204 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2023-10-06T21:09:29.204 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2023-10-06T21:09:29.204 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 msgr2=0x7fdc540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 0x7fdc540a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdc64101fd0 tx=0x7fdc58009260 comp rx=0 tx=0).stop 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 msgr2=0x7fdc640737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c007990 tx=0x7fdc5c008eb0 comp rx=0 tx=0).stop 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 shutdown_connections 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdc540a1510 0x7fdc540a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc64102b10 0x7fdc640737e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64100f70 0x7fdc64074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 --2- 172.21.15.26:0/2143281153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc64102170 0x7fdc640732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 >> 172.21.15.26:0/2143281153 conn(0x7fdc640fc4d0 msgr2=0x7fdc640fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 shutdown_connections 2023-10-06T21:09:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:29.204+0000 7fdc6acb4700 1 -- 172.21.15.26:0/2143281153 wait complete. 2023-10-06T21:09:29.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:28 smithi167 ceph-mon[134494]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:29.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-06T21:09:30.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:29 smithi026 ceph-mon[98490]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:30.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:29 smithi078 ceph-mon[134896]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:30.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:29 smithi167 ceph-mon[134494]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:30.454 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:31.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:30 smithi026 ceph-mon[98490]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:30 smithi078 ceph-mon[134896]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:30 smithi167 ceph-mon[134494]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 -- 172.21.15.26:0/1306132061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70102b10 msgr2=0x7f0d701093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 --2- 172.21.15.26:0/1306132061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70102b10 0x7f0d701093c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800b0d0 tx=0x7f0d6800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 -- 172.21.15.26:0/1306132061 shutdown_connections 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 --2- 172.21.15.26:0/1306132061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70102b10 0x7f0d701093c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 --2- 172.21.15.26:0/1306132061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70100f70 0x7f0d70101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 --2- 172.21.15.26:0/1306132061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 -- 172.21.15.26:0/1306132061 >> 172.21.15.26:0/1306132061 conn(0x7f0d700fc4d0 msgr2=0x7f0d700fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 -- 172.21.15.26:0/1306132061 shutdown_connections 2023-10-06T21:09:32.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.090+0000 7f0d767e6700 1 -- 172.21.15.26:0/1306132061 wait complete. 2023-10-06T21:09:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.091+0000 7f0d767e6700 1 Processor -- start 2023-10-06T21:09:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 -- start start 2023-10-06T21:09:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d70197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70102b10 0x7f0d7019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019e860 con 0x7f0d70100f70 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019e9a0 con 0x7f0d70102170 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d767e6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7019eb60 con 0x7f0d70102b10 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40204/0 (socket says 172.21.15.26:40204) 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2513837281 learned_addr learned my addr 172.21.15.26:0/2513837281 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d6f7fe700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d70197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.092+0000 7f0d74d83700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70102b10 0x7f0d7019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2513837281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70102b10 msgr2=0x7f0d7019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70102b10 0x7f0d7019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2513837281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 msgr2=0x7f0d70197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d70197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2513837281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68009d00 con 0x7f0d70100f70 2023-10-06T21:09:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6f7fe700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d70197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6ffff700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000e970 tx=0x7f0d6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60009790 con 0x7f0d70100f70 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6000c7e0 con 0x7f0d70100f70 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7019ee40 con 0x7f0d70100f70 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60003c70 con 0x7f0d70100f70 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.093+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7019f250 con 0x7f0d70100f70 2023-10-06T21:09:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.094+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d701084f0 con 0x7f0d70100f70 2023-10-06T21:09:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.097+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0d6000cc50 con 0x7f0d70100f70 2023-10-06T21:09:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.099+0000 7f0d6d7fa700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 0x7f0d580a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.099+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d60014070 con 0x7f0d70100f70 2023-10-06T21:09:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.099+0000 7f0d6f7fe700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 0x7f0d580a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.100+0000 7f0d6f7fe700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 0x7f0d580a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d64011490 tx=0x7f0d64009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.103+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0d60096560 con 0x7f0d70100f70 2023-10-06T21:09:32.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.322+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d7010c780 con 0x7f0d580a1510 2023-10-06T21:09:32.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.337+0000 7f0d6d7fa700 1 -- 172.21.15.26:0/2513837281 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f0d7010c780 con 0x7f0d580a1510 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (84s) 9s ago 84s 18.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 579a08620200 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi078 running (82s) 10s ago 82s 30.4M - 18.0.0-6616-g14d8d032 c754feb42fd5 f0f127b434a3 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi167 running (95s) 11s ago 95s 19.5M - 18.0.0-6616-g14d8d032 c754feb42fd5 fa74e3dda64c 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (102s) 9s ago 101s 13.7M - 18.0.0-6616-g14d8d032 c754feb42fd5 88b0cef1984d 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi078 running (100s) 10s ago 99s 20.3M - 18.0.0-6616-g14d8d032 c754feb42fd5 573aac2361ab 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi167 running (97s) 11s ago 97s 16.5M - 18.0.0-6616-g14d8d032 c754feb42fd5 7bfd323c5f0c 2023-10-06T21:09:32.340 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (88s) 9s ago 87s 13.7M - 18.0.0-6616-g14d8d032 c754feb42fd5 be12dbaefb96 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi078 running (93s) 10s ago 92s 21.7M - 18.0.0-6616-g14d8d032 c754feb42fd5 9cc262cd94e8 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi167 running (91s) 11s ago 90s 18.4M - 18.0.0-6616-g14d8d032 c754feb42fd5 fe178e39e8d6 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (105s) 9s ago 104s 13.8M - 18.0.0-6616-g14d8d032 c754feb42fd5 693fc5ce939d 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi078 running (108s) 10s ago 107s 15.2M - 18.0.0-6616-g14d8d032 c754feb42fd5 dcf57726f0ea 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi167 running (110s) 11s ago 110s 17.5M - 18.0.0-6616-g14d8d032 c754feb42fd5 a1f7060d066f 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (8m) 9s ago 8m 526M - 18.0.0-6616-g14d8d032 c754feb42fd5 57773bc11cd6 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi078 *:8443,8765 running (6m) 11s ago 6m 451M - 18.0.0-6616-g14d8d032 c754feb42fd5 1196f0b7f519 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi167 *:8443,8765 running (6m) 11s ago 6m 451M - 18.0.0-6616-g14d8d032 c754feb42fd5 0efcc8cf6f9b 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (8m) 9s ago 8m 92.6M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 3b01bf7adf1b 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi078 running (6m) 11s ago 6m 66.3M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 67126393c925 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi167 running (7m) 11s ago 7m 63.8M 2048M 18.0.0-6616-g14d8d032 c754feb42fd5 86b13814b181 2023-10-06T21:09:32.341 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (6m) 9s ago 6m 65.0M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 205a46981790 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi078 running (5m) 11s ago 5m 72.0M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 7ddcbcb35927 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi167 running (5m) 11s ago 5m 67.5M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 2db713565844 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (5m) 9s ago 5m 60.3M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 4bb04e4ae481 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi078 running (4m) 11s ago 4m 59.4M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 4ae038d1fc0c 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi167 running (4m) 11s ago 4m 64.2M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 2237ef1a66af 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (4m) 9s ago 4m 57.3M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 a92cc39c8058 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi078 running (3m) 11s ago 3m 60.5M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 42f4c94b6a70 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi167 running (3m) 11s ago 3m 59.7M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 4a5f10800e19 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (2m) 9s ago 2m 55.1M 4318M 18.0.0-6616-g14d8d032 c754feb42fd5 d11b0e43d3b7 2023-10-06T21:09:32.342 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi078 running (2m) 11s ago 2m 60.8M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 7c2dff36c78d 2023-10-06T21:09:32.343 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi167 running (2m) 11s ago 2m 52.8M 4292M 18.0.0-6616-g14d8d032 c754feb42fd5 15048537e2e4 2023-10-06T21:09:32.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 msgr2=0x7f0d580a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:32.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 0x7f0d580a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d64011490 tx=0x7f0d64009260 comp rx=0 tx=0).stop 2023-10-06T21:09:32.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 msgr2=0x7f0d70196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000e970 tx=0x7f0d6000ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 shutdown_connections 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d70102b10 0x7f0d7019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0d580a1510 0x7f0d580a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d70100f70 0x7f0d70196d10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 --2- 172.21.15.26:0/2513837281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d70102170 0x7f0d70197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.339+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 >> 172.21.15.26:0/2513837281 conn(0x7f0d700fc4d0 msgr2=0x7f0d70105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.340+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 shutdown_connections 2023-10-06T21:09:32.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:32.340+0000 7f0d767e6700 1 -- 172.21.15.26:0/2513837281 wait complete. 2023-10-06T21:09:32.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-06T21:09:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:33.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:33 smithi167 ceph-mon[134494]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:33.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:33 smithi167 ceph-mon[134494]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:33 smithi026 ceph-mon[98490]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:33.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:33 smithi026 ceph-mon[98490]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:33.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:33 smithi078 ceph-mon[134896]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:33.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:33 smithi078 ceph-mon[134896]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 -- 172.21.15.26:0/4160088249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac100f70 msgr2=0x7f6aac101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 --2- 172.21.15.26:0/4160088249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac100f70 0x7f6aac101390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0099c0 tx=0x7f6a9c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 -- 172.21.15.26:0/4160088249 shutdown_connections 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 --2- 172.21.15.26:0/4160088249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac102b10 0x7f6aac1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 --2- 172.21.15.26:0/4160088249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac100f70 0x7f6aac101390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 --2- 172.21.15.26:0/4160088249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102170 0x7f6aac1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.013+0000 7f6ab240f700 1 -- 172.21.15.26:0/4160088249 >> 172.21.15.26:0/4160088249 conn(0x7f6aac0fc4d0 msgr2=0x7f6aac0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:35.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.014+0000 7f6ab240f700 1 -- 172.21.15.26:0/4160088249 shutdown_connections 2023-10-06T21:09:35.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.014+0000 7f6ab240f700 1 -- 172.21.15.26:0/4160088249 wait complete. 2023-10-06T21:09:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 Processor -- start 2023-10-06T21:09:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 -- start start 2023-10-06T21:09:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:35.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 0x7f6aac199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102b10 0x7f6aac1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1a09b0 con 0x7f6aac102170 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1a0af0 con 0x7f6aac100f70 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.015+0000 7f6ab240f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1a0c80 con 0x7f6aac102b10 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 0x7f6aac199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55918/0 (socket says 172.21.15.26:55918) 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 -- 172.21.15.26:0/3603805595 learned_addr learned my addr 172.21.15.26:0/3603805595 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6ab09ac700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102b10 0x7f6aac1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 -- 172.21.15.26:0/3603805595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102b10 msgr2=0x7f6aac1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102b10 0x7f6aac1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 -- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 msgr2=0x7f6aac199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 0x7f6aac199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 -- 172.21.15.26:0/3603805595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0095f0 con 0x7f6aac100f70 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aab7fe700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 0x7f6aac199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.016+0000 7f6aabfff700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0058e0 tx=0x7f6a9c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.017+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c01d070 con 0x7f6aac100f70 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.017+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c005050 con 0x7f6aac100f70 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.017+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac1a0f00 con 0x7f6aac100f70 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.017+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c004b10 con 0x7f6aac100f70 2023-10-06T21:09:35.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.017+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac1a12b0 con 0x7f6aac100f70 2023-10-06T21:09:35.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.018+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac193560 con 0x7f6aac100f70 2023-10-06T21:09:35.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.020+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6a9c0051c0 con 0x7f6aac100f70 2023-10-06T21:09:35.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.022+0000 7f6aa97fa700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 0x7f6a940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.022+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a9c0cde80 con 0x7f6aac100f70 2023-10-06T21:09:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.022+0000 7f6aab7fe700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 0x7f6a940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:35.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.022+0000 7f6aab7fe700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 0x7f6a940a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0010420 tx=0x7f6aa00073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:35.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.024+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a9c0974f0 con 0x7f6aac100f70 2023-10-06T21:09:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.243+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6aac0008d0 con 0x7f6a940a1590 2023-10-06T21:09:35.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.249+0000 7f6aa97fa700 1 -- 172.21.15.26:0/3603805595 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f6aac0008d0 con 0x7f6a940a1590 2023-10-06T21:09:35.252 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-06T21:09:35.252 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 14s ago 115s smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi078=b;smithi078=e;smithi078=h;smithi078=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2023-10-06T21:09:35.252 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 14s ago 6m smithi026=x;smithi078=y;smithi167=z;count:3 2023-10-06T21:09:35.252 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 14s ago 7m smithi026:172.21.15.26=a;smithi078:172.21.15.78=b;smithi167:172.21.15.167=c;count:3 2023-10-06T21:09:35.252 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 14s ago - 2023-10-06T21:09:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 msgr2=0x7f6a940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 0x7f6a940a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0010420 tx=0x7f6aa00073d0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 msgr2=0x7f6aac198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0058e0 tx=0x7f6a9c00f690 comp rx=0 tx=0).stop 2023-10-06T21:09:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 shutdown_connections 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6aac102b10 0x7f6aac1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aac100f70 0x7f6aac198e20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f6a940a1590 0x7f6a940a3a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 --2- 172.21.15.26:0/3603805595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aac102170 0x7f6aac199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 >> 172.21.15.26:0/3603805595 conn(0x7f6aac0fc4d0 msgr2=0x7f6aac107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.251+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 shutdown_connections 2023-10-06T21:09:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:35.252+0000 7f6ab240f700 1 -- 172.21.15.26:0/3603805595 wait complete. 2023-10-06T21:09:35.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:35 smithi026 ceph-mon[98490]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:35.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-06T21:09:35.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:35 smithi167 ceph-mon[134494]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:35.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:35 smithi078 ceph-mon[134896]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:36.502 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:36 smithi167 ceph-mon[134494]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:36.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:36 smithi026 ceph-mon[98490]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:36.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:36 smithi078 ceph-mon[134896]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:37.748 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:37 smithi026 ceph-mon[98490]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:37 smithi167 ceph-mon[134494]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:37.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:37 smithi078 ceph-mon[134896]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 -- 172.21.15.26:0/3735949915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c100f70 msgr2=0x7f901c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 --2- 172.21.15.26:0/3735949915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c100f70 0x7f901c101390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f900c0099c0 tx=0x7f900c009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 -- 172.21.15.26:0/3735949915 shutdown_connections 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 --2- 172.21.15.26:0/3735949915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102b10 0x7f901c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 --2- 172.21.15.26:0/3735949915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c100f70 0x7f901c101390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 --2- 172.21.15.26:0/3735949915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c102170 0x7f901c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 -- 172.21.15.26:0/3735949915 >> 172.21.15.26:0/3735949915 conn(0x7f901c0fc4d0 msgr2=0x7f901c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 -- 172.21.15.26:0/3735949915 shutdown_connections 2023-10-06T21:09:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.007+0000 7f9021876700 1 -- 172.21.15.26:0/3735949915 wait complete. 2023-10-06T21:09:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.008+0000 7f9021876700 1 Processor -- start 2023-10-06T21:09:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.008+0000 7f9021876700 1 -- start start 2023-10-06T21:09:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102170 0x7f901c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c102b10 0x7f901c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c19e800 con 0x7f901c102b10 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c19e940 con 0x7f901c100f70 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f9021876700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c19eb00 con 0x7f901c102170 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102170 0x7f901c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55940/0 (socket says 172.21.15.26:55940) 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 -- 172.21.15.26:0/4067338689 learned_addr learned my addr 172.21.15.26:0/4067338689 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901b7fe700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c102b10 0x7f901c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 -- 172.21.15.26:0/4067338689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102170 msgr2=0x7f901c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102170 0x7f901c1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 -- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c102b10 msgr2=0x7f901c19e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c102b10 0x7f901c19e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.009+0000 7f901affd700 1 -- 172.21.15.26:0/4067338689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0095f0 con 0x7f901c100f70 2023-10-06T21:09:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f901affd700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f900c009fd0 tx=0x7f900c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c01d070 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f900c005050 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c19ed80 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c004a70 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.010+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c19f190 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.011+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c108560 con 0x7f901c100f70 2023-10-06T21:09:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.013+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f900c005450 con 0x7f901c100f70 2023-10-06T21:09:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.014+0000 7f9020874700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 0x7f90040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.015+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f900c0cdc90 con 0x7f901c100f70 2023-10-06T21:09:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.015+0000 7f901a7fc700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 0x7f90040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:38.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.015+0000 7f901a7fc700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 0x7f90040a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f901c198190 tx=0x7f9010009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.017+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f900c0971e0 con 0x7f901c100f70 2023-10-06T21:09:38.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.232+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f901c02d2d0 con 0x7f90040a1510 2023-10-06T21:09:38.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.234+0000 7f9020874700 1 -- 172.21.15.26:0/4067338689 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f901c02d2d0 con 0x7f90040a1510 2023-10-06T21:09:38.237 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2023-10-06T21:09:38.237 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2023-10-06T21:09:38.237 INFO:teuthology.orchestra.run.smithi026.stdout:smithi078 172.21.15.78 2023-10-06T21:09:38.237 INFO:teuthology.orchestra.run.smithi026.stdout:smithi167 172.21.15.167 2023-10-06T21:09:38.237 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2023-10-06T21:09:38.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.236+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 msgr2=0x7f90040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:38.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.236+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 0x7f90040a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f901c198190 tx=0x7f9010009260 comp rx=0 tx=0).stop 2023-10-06T21:09:38.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.236+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 msgr2=0x7f901c196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.236+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f900c009fd0 tx=0x7f900c00f690 comp rx=0 tx=0).stop 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 shutdown_connections 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f90040a1510 0x7f90040a39d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f901c102b10 0x7f901c19e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f901c100f70 0x7f901c196c80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 --2- 172.21.15.26:0/4067338689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f901c102170 0x7f901c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 >> 172.21.15.26:0/4067338689 conn(0x7f901c0fc4d0 msgr2=0x7f901c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 shutdown_connections 2023-10-06T21:09:38.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:38.237+0000 7f9021876700 1 -- 172.21.15.26:0/4067338689 wait complete. 2023-10-06T21:09:38.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-06T21:09:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:38 smithi026 ceph-mon[98490]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:38 smithi026 ceph-mon[98490]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:38 smithi078 ceph-mon[134896]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:38 smithi078 ceph-mon[134896]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:38 smithi167 ceph-mon[134494]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:38 smithi167 ceph-mon[134494]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-10-06T21:09:39.527 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 -- 172.21.15.26:0/754040397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 msgr2=0x7fbf701093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 --2- 172.21.15.26:0/754040397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf701093c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b0d0 tx=0x7fbf6800b3e0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 -- 172.21.15.26:0/754040397 shutdown_connections 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 --2- 172.21.15.26:0/754040397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf701093c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 --2- 172.21.15.26:0/754040397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 --2- 172.21.15.26:0/754040397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 0x7fbf701025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 -- 172.21.15.26:0/754040397 >> 172.21.15.26:0/754040397 conn(0x7fbf700fc4d0 msgr2=0x7fbf700fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 -- 172.21.15.26:0/754040397 shutdown_connections 2023-10-06T21:09:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.038+0000 7fbf77773700 1 -- 172.21.15.26:0/754040397 wait complete. 2023-10-06T21:09:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.039+0000 7fbf77773700 1 Processor -- start 2023-10-06T21:09:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 -- start start 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 0x7fbf700732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf700737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70073f60 con 0x7fbf70102b10 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700740d0 con 0x7fbf70102170 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf77773700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701a0dc0 con 0x7fbf70100f70 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf7550f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf7550f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45640/0 (socket says 172.21.15.26:45640) 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf7550f700 1 -- 172.21.15.26:0/1218058191 learned_addr learned my addr 172.21.15.26:0/1218058191 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf75d10700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf700737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.040+0000 7fbf74d0e700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 0x7fbf700732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 -- 172.21.15.26:0/1218058191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 msgr2=0x7fbf700732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 0x7fbf700732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 -- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 msgr2=0x7fbf700737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf700737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 -- 172.21.15.26:0/1218058191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68009d00 con 0x7fbf70100f70 2023-10-06T21:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf7550f700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d980 tx=0x7fbf6c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00f890 con 0x7fbf70100f70 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf701a10a0 con 0x7fbf70100f70 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf6c010460 con 0x7fbf70100f70 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.041+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0195c0 con 0x7fbf70100f70 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.042+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf701a15f0 con 0x7fbf70100f70 2023-10-06T21:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.043+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf701084b0 con 0x7fbf70100f70 2023-10-06T21:09:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.045+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbf6c0105d0 con 0x7fbf70100f70 2023-10-06T21:09:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.046+0000 7fbf667fc700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 0x7fbf5c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.046+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbf6c0ccce0 con 0x7fbf70100f70 2023-10-06T21:09:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.046+0000 7fbf74d0e700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 0x7fbf5c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.047+0000 7fbf74d0e700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 0x7fbf5c0a3a50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009be0 tx=0x7fbf60009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.048+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf6c0961a0 con 0x7fbf70100f70 2023-10-06T21:09:41.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.268+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf7002d1a0 con 0x7fbf5c0a1590 2023-10-06T21:09:41.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.270+0000 7fbf667fc700 1 -- 172.21.15.26:0/1218058191 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fbf7002d1a0 con 0x7fbf5c0a1590 2023-10-06T21:09:41.272 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-06T21:09:41.272 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-06T21:09:41.273 INFO:teuthology.orchestra.run.smithi026.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-06T21:09:41.273 INFO:teuthology.orchestra.run.smithi026.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 msgr2=0x7fbf5c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 0x7fbf5c0a3a50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009be0 tx=0x7fbf60009260 comp rx=0 tx=0).stop 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 msgr2=0x7fbf70074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d980 tx=0x7fbf6c00dc90 comp rx=0 tx=0).stop 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 shutdown_connections 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fbf5c0a1590 0x7fbf5c0a3a50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf70102b10 0x7fbf700737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70100f70 0x7fbf70074c50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 --2- 172.21.15.26:0/1218058191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf70102170 0x7fbf700732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 >> 172.21.15.26:0/1218058191 conn(0x7fbf700fc4d0 msgr2=0x7fbf701055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 shutdown_connections 2023-10-06T21:09:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:41.272+0000 7fbf77773700 1 -- 172.21.15.26:0/1218058191 wait complete. 2023-10-06T21:09:41.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:41 smithi026 ceph-mon[98490]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:41.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:41 smithi167 ceph-mon[134494]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:41.804 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-06T21:09:41.813 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:09:41.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-06T21:09:41.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:41 smithi078 ceph-mon[134896]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-06T21:09:42.552 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:42.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:42 smithi167 ceph-mon[134494]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:42.822 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:42 smithi026 ceph-mon[98490]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:42.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:42 smithi078 ceph-mon[134896]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:09:43.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:43 smithi167 ceph-mon[134494]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:43.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:43 smithi026 ceph-mon[98490]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:43 smithi078 ceph-mon[134896]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:09:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 -- 172.21.15.26:0/3564913163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4102b10 msgr2=0x7f1ce41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 --2- 172.21.15.26:0/3564913163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4102b10 0x7f1ce41093c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b0d0 tx=0x7f1cdc00b3e0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 -- 172.21.15.26:0/3564913163 shutdown_connections 2023-10-06T21:09:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 --2- 172.21.15.26:0/3564913163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4102b10 0x7f1ce41093c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 --2- 172.21.15.26:0/3564913163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4100f70 0x7f1ce4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 --2- 172.21.15.26:0/3564913163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102170 0x7f1ce41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 -- 172.21.15.26:0/3564913163 >> 172.21.15.26:0/3564913163 conn(0x7f1ce40fc4d0 msgr2=0x7f1ce40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:44.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 -- 172.21.15.26:0/3564913163 shutdown_connections 2023-10-06T21:09:44.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.115+0000 7f1cea33f700 1 -- 172.21.15.26:0/3564913163 wait complete. 2023-10-06T21:09:44.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.116+0000 7f1cea33f700 1 Processor -- start 2023-10-06T21:09:44.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 -- start start 2023-10-06T21:09:44.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4102170 0x7f1ce4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 0x7f1ce41a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a0a80 con 0x7f1ce4100f70 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a0bc0 con 0x7f1ce4102170 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1cea33f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41a0d50 con 0x7f1ce4102b10 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1ce3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55326/0 (socket says 172.21.15.26:55326) 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1ce3fff700 1 -- 172.21.15.26:0/1844933354 learned_addr learned my addr 172.21.15.26:0/1844933354 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1ce37fe700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4102170 0x7f1ce4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.117+0000 7f1ce88dc700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 0x7f1ce41a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 -- 172.21.15.26:0/1844933354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 msgr2=0x7f1ce41a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 0x7f1ce41a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 -- 172.21.15.26:0/1844933354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4102170 msgr2=0x7f1ce4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4102170 0x7f1ce4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 -- 172.21.15.26:0/1844933354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc009d00 con 0x7f1ce4100f70 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce88dc700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 0x7f1ce41a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce3fff700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9a0 tx=0x7f1cd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4009790 con 0x7f1ce4100f70 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd40044e0 con 0x7f1ce4100f70 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4003d20 con 0x7f1ce4100f70 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce41a0fd0 con 0x7f1ce4100f70 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.118+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce41a1390 con 0x7f1ce4100f70 2023-10-06T21:09:44.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.119+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce410c7f0 con 0x7f1ce4100f70 2023-10-06T21:09:44.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.121+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1cd4003e80 con 0x7f1ce4100f70 2023-10-06T21:09:44.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.123+0000 7f1ce17fa700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 0x7f1ccc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:44.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.123+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1cd4014070 con 0x7f1ce4100f70 2023-10-06T21:09:44.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.123+0000 7f1ce37fe700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 0x7f1ccc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:44.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.124+0000 7f1ce37fe700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 0x7f1ccc0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009c60 tx=0x7f1cd8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.127+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1cd4096600 con 0x7f1ce4100f70 2023-10-06T21:09:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.399+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ce41a1b70 con 0x7f1ce4100f70 2023-10-06T21:09:44.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.410+0000 7f1ce17fa700 1 -- 172.21.15.26:0/1844933354 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2935 (secure 0 0 0) 0x7f1cd4099c20 con 0x7f1ce4100f70 2023-10-06T21:09:44.413 INFO:teuthology.orchestra.run.smithi026.stdout:e23 2023-10-06T21:09:44.413 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-06T21:09:44.413 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:09:44.413 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2023-10-06T21:09:44.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 23 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:created 2023-10-06T21:08:16.065653+0000 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2023-10-06T21:08:21.849944+0000 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2023-10-06T21:09:44.414 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 3 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24587,1=24559,2=24539} 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2023-10-06T21:09:44.415 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{0:24587} state up:active seq 3 addr [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{0:24535} state up:standby-replay seq 1 addr [v2:172.21.15.167:6838/1710830195,v1:172.21.15.167:6839/1710830195] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{1:24559} state up:active seq 5 addr [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{1:24514} state up:standby-replay seq 1 addr [v2:172.21.15.167:6836/2946984451,v1:172.21.15.167:6837/2946984451] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{2:24539} state up:active seq 7 addr [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.416 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{2:24506} state up:standby-replay seq 1 addr [v2:172.21.15.78:6834/4281425672,v1:172.21.15.78:6835/4281425672] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:14448} state up:standby seq 1 addr [v2:172.21.15.26:6834/4105384407,v1:172.21.15.26:6835/4105384407] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:14460} state up:standby seq 1 addr [v2:172.21.15.26:6836/2894864282,v1:172.21.15.26:6837/2894864282] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{-1:14511} state up:standby seq 1 addr [v2:172.21.15.26:6838/4229413867,v1:172.21.15.26:6839/4229413867] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.167:6832/4048987,v1:172.21.15.167:6833/4048987] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{-1:24470} state up:standby seq 1 addr [v2:172.21.15.78:6832/3813856521,v1:172.21.15.78:6833/3813856521] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.417 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{-1:24502} state up:standby seq 1 addr [v2:172.21.15.167:6834/3142312308,v1:172.21.15.167:6835/3142312308] compat {c=[1],r=[1],i=[fff]}] 2023-10-06T21:09:44.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.424+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 msgr2=0x7f1ccc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:44.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.424+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 0x7f1ccc0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009c60 tx=0x7f1cd8009260 comp rx=0 tx=0).stop 2023-10-06T21:09:44.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.424+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 msgr2=0x7f1ce4198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:44.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.424+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9a0 tx=0x7f1cd400ecb0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 shutdown_connections 2023-10-06T21:09:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce4102b10 0x7f1ce41a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1ccc0a1510 0x7f1ccc0a39d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce4100f70 0x7f1ce4198ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 --2- 172.21.15.26:0/1844933354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce4102170 0x7f1ce4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 >> 172.21.15.26:0/1844933354 conn(0x7f1ce40fc4d0 msgr2=0x7f1ce41079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 shutdown_connections 2023-10-06T21:09:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:44.427+0000 7f1cea33f700 1 -- 172.21.15.26:0/1844933354 wait complete. 2023-10-06T21:09:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:44 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1844933354' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:09:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:44 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1844933354' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:09:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:44 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1844933354' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-06T21:09:44.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-06T21:09:45.684 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:45.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:45 smithi167 ceph-mon[134494]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:45.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:45 smithi026 ceph-mon[98490]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:45.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:45 smithi078 ceph-mon[134896]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-06T21:09:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:46 smithi026 ceph-mon[98490]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T21:09:46.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:46 smithi167 ceph-mon[134494]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T21:09:46.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:46 smithi078 ceph-mon[134896]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.385+0000 7fdf45031700 1 -- 172.21.15.26:0/2705886597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40100f70 msgr2=0x7fdf40101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.385+0000 7fdf45031700 1 --2- 172.21.15.26:0/2705886597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40100f70 0x7fdf40101390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdf300099c0 tx=0x7fdf30009cd0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 -- 172.21.15.26:0/2705886597 shutdown_connections 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 --2- 172.21.15.26:0/2705886597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 0x7fdf401093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 --2- 172.21.15.26:0/2705886597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40100f70 0x7fdf40101390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 --2- 172.21.15.26:0/2705886597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40102170 0x7fdf401025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 -- 172.21.15.26:0/2705886597 >> 172.21.15.26:0/2705886597 conn(0x7fdf400fc4d0 msgr2=0x7fdf400fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 -- 172.21.15.26:0/2705886597 shutdown_connections 2023-10-06T21:09:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.386+0000 7fdf45031700 1 -- 172.21.15.26:0/2705886597 wait complete. 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 Processor -- start 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 -- start start 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40100f70 0x7fdf40196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 0x7fdf4019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4019e800 con 0x7fdf40102170 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4019e940 con 0x7fdf40102b10 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.387+0000 7fdf45031700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4019eb00 con 0x7fdf40100f70 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55346/0 (socket says 172.21.15.26:55346) 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 -- 172.21.15.26:0/844720879 learned_addr learned my addr 172.21.15.26:0/844720879 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3ed9d700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40100f70 0x7fdf40196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3f59e700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 0x7fdf4019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 -- 172.21.15.26:0/844720879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40100f70 msgr2=0x7fdf40196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40100f70 0x7fdf40196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 -- 172.21.15.26:0/844720879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 msgr2=0x7fdf4019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 0x7fdf4019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 -- 172.21.15.26:0/844720879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300095f0 con 0x7fdf40102170 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.388+0000 7fdf3e59c700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400d980 tx=0x7fdf3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.389+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34004220 con 0x7fdf40102170 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.389+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf34007640 con 0x7fdf40102170 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.389+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4019ede0 con 0x7fdf40102170 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.389+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34005690 con 0x7fdf40102170 2023-10-06T21:09:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.389+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4019f250 con 0x7fdf40102170 2023-10-06T21:09:47.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.390+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf40108560 con 0x7fdf40102170 2023-10-06T21:09:47.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.392+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdf34004750 con 0x7fdf40102170 2023-10-06T21:09:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.393+0000 7fdf27fff700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 0x7fdf280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.393+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf340ccb60 con 0x7fdf40102170 2023-10-06T21:09:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.393+0000 7fdf3ed9d700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 0x7fdf280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:47.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.394+0000 7fdf3ed9d700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 0x7fdf280a3940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009f50 tx=0x7fdf30005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:47.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.397+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf34096180 con 0x7fdf40102170 2023-10-06T21:09:47.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.607+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fdf4010c7c0 con 0x7fdf40102170 2023-10-06T21:09:47.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.608+0000 7fdf27fff700 1 -- 172.21.15.26:0/844720879 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5289 (secure 0 0 0) 0x7fdf34016070 con 0x7fdf40102170 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 87 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:created 2023-10-06T21:00:45.549995+0000 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2023-10-06T21:08:45.432140+0000 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 26 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2023-10-06T21:09:47.610 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release reef 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/3569177052,v1:172.21.15.26:6803/3569177052] [v2:172.21.15.26:6804/3569177052,v1:172.21.15.26:6805/3569177052] exists,up 9e106834-15f2-43e2-86a4-4256e7169c93 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/4119914658,v1:172.21.15.78:6801/4119914658] [v2:172.21.15.78:6802/4119914658,v1:172.21.15.78:6803/4119914658] exists,up cc8d8aba-6316-4996-bb52-6f32af502c53 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/2625559471,v1:172.21.15.167:6801/2625559471] [v2:172.21.15.167:6802/2625559471,v1:172.21.15.167:6803/2625559471] exists,up 30a5233e-8442-48f1-af3e-3a0752ff274e 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/1210213318,v1:172.21.15.26:6811/1210213318] [v2:172.21.15.26:6812/1210213318,v1:172.21.15.26:6813/1210213318] exists,up 50d5d2ee-19a4-407b-bec4-92dd3e7c684a 2023-10-06T21:09:47.611 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/3086176217,v1:172.21.15.78:6809/3086176217] [v2:172.21.15.78:6810/3086176217,v1:172.21.15.78:6811/3086176217] exists,up 22b765a7-0ed8-452b-a897-6b5a7c720e59 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/644360584,v1:172.21.15.167:6809/644360584] [v2:172.21.15.167:6810/644360584,v1:172.21.15.167:6811/644360584] exists,up 4c5d6f6e-9881-41e0-859d-2e8702b70d22 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/1485072774,v1:172.21.15.26:6819/1485072774] [v2:172.21.15.26:6820/1485072774,v1:172.21.15.26:6821/1485072774] exists,up cba40219-6497-43c6-a864-79afb928c52a 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/2278627403,v1:172.21.15.78:6817/2278627403] [v2:172.21.15.78:6818/2278627403,v1:172.21.15.78:6819/2278627403] exists,up 28a181f7-1d70-4d23-a95c-3fd0309f720d 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/263711994,v1:172.21.15.167:6817/263711994] [v2:172.21.15.167:6818/263711994,v1:172.21.15.167:6819/263711994] exists,up cd2b8aa3-49f9-4263-825a-bd720b77e281 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/157001209,v1:172.21.15.26:6827/157001209] [v2:172.21.15.26:6828/157001209,v1:172.21.15.26:6829/157001209] exists,up fc3a0395-6144-480c-aaf7-d2f8f8f22f3d 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/1966991917,v1:172.21.15.78:6825/1966991917] [v2:172.21.15.78:6826/1966991917,v1:172.21.15.78:6827/1966991917] exists,up 2bda1e71-4772-49f7-8d68-cc56c7f174ba 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/4274564614,v1:172.21.15.167:6825/4274564614] [v2:172.21.15.167:6826/4274564614,v1:172.21.15.167:6827/4274564614] exists,up d023d04f-16f7-4e4c-8fef-e2ac8e7db619 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.5 [8,11] 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.a [2,11] 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.14 [2,4] 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.0 [4,8] 2023-10-06T21:09:47.612 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.6 [3,6] 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.10 [5,9] 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.23 [4,7] 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.37 [1,9] 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.3c [4,9] 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1380403953 expires 2023-10-07T21:01:44.284577+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/10926064 expires 2023-10-07T21:01:44.284577+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/118366491 expires 2023-10-07T21:01:44.284577+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/4033583135 expires 2023-10-07T21:01:17.251649+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2078185111 expires 2023-10-07T21:01:17.251649+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/3098538275 expires 2023-10-07T21:01:17.251649+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1222842153 expires 2023-10-07T21:01:17.251649+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2655877248 expires 2023-10-07T21:01:44.284577+0000 2023-10-06T21:09:47.613 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/3098538275 expires 2023-10-07T21:01:17.251649+0000 2023-10-06T21:09:47.614 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/118366491 expires 2023-10-07T21:01:44.284577+0000 2023-10-06T21:09:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 msgr2=0x7fdf280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 0x7fdf280a3940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009f50 tx=0x7fdf30005bc0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 msgr2=0x7fdf401971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400d980 tx=0x7fdf3400dc90 comp rx=0 tx=0).stop 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 shutdown_connections 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf40102b10 0x7fdf4019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf40100f70 0x7fdf40196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdf280a1480 0x7fdf280a3940 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 --2- 172.21.15.26:0/844720879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf40102170 0x7fdf401971c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 >> 172.21.15.26:0/844720879 conn(0x7fdf400fc4d0 msgr2=0x7fdf401056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 shutdown_connections 2023-10-06T21:09:47.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:47.610+0000 7fdf45031700 1 -- 172.21.15.26:0/844720879 wait complete. 2023-10-06T21:09:47.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:47 smithi167 ceph-mon[134494]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:09:47.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:47 smithi026 ceph-mon[98490]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:09:47.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:47 smithi078 ceph-mon[134896]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:09:48.164 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-06T21:09:48.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 -- 172.21.15.26:0/580219050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8073910 msgr2=0x7f76c8073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/580219050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8073910 0x7f76c8073d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009990 tx=0x7f76bc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 -- 172.21.15.26:0/580219050 shutdown_connections 2023-10-06T21:09:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/580219050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 0x7f76c810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/580219050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8074ef0 0x7f76c8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/580219050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8073910 0x7f76c8073d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 -- 172.21.15.26:0/580219050 >> 172.21.15.26:0/580219050 conn(0x7f76c80fc420 msgr2=0x7f76c80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:48.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 -- 172.21.15.26:0/580219050 shutdown_connections 2023-10-06T21:09:48.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.380+0000 7f76cd1ff700 1 -- 172.21.15.26:0/580219050 wait complete. 2023-10-06T21:09:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.381+0000 7f76cd1ff700 1 Processor -- start 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.381+0000 7f76cd1ff700 1 -- start start 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8073910 0x7f76c8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 0x7f76c81a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a0960 con 0x7f76c8074ef0 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a0aa0 con 0x7f76c8073910 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76cd1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a0c30 con 0x7f76c8108db0 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 0x7f76c81a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55360/0 (socket says 172.21.15.26:55360) 2023-10-06T21:09:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 -- 172.21.15.26:0/3571035252 learned_addr learned my addr 172.21.15.26:0/3571035252 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c6d9d700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8073910 0x7f76c8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 -- 172.21.15.26:0/3571035252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 msgr2=0x7f76c81a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 0x7f76c81a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 -- 172.21.15.26:0/3571035252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8073910 msgr2=0x7f76c8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8073910 0x7f76c8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 -- 172.21.15.26:0/3571035252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0095f0 con 0x7f76c8074ef0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.382+0000 7f76c659c700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098e0 tx=0x7f76bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.383+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc01d070 con 0x7f76c8074ef0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.383+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76bc0054f0 con 0x7f76c8074ef0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.383+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c81a0eb0 con 0x7f76c8074ef0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.383+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004be0 con 0x7f76c8074ef0 2023-10-06T21:09:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.383+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c81a1220 con 0x7f76c8074ef0 2023-10-06T21:09:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.384+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c8107460 con 0x7f76c8074ef0 2023-10-06T21:09:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.386+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f76bc00fbc0 con 0x7f76c8074ef0 2023-10-06T21:09:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.387+0000 7f76affff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 0x7f76b00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.387+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76bc0ce3b0 con 0x7f76c8074ef0 2023-10-06T21:09:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.388+0000 7f76c6d9d700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 0x7f76b00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.388+0000 7f76c6d9d700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 0x7f76b00a3990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005950 tx=0x7f76b800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:48.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.390+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f76bc097a60 con 0x7f76c8074ef0 2023-10-06T21:09:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.604+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76c810e930 con 0x7f76c8074ef0 2023-10-06T21:09:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.606+0000 7f76affff700 1 -- 172.21.15.26:0/3571035252 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f76bc09b080 con 0x7f76c8074ef0 2023-10-06T21:09:48.608 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:48.608 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 msgr2=0x7f76b00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 0x7f76b00a3990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76b8005950 tx=0x7f76b800b410 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 msgr2=0x7f76c8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098e0 tx=0x7f76bc004730 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 shutdown_connections 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76c8108db0 0x7f76c81a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76c8073910 0x7f76c8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76b00a14d0 0x7f76b00a3990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 --2- 172.21.15.26:0/3571035252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76c8074ef0 0x7f76c8199320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 >> 172.21.15.26:0/3571035252 conn(0x7f76c80fc420 msgr2=0x7f76c80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:48.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 shutdown_connections 2023-10-06T21:09:48.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.609+0000 7f76cd1ff700 1 -- 172.21.15.26:0/3571035252 wait complete. 2023-10-06T21:09:48.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:09:48.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:09:48.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:48 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/844720879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:09:48.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:48 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/844720879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:09:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/1067366278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8102a30 msgr2=0x7fc5a8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/1067366278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8102a30 0x7fc5a8109350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b0a0 tx=0x7fc5a000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/1067366278 shutdown_connections 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/1067366278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8102a30 0x7fc5a8109350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/1067366278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8100e90 0x7fc5a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/1067366278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 0x7fc5a81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/1067366278 >> 172.21.15.26:0/1067366278 conn(0x7fc5a80fc430 msgr2=0x7fc5a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/1067366278 shutdown_connections 2023-10-06T21:09:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.836+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/1067366278 wait complete. 2023-10-06T21:09:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.837+0000 7fc5ad3ff700 1 Processor -- start 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.837+0000 7fc5ad3ff700 1 -- start start 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 0x7fc5a81971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8102a30 0x7fc5a819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819e820 con 0x7fc5a8100e90 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819e990 con 0x7fc5a8102a30 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5ad3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a819eb50 con 0x7fc5a8102070 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55374/0 (socket says 172.21.15.26:55374) 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 -- 172.21.15.26:0/4031026012 learned_addr learned my addr 172.21.15.26:0/4031026012 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5a6ffd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 0x7fc5a81971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc5a77fe700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8102a30 0x7fc5a819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 -- 172.21.15.26:0/4031026012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 msgr2=0x7fc5a81971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 0x7fc5a81971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 -- 172.21.15.26:0/4031026012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8102a30 msgr2=0x7fc5a819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8102a30 0x7fc5a819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.838+0000 7fc59effd700 1 -- 172.21.15.26:0/4031026012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009d00 con 0x7fc5a8100e90 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc59effd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e970 tx=0x7fc59800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598009790 con 0x7fc5a8100e90 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc59800c7e0 con 0x7fc5a8100e90 2023-10-06T21:09:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a819ee30 con 0x7fc5a8100e90 2023-10-06T21:09:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598003c70 con 0x7fc5a8100e90 2023-10-06T21:09:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.839+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a819f2b0 con 0x7fc5a8100e90 2023-10-06T21:09:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.840+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a81084a0 con 0x7fc5a8100e90 2023-10-06T21:09:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.842+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc59800cc50 con 0x7fc5a8100e90 2023-10-06T21:09:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.843+0000 7fc5a4ff9700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 0x7fc5940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.844+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc598014070 con 0x7fc5a8100e90 2023-10-06T21:09:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.844+0000 7fc5a6ffd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 0x7fc5940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.844+0000 7fc5a6ffd700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 0x7fc5940a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5a81981b0 tx=0x7fc590009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:48.847+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc598096490 con 0x7fc5a8100e90 2023-10-06T21:09:48.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:48 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/844720879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-06T21:09:49.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.058+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5a802cfa0 con 0x7fc5a8100e90 2023-10-06T21:09:49.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.059+0000 7fc5a4ff9700 1 -- 172.21.15.26:0/4031026012 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc598003dd0 con 0x7fc5a8100e90 2023-10-06T21:09:49.061 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 msgr2=0x7fc5940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 0x7fc5940a39c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5a81981b0 tx=0x7fc590009260 comp rx=0 tx=0).stop 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 msgr2=0x7fc5a8196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e970 tx=0x7fc59800ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 shutdown_connections 2023-10-06T21:09:49.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5a8102a30 0x7fc5a819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc5940a1500 0x7fc5940a39c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5a8100e90 0x7fc5a8196cb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 --2- 172.21.15.26:0/4031026012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5a8102070 0x7fc5a81971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 >> 172.21.15.26:0/4031026012 conn(0x7fc5a80fc430 msgr2=0x7fc5a81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.061+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 shutdown_connections 2023-10-06T21:09:49.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.062+0000 7fc5ad3ff700 1 -- 172.21.15.26:0/4031026012 wait complete. 2023-10-06T21:09:49.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 -- 172.21.15.26:0/2373668792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 msgr2=0x7f23bc103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2373668792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc103d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009990 tx=0x7f23ac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 -- 172.21.15.26:0/2373668792 shutdown_connections 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2373668792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 0x7f23bc109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2373668792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc103d90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2373668792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 0x7f23bc1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 -- 172.21.15.26:0/2373668792 >> 172.21.15.26:0/2373668792 conn(0x7f23bc0fb330 msgr2=0x7f23bc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 -- 172.21.15.26:0/2373668792 shutdown_connections 2023-10-06T21:09:49.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.287+0000 7f23c19ff700 1 -- 172.21.15.26:0/2373668792 wait complete. 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.288+0000 7f23c19ff700 1 Processor -- start 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.288+0000 7f23c19ff700 1 -- start start 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.288+0000 7f23c19ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.288+0000 7f23c19ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 0x7f23bc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23c19ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 0x7f23bc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23c19ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a09a0 con 0x7f23bc1019a0 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23c19ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0ae0 con 0x7f23bc106c00 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23c19ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0c70 con 0x7f23bc1042d0 2023-10-06T21:09:49.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55390/0 (socket says 172.21.15.26:55390) 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 -- 172.21.15.26:0/2306865710 learned_addr learned my addr 172.21.15.26:0/2306865710 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23bb7fe700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 0x7f23bc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23ba7fc700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 0x7f23bc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 -- 172.21.15.26:0/2306865710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 msgr2=0x7f23bc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 0x7f23bc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 -- 172.21.15.26:0/2306865710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 msgr2=0x7f23bc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 0x7f23bc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 -- 172.21.15.26:0/2306865710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0095f0 con 0x7f23bc1019a0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.289+0000 7f23baffd700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004730 tx=0x7f23ac004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.290+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01d070 con 0x7f23bc1019a0 2023-10-06T21:09:49.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.290+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23ac004c90 con 0x7f23bc1019a0 2023-10-06T21:09:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.290+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc1a0ef0 con 0x7f23bc1019a0 2023-10-06T21:09:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.290+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0054b0 con 0x7f23bc1019a0 2023-10-06T21:09:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.290+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc1a1260 con 0x7f23bc1019a0 2023-10-06T21:09:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.291+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc108780 con 0x7f23bc1019a0 2023-10-06T21:09:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.293+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f23ac00fbc0 con 0x7f23bc1019a0 2023-10-06T21:09:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.295+0000 7f23c09fd700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 0x7f23a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.295+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23ac0cdf80 con 0x7f23bc1019a0 2023-10-06T21:09:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.295+0000 7f23ba7fc700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 0x7f23a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.296+0000 7f23ba7fc700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 0x7f23a40a3940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009c60 tx=0x7f23b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:49.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.299+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f23ac05b100 con 0x7f23bc1019a0 2023-10-06T21:09:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.513+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23bc10c6f0 con 0x7f23bc1019a0 2023-10-06T21:09:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.514+0000 7f23c09fd700 1 -- 172.21.15.26:0/2306865710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23ac097630 con 0x7f23bc1019a0 2023-10-06T21:09:49.519 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:09:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.516+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 msgr2=0x7f23a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.516+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 0x7f23a40a3940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009c60 tx=0x7f23b0009260 comp rx=0 tx=0).stop 2023-10-06T21:09:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.516+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 msgr2=0x7f23bc198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.516+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004730 tx=0x7f23ac004760 comp rx=0 tx=0).stop 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 shutdown_connections 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23bc106c00 0x7f23bc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f23a40a1480 0x7f23a40a3940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23bc1019a0 0x7f23bc198e20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 --2- 172.21.15.26:0/2306865710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23bc1042d0 0x7f23bc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 >> 172.21.15.26:0/2306865710 conn(0x7f23bc0fb330 msgr2=0x7f23bc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 shutdown_connections 2023-10-06T21:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.517+0000 7f23c19ff700 1 -- 172.21.15.26:0/2306865710 wait complete. 2023-10-06T21:09:49.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:09:49.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.741+0000 7f39a31ff700 1 -- 172.21.15.26:0/3379523566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c073910 msgr2=0x7f399c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.741+0000 7f39a31ff700 1 --2- 172.21.15.26:0/3379523566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c073910 0x7f399c073d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3988009990 tx=0x7f3988009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 -- 172.21.15.26:0/3379523566 shutdown_connections 2023-10-06T21:09:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 --2- 172.21.15.26:0/3379523566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c108da0 0x7f399c10b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 --2- 172.21.15.26:0/3379523566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c074ef0 0x7f399c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 --2- 172.21.15.26:0/3379523566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c073910 0x7f399c073d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 -- 172.21.15.26:0/3379523566 >> 172.21.15.26:0/3379523566 conn(0x7f399c0fc430 msgr2=0x7f399c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 -- 172.21.15.26:0/3379523566 shutdown_connections 2023-10-06T21:09:49.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.742+0000 7f39a31ff700 1 -- 172.21.15.26:0/3379523566 wait complete. 2023-10-06T21:09:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 Processor -- start 2023-10-06T21:09:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 -- start start 2023-10-06T21:09:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c073910 0x7f399c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c108da0 0x7f399c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1a09c0 con 0x7f399c074ef0 2023-10-06T21:09:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1a0b00 con 0x7f399c108da0 2023-10-06T21:09:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f39a31ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c1a0c90 con 0x7f399c073910 2023-10-06T21:09:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f399bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f399bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55404/0 (socket says 172.21.15.26:55404) 2023-10-06T21:09:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.743+0000 7f399bfff700 1 -- 172.21.15.26:0/1902399448 learned_addr learned my addr 172.21.15.26:0/1902399448 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f39a0f9b700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c073910 0x7f399c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f39a179c700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c108da0 0x7f399c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 -- 172.21.15.26:0/1902399448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c073910 msgr2=0x7f399c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c073910 0x7f399c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 -- 172.21.15.26:0/1902399448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c108da0 msgr2=0x7f399c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c108da0 0x7f399c1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 -- 172.21.15.26:0/1902399448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009520 con 0x7f399c074ef0 2023-10-06T21:09:49.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f399bfff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3988005720 tx=0x7f3988009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:49.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398801d070 con 0x7f399c074ef0 2023-10-06T21:09:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3988021410 con 0x7f399c074ef0 2023-10-06T21:09:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880095f0 con 0x7f399c074ef0 2023-10-06T21:09:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.744+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988017650 con 0x7f399c074ef0 2023-10-06T21:09:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.745+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c1a11a0 con 0x7f399c074ef0 2023-10-06T21:09:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.746+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c107450 con 0x7f399c074ef0 2023-10-06T21:09:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.747+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3988021580 con 0x7f399c074ef0 2023-10-06T21:09:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.750+0000 7f3999ffb700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 0x7f398c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.750+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39880cdee0 con 0x7f399c074ef0 2023-10-06T21:09:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.750+0000 7f39a0f9b700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 0x7f398c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.751+0000 7f39a0f9b700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 0x7f398c0a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3990009df0 tx=0x7f3990009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.754+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3988097590 con 0x7f399c074ef0 2023-10-06T21:09:49.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:49 smithi167 ceph-mon[134494]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 596 B/s wr, 7 op/s 2023-10-06T21:09:49.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:49 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3571035252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:49.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:49 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4031026012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:49.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:49 smithi026 ceph-mon[98490]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 596 B/s wr, 7 op/s 2023-10-06T21:09:49.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:49 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3571035252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:49.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:49 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4031026012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:49.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:49 smithi078 ceph-mon[134896]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 596 B/s wr, 7 op/s 2023-10-06T21:09:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:49 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3571035252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:49 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4031026012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.967+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f399c02d020 con 0x7f399c074ef0 2023-10-06T21:09:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.968+0000 7f3999ffb700 1 -- 172.21.15.26:0/1902399448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3988026020 con 0x7f399c074ef0 2023-10-06T21:09:49.970 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:09:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 msgr2=0x7f398c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 0x7f398c0a3940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3990009df0 tx=0x7f3990009260 comp rx=0 tx=0).stop 2023-10-06T21:09:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 msgr2=0x7f399c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3988005720 tx=0x7f3988009580 comp rx=0 tx=0).stop 2023-10-06T21:09:49.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 shutdown_connections 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f399c108da0 0x7f399c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f399c073910 0x7f399c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f398c0a1480 0x7f398c0a3940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 --2- 172.21.15.26:0/1902399448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f399c074ef0 0x7f399c199380 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 >> 172.21.15.26:0/1902399448 conn(0x7f399c0fc430 msgr2=0x7f399c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 shutdown_connections 2023-10-06T21:09:49.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:49.970+0000 7f39a31ff700 1 -- 172.21.15.26:0/1902399448 wait complete. 2023-10-06T21:09:49.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:09:50.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 -- 172.21.15.26:0/1226846609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4100ea0 msgr2=0x7faeb41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 --2- 172.21.15.26:0/1226846609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4100ea0 0x7faeb41012c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009990 tx=0x7faeb0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 -- 172.21.15.26:0/1226846609 shutdown_connections 2023-10-06T21:09:50.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 --2- 172.21.15.26:0/1226846609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4102a40 0x7faeb4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 --2- 172.21.15.26:0/1226846609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4100ea0 0x7faeb41012c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 --2- 172.21.15.26:0/1226846609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 -- 172.21.15.26:0/1226846609 >> 172.21.15.26:0/1226846609 conn(0x7faeb40fc420 msgr2=0x7faeb40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 -- 172.21.15.26:0/1226846609 shutdown_connections 2023-10-06T21:09:50.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.192+0000 7faebb9ff700 1 -- 172.21.15.26:0/1226846609 wait complete. 2023-10-06T21:09:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 Processor -- start 2023-10-06T21:09:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 -- start start 2023-10-06T21:09:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4100ea0 0x7faeb4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419e770 con 0x7faeb4102a40 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419e8e0 con 0x7faeb4102080 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.194+0000 7faebb9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419eaa0 con 0x7faeb4100ea0 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55430/0 (socket says 172.21.15.26:55430) 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb9f9c700 1 -- 172.21.15.26:0/3267876619 learned_addr learned my addr 172.21.15.26:0/3267876619 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48664/0 (socket says 172.21.15.26:48664) 2023-10-06T21:09:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb979b700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4100ea0 0x7faeb4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 -- 172.21.15.26:0/3267876619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4100ea0 msgr2=0x7faeb4196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4100ea0 0x7faeb4196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 -- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 msgr2=0x7faeb419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 -- 172.21.15.26:0/3267876619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb00095f0 con 0x7faeb4102080 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb9f9c700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.195+0000 7faeb8f9a700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faea400d740 tx=0x7faea400da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.196+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4004220 con 0x7faeb4102080 2023-10-06T21:09:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.196+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea4004380 con 0x7faeb4102080 2023-10-06T21:09:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.196+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb419ed80 con 0x7faeb4102080 2023-10-06T21:09:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.196+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4003cf0 con 0x7faeb4102080 2023-10-06T21:09:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.196+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb419f1d0 con 0x7faeb4102080 2023-10-06T21:09:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.197+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb41084b0 con 0x7faeb4102080 2023-10-06T21:09:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.199+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faea4045790 con 0x7faeb4102080 2023-10-06T21:09:50.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.200+0000 7faeaa7fc700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 0x7faea00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.201+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faea40ccb00 con 0x7faeb4102080 2023-10-06T21:09:50.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.201+0000 7faeb979b700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 0x7faea00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.201+0000 7faeb979b700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 0x7faea00a3990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faeb4198100 tx=0x7faeb0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.202+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faea4096120 con 0x7faeb4102080 2023-10-06T21:09:50.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.462+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeb410c9b0 con 0x7faeb4102080 2023-10-06T21:09:50.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.463+0000 7faeaa7fc700 1 -- 172.21.15.26:0/3267876619 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7faea401f090 con 0x7faeb4102080 2023-10-06T21:09:50.466 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:50.466 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:09:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 msgr2=0x7faea00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 0x7faea00a3990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faeb4198100 tx=0x7faeb0005fd0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 msgr2=0x7faeb4197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faea400d740 tx=0x7faea400da50 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 shutdown_connections 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faea00a14d0 0x7faea00a3990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faeb4102a40 0x7faeb419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faeb4100ea0 0x7faeb4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 --2- 172.21.15.26:0/3267876619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faeb4102080 0x7faeb4197170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 >> 172.21.15.26:0/3267876619 conn(0x7faeb40fc420 msgr2=0x7faeb41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 shutdown_connections 2023-10-06T21:09:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.466+0000 7faebb9ff700 1 -- 172.21.15.26:0/3267876619 wait complete. 2023-10-06T21:09:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:50.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 -- 172.21.15.26:0/2052717115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 msgr2=0x7fe900102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 --2- 172.21.15.26:0/2052717115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900102500 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009990 tx=0x7fe8f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 -- 172.21.15.26:0/2052717115 shutdown_connections 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 --2- 172.21.15.26:0/2052717115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe900109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 --2- 172.21.15.26:0/2052717115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 0x7fe9001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 --2- 172.21.15.26:0/2052717115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900102500 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 -- 172.21.15.26:0/2052717115 >> 172.21.15.26:0/2052717115 conn(0x7fe9000fc420 msgr2=0x7fe9000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 -- 172.21.15.26:0/2052717115 shutdown_connections 2023-10-06T21:09:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.689+0000 7fe9077ff700 1 -- 172.21.15.26:0/2052717115 wait complete. 2023-10-06T21:09:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 Processor -- start 2023-10-06T21:09:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 -- start start 2023-10-06T21:09:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 0x7fe900198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe9001a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001a09a0 con 0x7fe900102080 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001a0ae0 con 0x7fe900100ea0 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe9077ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001a0c70 con 0x7fe900102a40 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe904d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe904d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55450/0 (socket says 172.21.15.26:55450) 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.690+0000 7fe904d9a700 1 -- 172.21.15.26:0/3314906086 learned_addr learned my addr 172.21.15.26:0/3314906086 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe90559b700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 0x7fe900198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe905d9c700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe9001a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 -- 172.21.15.26:0/3314906086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 msgr2=0x7fe9001a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe9001a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 -- 172.21.15.26:0/3314906086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 msgr2=0x7fe900198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 0x7fe900198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 -- 172.21.15.26:0/3314906086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f00095f0 con 0x7fe900102080 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe905d9c700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe9001a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe904d9a700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098e0 tx=0x7fe8f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f001d070 con 0x7fe900102080 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8f00054f0 con 0x7fe900102080 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9001a0ef0 con 0x7fe900102080 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.691+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004ca0 con 0x7fe900102080 2023-10-06T21:09:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.692+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9001a1260 con 0x7fe900102080 2023-10-06T21:09:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.693+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90010c7d0 con 0x7fe900102080 2023-10-06T21:09:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.694+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe8f000fc70 con 0x7fe900102080 2023-10-06T21:09:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.696+0000 7fe8f67fc700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 0x7fe8ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.696+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe8f00ce080 con 0x7fe900102080 2023-10-06T21:09:50.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.696+0000 7fe90559b700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 0x7fe8ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:50.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.697+0000 7fe90559b700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 0x7fe8ec0a3990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe900101bf0 tx=0x7fe8fc007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.700+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe8f005b100 con 0x7fe900102080 2023-10-06T21:09:50.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:50 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2306865710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:50 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1902399448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:50 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3267876619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2306865710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1902399448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:50 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3267876619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:50.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:50 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2306865710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:50 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1902399448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:50.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:50 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3267876619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.912+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9001a1450 con 0x7fe900102080 2023-10-06T21:09:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.913+0000 7fe8f67fc700 1 -- 172.21.15.26:0/3314906086 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7fe8f0097730 con 0x7fe900102080 2023-10-06T21:09:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:50.916 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 msgr2=0x7fe8ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 0x7fe8ec0a3990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe900101bf0 tx=0x7fe8fc007400 comp rx=0 tx=0).stop 2023-10-06T21:09:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 msgr2=0x7fe900199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00098e0 tx=0x7fe8f0004730 comp rx=0 tx=0).stop 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 shutdown_connections 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe900102a40 0x7fe9001a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe900100ea0 0x7fe900198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fe8ec0a14d0 0x7fe8ec0a3990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 --2- 172.21.15.26:0/3314906086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe900102080 0x7fe900199360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 >> 172.21.15.26:0/3314906086 conn(0x7fe9000fc420 msgr2=0x7fe9000fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 shutdown_connections 2023-10-06T21:09:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:50.916+0000 7fe9077ff700 1 -- 172.21.15.26:0/3314906086 wait complete. 2023-10-06T21:09:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:09:50.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:09:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 -- 172.21.15.26:0/4110341908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c102070 msgr2=0x7f315c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 --2- 172.21.15.26:0/4110341908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c102070 0x7f315c1024f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3150009990 tx=0x7f3150009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 -- 172.21.15.26:0/4110341908 shutdown_connections 2023-10-06T21:09:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 --2- 172.21.15.26:0/4110341908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102a30 0x7f315c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 --2- 172.21.15.26:0/4110341908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c100e90 0x7f315c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 --2- 172.21.15.26:0/4110341908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c102070 0x7f315c1024f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 -- 172.21.15.26:0/4110341908 >> 172.21.15.26:0/4110341908 conn(0x7f315c0fc430 msgr2=0x7f315c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 -- 172.21.15.26:0/4110341908 shutdown_connections 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.144+0000 7f31621ff700 1 -- 172.21.15.26:0/4110341908 wait complete. 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 Processor -- start 2023-10-06T21:09:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 -- start start 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102070 0x7f315c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c102a30 0x7f315c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c19e770 con 0x7f315c100e90 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c19e8e0 con 0x7f315c102a30 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.145+0000 7f31621ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c19eaa0 con 0x7f315c102070 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49538/0 (socket says 172.21.15.26:49538) 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 -- 172.21.15.26:0/3610446257 learned_addr learned my addr 172.21.15.26:0/3610446257 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315bfff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c102a30 0x7f315c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315affd700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102070 0x7f315c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 -- 172.21.15.26:0/3610446257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102070 msgr2=0x7f315c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102070 0x7f315c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 -- 172.21.15.26:0/3610446257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c102a30 msgr2=0x7f315c19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c102a30 0x7f315c19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 -- 172.21.15.26:0/3610446257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500095f0 con 0x7f315c100e90 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.146+0000 7f315b7fe700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f314c00e970 tx=0x7f314c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.147+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c009790 con 0x7f315c100e90 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.147+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f314c00c7e0 con 0x7f315c100e90 2023-10-06T21:09:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.147+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315c19ed80 con 0x7f315c100e90 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.147+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c003c70 con 0x7f315c100e90 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.147+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c19f220 con 0x7f315c100e90 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.148+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315c107e90 con 0x7f315c100e90 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.150+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f314c00cc50 con 0x7f315c100e90 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.151+0000 7f3158ff9700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 0x7f31440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.151+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f314c014070 con 0x7f315c100e90 2023-10-06T21:09:51.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.152+0000 7f315affd700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 0x7f31440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.152+0000 7f315affd700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 0x7f31440a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f315000b2e0 tx=0x7f3150005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:51.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.155+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f314c0d3080 con 0x7f315c100e90 2023-10-06T21:09:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.365+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f315c10c750 con 0x7f315c100e90 2023-10-06T21:09:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.365+0000 7f3158ff9700 1 -- 172.21.15.26:0/3610446257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f314c0964a0 con 0x7f315c100e90 2023-10-06T21:09:51.369 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:09:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.367+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 msgr2=0x7f31440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.367+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 0x7f31440a39c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f315000b2e0 tx=0x7f3150005fb0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.367+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 msgr2=0x7f315c196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.367+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f314c00e970 tx=0x7f314c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 shutdown_connections 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f315c102a30 0x7f315c19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f31440a1500 0x7f31440a39c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f315c100e90 0x7f315c196c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 --2- 172.21.15.26:0/3610446257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f315c102070 0x7f315c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 >> 172.21.15.26:0/3610446257 conn(0x7f315c0fc430 msgr2=0x7f315c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 shutdown_connections 2023-10-06T21:09:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.368+0000 7f31621ff700 1 -- 172.21.15.26:0/3610446257 wait complete. 2023-10-06T21:09:51.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:09:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.594+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/207465684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14074ef0 msgr2=0x7f1a14073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.594+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/207465684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14074ef0 0x7f1a14073340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009990 tx=0x7f1a08009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/207465684 shutdown_connections 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/207465684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14108da0 0x7f1a1410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/207465684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14074ef0 0x7f1a14073340 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/207465684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/207465684 >> 172.21.15.26:0/207465684 conn(0x7f1a140fc430 msgr2=0x7f1a140fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/207465684 shutdown_connections 2023-10-06T21:09:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.595+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/207465684 wait complete. 2023-10-06T21:09:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 Processor -- start 2023-10-06T21:09:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 -- start start 2023-10-06T21:09:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14074ef0 0x7f1a14199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141a0950 con 0x7f1a14108da0 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141a0a90 con 0x7f1a14074ef0 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.596+0000 7f1a1b1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141a0c20 con 0x7f1a14073910 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a18f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49558/0 (socket says 172.21.15.26:49558) 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a18f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54912/0 (socket says 172.21.15.26:54912) 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 -- 172.21.15.26:0/3204093509 learned_addr learned my addr 172.21.15.26:0/3204093509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a13fff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14074ef0 0x7f1a14199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 -- 172.21.15.26:0/3204093509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 msgr2=0x7f1a14198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 -- 172.21.15.26:0/3204093509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14074ef0 msgr2=0x7f1a14199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14074ef0 0x7f1a14199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 -- 172.21.15.26:0/3204093509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080095f0 con 0x7f1a14108da0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1979c700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00ef30 tx=0x7f1a0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c018070 con 0x7f1a14108da0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0c00f040 con 0x7f1a14108da0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00cd00 con 0x7f1a14108da0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.597+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a141a0ea0 con 0x7f1a14108da0 2023-10-06T21:09:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.598+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a141a12d0 con 0x7f1a14108da0 2023-10-06T21:09:51.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.599+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a14107450 con 0x7f1a14108da0 2023-10-06T21:09:51.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.601+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1a0c003a00 con 0x7f1a14108da0 2023-10-06T21:09:51.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.602+0000 7f1a11ffb700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 0x7f1a040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.602+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a0c0cc7f0 con 0x7f1a14108da0 2023-10-06T21:09:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.602+0000 7f1a18f9b700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 0x7f1a040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.603+0000 7f1a18f9b700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 0x7f1a040a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b2e0 tx=0x7f1a08005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.606+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1a0c095ee0 con 0x7f1a14108da0 2023-10-06T21:09:51.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:51 smithi167 ceph-mon[134494]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:51 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3314906086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:51.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:51 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3610446257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.817+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a1402d270 con 0x7f1a14108da0 2023-10-06T21:09:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.818+0000 7f1a11ffb700 1 -- 172.21.15.26:0/3204093509 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a0c099500 con 0x7f1a14108da0 2023-10-06T21:09:51.820 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:09:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 msgr2=0x7f1a040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 0x7f1a040a39c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800b2e0 tx=0x7f1a08005fb0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 msgr2=0x7f1a141a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:51.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00ef30 tx=0x7f1a0c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 shutdown_connections 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f1a040a1500 0x7f1a040a39c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a14108da0 0x7f1a141a0380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a14073910 0x7f1a14198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 --2- 172.21.15.26:0/3204093509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a14074ef0 0x7f1a14199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.820+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 >> 172.21.15.26:0/3204093509 conn(0x7f1a140fc430 msgr2=0x7f1a140fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.821+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 shutdown_connections 2023-10-06T21:09:51.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:51 smithi026 ceph-mon[98490]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:51.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:51 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3314906086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:51.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:51 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3610446257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:51.821+0000 7f1a1b1ff700 1 -- 172.21.15.26:0/3204093509 wait complete. 2023-10-06T21:09:51.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:09:51.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:51 smithi078 ceph-mon[134896]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:51.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:51 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3314906086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:51.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:51 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3610446257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.045+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1671679530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 msgr2=0x7fc7141024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.045+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1671679530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7141024f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc704009990 tx=0x7fc704009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1671679530 shutdown_connections 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1671679530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714102a30 0x7fc714109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1671679530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714100e90 0x7fc7141012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1671679530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7141024f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1671679530 >> 172.21.15.26:0/1671679530 conn(0x7fc7140fc430 msgr2=0x7fc7140fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1671679530 shutdown_connections 2023-10-06T21:09:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.046+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1671679530 wait complete. 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.047+0000 7fc71b3ff700 1 Processor -- start 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.047+0000 7fc71b3ff700 1 -- start start 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 0x7fc714074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714102a30 0x7fc7140737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714073f80 con 0x7fc714102070 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140740f0 con 0x7fc714102a30 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71b3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141a0c70 con 0x7fc714100e90 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71899a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71899a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49564/0 (socket says 172.21.15.26:49564) 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71919b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 0x7fc714074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71899a700 1 -- 172.21.15.26:0/1204380070 learned_addr learned my addr 172.21.15.26:0/1204380070 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.048+0000 7fc71999c700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714102a30 0x7fc7140737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 -- 172.21.15.26:0/1204380070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 msgr2=0x7fc714074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 0x7fc714074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 -- 172.21.15.26:0/1204380070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714102a30 msgr2=0x7fc7140737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714102a30 0x7fc7140737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 -- 172.21.15.26:0/1204380070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040095f0 con 0x7fc714102070 2023-10-06T21:09:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71919b700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 0x7fc714074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71899a700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc704005f50 tx=0x7fc704005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70401d070 con 0x7fc714102070 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc704022470 con 0x7fc714102070 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7141a0ef0 con 0x7fc714102070 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704017650 con 0x7fc714102070 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.049+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141a1260 con 0x7fc714102070 2023-10-06T21:09:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.050+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714107e90 con 0x7fc714102070 2023-10-06T21:09:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.053+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc704022850 con 0x7fc714102070 2023-10-06T21:09:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.054+0000 7fc70a7fc700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 0x7fc7000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.054+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7040ce2b0 con 0x7fc714102070 2023-10-06T21:09:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.054+0000 7fc71919b700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 0x7fc7000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.055+0000 7fc71919b700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 0x7fc7000a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc714101be0 tx=0x7fc710009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.058+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc704097a80 con 0x7fc714102070 2023-10-06T21:09:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.267+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc71402d020 con 0x7fc714102070 2023-10-06T21:09:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.267+0000 7fc70a7fc700 1 -- 172.21.15.26:0/1204380070 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc704027020 con 0x7fc714102070 2023-10-06T21:09:52.270 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 msgr2=0x7fc7000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 0x7fc7000a3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc714101be0 tx=0x7fc710009260 comp rx=0 tx=0).stop 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 msgr2=0x7fc7140732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc704005f50 tx=0x7fc704005740 comp rx=0 tx=0).stop 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 shutdown_connections 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc714102a30 0x7fc7140737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc714100e90 0x7fc714074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fc7000a1480 0x7fc7000a3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 --2- 172.21.15.26:0/1204380070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc714102070 0x7fc7140732a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 >> 172.21.15.26:0/1204380070 conn(0x7fc7140fc430 msgr2=0x7fc7140fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 shutdown_connections 2023-10-06T21:09:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.270+0000 7fc71b3ff700 1 -- 172.21.15.26:0/1204380070 wait complete. 2023-10-06T21:09:52.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:09:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 -- 172.21.15.26:0/3232284808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea38101980 msgr2=0x7fea38103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/3232284808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea38101980 0x7fea38103d70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea28009990 tx=0x7fea28009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 -- 172.21.15.26:0/3232284808 shutdown_connections 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/3232284808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38106be0 0x7fea38109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/3232284808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea38101980 0x7fea38103d70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/3232284808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea381042b0 0x7fea381066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 -- 172.21.15.26:0/3232284808 >> 172.21.15.26:0/3232284808 conn(0x7fea380fb310 msgr2=0x7fea380fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 -- 172.21.15.26:0/3232284808 shutdown_connections 2023-10-06T21:09:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.494+0000 7fea3e5ff700 1 -- 172.21.15.26:0/3232284808 wait complete. 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 Processor -- start 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 -- start start 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 0x7fea381992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea38106be0 0x7fea381a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea381a08f0 con 0x7fea38106be0 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea381a0a60 con 0x7fea38101980 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea3e5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea381a0c20 con 0x7fea381042b0 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea37fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49178/0 (socket says 172.21.15.26:49178) 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.496+0000 7fea37fff700 1 -- 172.21.15.26:0/566378350 learned_addr learned my addr 172.21.15.26:0/566378350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea377fe700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 0x7fea381992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea3cb9c700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea38106be0 0x7fea381a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 -- 172.21.15.26:0/566378350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 msgr2=0x7fea381992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 0x7fea381992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 -- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea38106be0 msgr2=0x7fea381a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea38106be0 0x7fea381a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 -- 172.21.15.26:0/566378350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea280095f0 con 0x7fea38101980 2023-10-06T21:09:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea377fe700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 0x7fea381992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:52.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.497+0000 7fea37fff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea280059d0 tx=0x7fea28004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.499+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2801d070 con 0x7fea38101980 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.499+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2800fb90 con 0x7fea38101980 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.499+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea280173f0 con 0x7fea38101980 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.499+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea381a0ea0 con 0x7fea38101980 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.500+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea381a1210 con 0x7fea38101980 2023-10-06T21:09:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.501+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea381087a0 con 0x7fea38101980 2023-10-06T21:09:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.502+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fea2800fd00 con 0x7fea38101980 2023-10-06T21:09:52.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.504+0000 7fea357fa700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 0x7fea200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.504+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea280ce0d0 con 0x7fea38101980 2023-10-06T21:09:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.504+0000 7fea377fe700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 0x7fea200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:52.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.505+0000 7fea377fe700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 0x7fea200a3a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea2c011490 tx=0x7fea2c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:52.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.509+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fea2805d100 con 0x7fea38101980 2023-10-06T21:09:52.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:52 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3204093509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:52 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1204380070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.770+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea3810c6b0 con 0x7fea38101980 2023-10-06T21:09:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.771+0000 7fea357fa700 1 -- 172.21.15.26:0/566378350 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7fea28097780 con 0x7fea38101980 2023-10-06T21:09:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:52.774 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:09:52.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 msgr2=0x7fea200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 0x7fea200a3a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea2c011490 tx=0x7fea2c009260 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 msgr2=0x7fea38198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea280059d0 tx=0x7fea28004730 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 shutdown_connections 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fea200a1550 0x7fea200a3a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea38106be0 0x7fea381a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea38101980 0x7fea38198db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 --2- 172.21.15.26:0/566378350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea381042b0 0x7fea381992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 >> 172.21.15.26:0/566378350 conn(0x7fea380fb310 msgr2=0x7fea381025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 shutdown_connections 2023-10-06T21:09:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.774+0000 7fea3e5ff700 1 -- 172.21.15.26:0/566378350 wait complete. 2023-10-06T21:09:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:09:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:52 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3204093509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:52 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1204380070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:09:52.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:52 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3204093509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:09:52.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:52 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1204380070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 -- 172.21.15.26:0/3007248464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078102080 msgr2=0x7fa078102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/3007248464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078102080 0x7fa078102500 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009990 tx=0x7fa06c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 -- 172.21.15.26:0/3007248464 shutdown_connections 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/3007248464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102a40 0x7fa078109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/3007248464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078100ea0 0x7fa0781012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/3007248464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078102080 0x7fa078102500 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 -- 172.21.15.26:0/3007248464 >> 172.21.15.26:0/3007248464 conn(0x7fa0780fc420 msgr2=0x7fa0780fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 -- 172.21.15.26:0/3007248464 shutdown_connections 2023-10-06T21:09:53.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.998+0000 7fa07e1ff700 1 -- 172.21.15.26:0/3007248464 wait complete. 2023-10-06T21:09:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.999+0000 7fa07e1ff700 1 Processor -- start 2023-10-06T21:09:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.999+0000 7fa07e1ff700 1 -- start start 2023-10-06T21:09:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.999+0000 7fa07e1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:52.999+0000 7fa07e1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102080 0x7fa078197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa07e1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 0x7fa07819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa07e1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819e770 con 0x7fa078100ea0 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa07e1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819e8b0 con 0x7fa078102a40 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa07e1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819ea40 con 0x7fa078102080 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49610/0 (socket says 172.21.15.26:49610) 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa076ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102080 0x7fa078197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 -- 172.21.15.26:0/487820128 learned_addr learned my addr 172.21.15.26:0/487820128 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa077fff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 0x7fa07819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 -- 172.21.15.26:0/487820128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102080 msgr2=0x7fa078197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102080 0x7fa078197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 -- 172.21.15.26:0/487820128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 msgr2=0x7fa07819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 0x7fa07819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 -- 172.21.15.26:0/487820128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0095f0 con 0x7fa078100ea0 2023-10-06T21:09:53.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa077fff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 0x7fa07819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.000+0000 7fa0777fe700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa06800e970 tx=0x7fa06800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.001+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068009790 con 0x7fa078100ea0 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.001+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa06800c7e0 con 0x7fa078100ea0 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.001+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07819ed20 con 0x7fa078100ea0 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.001+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003c70 con 0x7fa078100ea0 2023-10-06T21:09:53.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.001+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07819f1d0 con 0x7fa078100ea0 2023-10-06T21:09:53.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.002+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07810c760 con 0x7fa078100ea0 2023-10-06T21:09:53.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.004+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa06800cc50 con 0x7fa078100ea0 2023-10-06T21:09:53.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.005+0000 7fa074ff9700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.005+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa068014070 con 0x7fa078100ea0 2023-10-06T21:09:53.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.005+0000 7fa076ffd700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.006+0000 7fa076ffd700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 0x7fa0600a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b2e0 tx=0x7fa06c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.009+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa0680d3080 con 0x7fa078100ea0 2023-10-06T21:09:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.223+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa078107e40 con 0x7fa078100ea0 2023-10-06T21:09:53.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.225+0000 7fa074ff9700 1 -- 172.21.15.26:0/487820128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7fa0680964a0 con 0x7fa078100ea0 2023-10-06T21:09:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:09:53.228 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:09:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 msgr2=0x7fa0600a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 0x7fa0600a3a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b2e0 tx=0x7fa06c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 msgr2=0x7fa078196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa06800e970 tx=0x7fa06800ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 shutdown_connections 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa078102a40 0x7fa07819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa0600a1550 0x7fa0600a3a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa078100ea0 0x7fa078196bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 --2- 172.21.15.26:0/487820128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078102080 0x7fa078197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.227+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 >> 172.21.15.26:0/487820128 conn(0x7fa0780fc420 msgr2=0x7fa0780fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.228+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 shutdown_connections 2023-10-06T21:09:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.228+0000 7fa07e1ff700 1 -- 172.21.15.26:0/487820128 wait complete. 2023-10-06T21:09:53.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-06T21:08:16.065653+0000', 'modified': '2023-10-06T21:08:21.849944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24587, 'mds_1': 24559, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6835/4281425672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 4281425672}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 4281425672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2946984451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2946984451}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2946984451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/1710830195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 1710830195}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 1710830195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.78:6837/1931966907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6836', 'nonce': 1931966907}, {'type': 'v1', 'addr': '172.21.15.78:6837', 'nonce': 1931966907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6841/2891556508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2891556508}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2891556508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24587': {'gid': 24587, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6839/3790769886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3790769886}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3790769886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-06T21:09:53.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-06T21:09:53.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-06T21:09:53.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2987986610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 msgr2=0x7fdbb80732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2987986610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb80732e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009990 tx=0x7fdbb4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2987986610 shutdown_connections 2023-10-06T21:09:53.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2987986610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb8108dc0 0x7fdbb810b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2987986610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb80732e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2987986610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb80738b0 0x7fdbb8073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2987986610 >> 172.21.15.26:0/2987986610 conn(0x7fdbb80fc460 msgr2=0x7fdbb80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2987986610 shutdown_connections 2023-10-06T21:09:53.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.451+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2987986610 wait complete. 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.452+0000 7fdbc07ff700 1 Processor -- start 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.452+0000 7fdbc07ff700 1 -- start start 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb80738b0 0x7fdbb8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108dc0 0x7fdbb81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a09b0 con 0x7fdbb8074e90 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a0af0 con 0x7fdbb8108dc0 2023-10-06T21:09:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbc07ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81a0c80 con 0x7fdbb80738b0 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49630/0 (socket says 172.21.15.26:49630) 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 -- 172.21.15.26:0/2450437341 learned_addr learned my addr 172.21.15.26:0/2450437341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb80738b0 0x7fdbb8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108dc0 0x7fdbb81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 -- 172.21.15.26:0/2450437341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb80738b0 msgr2=0x7fdbb8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb80738b0 0x7fdbb8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 -- 172.21.15.26:0/2450437341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108dc0 msgr2=0x7fdbb81a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108dc0 0x7fdbb81a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.453+0000 7fdbbdd9a700 1 -- 172.21.15.26:0/2450437341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009520 con 0x7fdbb8074e90 2023-10-06T21:09:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdba800eb30 tx=0x7fdba800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/2450437341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8009880 con 0x7fdbb8074e90 2023-10-06T21:09:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/2450437341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba80044e0 con 0x7fdbb8074e90 2023-10-06T21:09:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb40095f0 con 0x7fdbb8074e90 2023-10-06T21:09:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/2450437341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8003d80 con 0x7fdbb8074e90 2023-10-06T21:09:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.454+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb81a1260 con 0x7fdbb8074e90 2023-10-06T21:09:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.459+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/2450437341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdba8004000 con 0x7fdbb8074e90 2023-10-06T21:09:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 Processor -- start 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 -- start start 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbaf7fe700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 0x7fdba40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc6b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 0x56166fc6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbd2410 con 0x56166fc6ba10 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbc3bd0 con 0x56166fc6ab70 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbc14f0 con 0x56166fc6a210 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/2450437341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdba8014070 con 0x7fdbb8074e90 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.460+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 0x7fdba40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55014/0 (socket says 172.21.15.26:55014) 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 -- 172.21.15.26:0/2928118527 learned_addr learned my addr 172.21.15.26:0/2928118527 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc6b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 0x56166fc6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 -- 172.21.15.26:0/2928118527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 msgr2=0x56166fc6b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc6b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 -- 172.21.15.26:0/2928118527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 msgr2=0x56166fc6be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 0x56166fc6be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 -- 172.21.15.26:0/2928118527 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166f9e7d90 con 0x56166fc6a210 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 0x56166fc6be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4015000 tx=0x7fdbb4005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5498e771b2a39a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.461+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 0x7fdba40a39c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56166fc6b1f0 tx=0x7fdbb0009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbacff9700 1 -- 172.21.15.26:0/2928118527 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401d070 con 0x56166fc6a210 2023-10-06T21:09:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbacff9700 1 -- 172.21.15.26:0/2928118527 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb401e040 con 0x56166fc6a210 2023-10-06T21:09:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbacff9700 1 -- 172.21.15.26:0/2928118527 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40049c0 con 0x56166fc6a210 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbd408eb80 1 -- 172.21.15.26:0/2928118527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 msgr2=0x56166fc6a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4015000 tx=0x7fdbb4005ea0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbd408eb80 1 -- 172.21.15.26:0/2928118527 shutdown_connections 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.462+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6ba10 0x56166fc6be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6a210 0x56166fc6a630 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/2928118527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 -- 172.21.15.26:0/2928118527 >> 172.21.15.26:0/2928118527 conn(0x56166fbde210 msgr2=0x56166fc650f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 -- 172.21.15.26:0/2928118527 shutdown_connections 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 -- 172.21.15.26:0/2928118527 wait complete. 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.463+0000 7fdbd408eb80 1 -- start start 2023-10-06T21:09:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 Processor -- start 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fbbcde0 con 0x56166fc6a210 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fb596c0 con 0x56166fc6ba10 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fb59360 con 0x56166fc6ab70 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.464+0000 7fdbd408eb80 10 client.0 _ll_register_callbacks cb 0x56166f9b67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49648/0 (socket says 172.21.15.26:49648) 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 -- 172.21.15.26:0/3618512670 learned_addr learned my addr 172.21.15.26:0/3618512670 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 -- 172.21.15.26:0/3618512670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 msgr2=0x56166fc74460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc74460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 -- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 msgr2=0x56166fc7b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 -- 172.21.15.26:0/3618512670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb810be50 con 0x56166fc6a210 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc74460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009ce0 tx=0x7fdbb4022b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcb9501761a9e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.465+0000 7fdba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401d030 con 0x56166fc6a210 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56166fc5cec0 con 0x56166fc6a210 2023-10-06T21:09:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdbb400f660 con 0x56166fc6a210 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb4005680 con 0x56166fc6a210 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdbd408eb80 10 client.15090 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56166f9d0c10 con 0x56166fc6a210 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56166fc5cec0 con 0x56166fc6a210 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.466+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdbb402c490 con 0x56166fc6a210 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 20 client.15090 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fc7f660 0x56166fc7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 4 client.15090 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56166fbb5dc0 con 0x56166fc7f660 2023-10-06T21:09:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fc7f660 0x56166fc7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fc7f660 0x56166fc7e3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf45e0280bea5699 server_cookie=4ec1105c0f2f25f9 in_seq=0 out_seq=0 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.467+0000 7fdba1ffb700 10 client.15090 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.468+0000 7fdba1ffb700 1 -- 172.21.15.26:0/3618512670 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56166fbb5dc0 con 0x56166fc7f660 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.468+0000 7fdba1ffb700 10 client.15090 handle_command_reply: tid=0 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.468+0000 7fdbd408eb80 1 client.15090 shutdown 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 msgr2=0x56166fc73f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009ce0 tx=0x7fdbb4022b90 comp rx=0 tx=0).stop 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 shutdown_connections 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fc7f660 0x56166fc7e3d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc73f20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/3618512670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc74460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdba1ffb700 0 client.15090 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 >> 172.21.15.26:0/3618512670 conn(0x56166fbde210 msgr2=0x56166fc64f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 shutdown_connections 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 1 -- 172.21.15.26:0/3618512670 wait complete. 2023-10-06T21:09:53.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.469+0000 7fdbd408eb80 20 client.15090 trim_cache size 0 max 16384 2023-10-06T21:09:53.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 Processor -- start 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 -- start start 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc7b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fa3d8d0 con 0x56166fc6a210 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc803f0 con 0x56166fc6ba10 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.487+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc66800 con 0x56166fc6ab70 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49664/0 (socket says 172.21.15.26:49664) 2023-10-06T21:09:53.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 -- 172.21.15.26:0/1846274717 learned_addr learned my addr 172.21.15.26:0/1846274717 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc7b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 -- 172.21.15.26:0/1846274717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 msgr2=0x56166fc7b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc7b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 -- 172.21.15.26:0/1846274717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 msgr2=0x56166fc7e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 -- 172.21.15.26:0/1846274717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56166fa0e200 con 0x56166fc6a210 2023-10-06T21:09:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb400bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d949f277aa44e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.488+0000 7fdbad7fa700 1 -- 172.21.15.26:0/1846274717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401d040 con 0x56166fc6a210 2023-10-06T21:09:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbad7fa700 1 -- 172.21.15.26:0/1846274717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb402cb10 con 0x56166fc6a210 2023-10-06T21:09:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbad7fa700 1 -- 172.21.15.26:0/1846274717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4004760 con 0x56166fc6a210 2023-10-06T21:09:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 -- 172.21.15.26:0/1846274717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 msgr2=0x56166fc71e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4000c00 tx=0x7fdbb400bb70 comp rx=0 tx=0).stop 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 -- 172.21.15.26:0/1846274717 shutdown_connections 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ba10 0x56166fc7e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc71e80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/1846274717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ab70 0x56166fc7b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 -- 172.21.15.26:0/1846274717 >> 172.21.15.26:0/1846274717 conn(0x56166fbde210 msgr2=0x56166fc63b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 -- 172.21.15.26:0/1846274717 shutdown_connections 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.489+0000 7fdbd408eb80 1 -- 172.21.15.26:0/1846274717 wait complete. 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 -- start start 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 Processor -- start 2023-10-06T21:09:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ba10 0x56166fc7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc7aa00 con 0x56166fc6a210 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc7ab70 con 0x56166fc6ab70 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56166fc7ad30 con 0x56166fc6ba10 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbd408eb80 10 client.0 _ll_register_callbacks cb 0x7fdba8003720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.490+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49668/0 (socket says 172.21.15.26:49668) 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 -- 172.21.15.26:0/140121074 learned_addr learned my addr 172.21.15.26:0/140121074 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbed9c700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ba10 0x56166fc7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 -- 172.21.15.26:0/140121074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ba10 msgr2=0x56166fc7a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ba10 0x56166fc7a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 -- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 msgr2=0x56166fc774a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc774a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 -- 172.21.15.26:0/140121074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56166f9d0c10 con 0x56166fc6a210 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdbbe59b700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c010 tx=0x7fdbb4005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4413997fb573a9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.491+0000 7fdba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401d040 con 0x56166fc6a210 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbb401e040 con 0x56166fc6a210 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdbb4000e40 con 0x56166fc6a210 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb403c930 con 0x56166fc6a210 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 10 client.15096 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56166f9e6260 con 0x56166fc6a210 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb4053020 con 0x56166fc6a210 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdbb400fcf0 con 0x56166fc6a210 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 20 client.15096 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fcd7170 0x56166fcd35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 4 client.15096 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.492+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56166fbb5ae0 con 0x56166fcd7170 2023-10-06T21:09:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fcd7170 0x56166fcd35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdbbdd9a700 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fcd7170 0x56166fcd35d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0450a412836eb60 server_cookie=d3846b7b674aada7 in_seq=0 out_seq=0 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdba1ffb700 10 client.15096 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdba1ffb700 1 -- 172.21.15.26:0/140121074 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56166fbb5ae0 con 0x56166fcd7170 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdba1ffb700 10 client.15096 handle_command_reply: tid=0 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.493+0000 7fdbd408eb80 1 client.15096 shutdown 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 msgr2=0x56166fc76f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400c010 tx=0x7fdbb4005d00 comp rx=0 tx=0).stop 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 shutdown_connections 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x56166fcd7170 0x56166fcd35d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56166fc6ba10 0x56166fc7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166fc6a210 0x56166fc76f60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdba1ffb700 0 client.15096 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 --2- 172.21.15.26:0/140121074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56166fc6ab70 0x56166fc774a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 >> 172.21.15.26:0/140121074 conn(0x56166fbde210 msgr2=0x56166fc71e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 shutdown_connections 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 1 -- 172.21.15.26:0/140121074 wait complete. 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbd408eb80 20 client.15096 trim_cache size 0 max 16384 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:53.502 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24539, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 92.648429823000001, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 114.106956529 2023-10-06T21:09:53.503 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 msgr2=0x7fdba40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 0x7fdba40a39c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56166fc6b1f0 tx=0x7fdbb0009920 comp rx=0 tx=0).stop 2023-10-06T21:09:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 msgr2=0x7fdbb8199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.494+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdba800eb30 tx=0x7fdba800ee40 comp rx=0 tx=0).stop 2023-10-06T21:09:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 shutdown_connections 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108dc0 0x7fdbb81a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb80738b0 0x7fdbb8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fdba40a1500 0x7fdba40a39c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 --2- 172.21.15.26:0/2450437341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb8074e90 0x7fdbb8199370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 >> 172.21.15.26:0/2450437341 conn(0x7fdbb80fc460 msgr2=0x7fdbb80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 shutdown_connections 2023-10-06T21:09:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.495+0000 7fdbc07ff700 1 -- 172.21.15.26:0/2450437341 wait complete. 2023-10-06T21:09:53.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T21:09:53.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 -- 172.21.15.26:0/2822978389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c102a30 msgr2=0x7fd24c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 --2- 172.21.15.26:0/2822978389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c102a30 0x7fd24c109350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b0a0 tx=0x7fd24400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 -- 172.21.15.26:0/2822978389 shutdown_connections 2023-10-06T21:09:53.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 --2- 172.21.15.26:0/2822978389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c102a30 0x7fd24c109350 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 --2- 172.21.15.26:0/2822978389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c100e90 0x7fd24c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 --2- 172.21.15.26:0/2822978389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102070 0x7fd24c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 -- 172.21.15.26:0/2822978389 >> 172.21.15.26:0/2822978389 conn(0x7fd24c0fc430 msgr2=0x7fd24c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 -- 172.21.15.26:0/2822978389 shutdown_connections 2023-10-06T21:09:53.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.717+0000 7fd2519ff700 1 -- 172.21.15.26:0/2822978389 wait complete. 2023-10-06T21:09:53.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.718+0000 7fd2519ff700 1 Processor -- start 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.718+0000 7fd2519ff700 1 -- start start 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c102070 0x7fd24c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102a30 0x7fd24c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c073f80 con 0x7fd24c100e90 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0740f0 con 0x7fd24c102070 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd2519ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c1a0d60 con 0x7fd24c102a30 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49684/0 (socket says 172.21.15.26:49684) 2023-10-06T21:09:53.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 -- 172.21.15.26:0/4038823154 learned_addr learned my addr 172.21.15.26:0/4038823154 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c102070 0x7fd24c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102a30 0x7fd24c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 -- 172.21.15.26:0/4038823154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102a30 msgr2=0x7fd24c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102a30 0x7fd24c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 -- 172.21.15.26:0/4038823154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c102070 msgr2=0x7fd24c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c102070 0x7fd24c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.719+0000 7fd24affd700 1 -- 172.21.15.26:0/4038823154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244009d00 con 0x7fd24c100e90 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd24affd700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e970 tx=0x7fd23c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd2509fd700 1 -- 172.21.15.26:0/4038823154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c009790 con 0x7fd24c100e90 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd2509fd700 1 -- 172.21.15.26:0/4038823154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c00eec0 con 0x7fd24c100e90 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c1a0fe0 con 0x7fd24c100e90 2023-10-06T21:09:53.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd2509fd700 1 -- 172.21.15.26:0/4038823154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003cf0 con 0x7fd24c100e90 2023-10-06T21:09:53.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.720+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c1a14b0 con 0x7fd24c100e90 2023-10-06T21:09:53.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.723+0000 7fd2509fd700 1 -- 172.21.15.26:0/4038823154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd23c00ccd0 con 0x7fd24c100e90 2023-10-06T21:09:53.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.725+0000 7fd2653d5b80 1 Processor -- start 2023-10-06T21:09:53.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 -- start start 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efccb2410 con 0x564efcd4ba10 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcca3bd0 con 0x564efcd4ab70 2023-10-06T21:09:53.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcca14f0 con 0x564efcd4a210 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2509fd700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 0x7fd2340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49278/0 (socket says 172.21.15.26:49278) 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24affd700 1 -- 172.21.15.26:0/610437819 learned_addr learned my addr 172.21.15.26:0/610437819 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd4be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49696/0 (socket says 172.21.15.26:49696) 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd2509fd700 1 -- 172.21.15.26:0/4038823154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd23c014070 con 0x7fd24c100e90 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.726+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 0x7fd2340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 -- 172.21.15.26:0/610437819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 msgr2=0x564efcd4a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd4a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 -- 172.21.15.26:0/610437819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 msgr2=0x564efcd4be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd4be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 -- 172.21.15.26:0/610437819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564efcac7d90 con 0x564efcd4ab70 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24affd700 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0caa20 tx=0x7fd23c007550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c6002c5d52d8496 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 0x7fd2340a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0752c0 tx=0x7fd24400b3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd231ffb700 1 -- 172.21.15.26:0/610437819 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c099ba0 con 0x564efcd4ab70 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd231ffb700 1 -- 172.21.15.26:0/610437819 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c09a940 con 0x564efcd4ab70 2023-10-06T21:09:53.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.727+0000 7fd231ffb700 1 -- 172.21.15.26:0/610437819 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c096950 con 0x564efcd4ab70 2023-10-06T21:09:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 -- 172.21.15.26:0/610437819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 msgr2=0x564efcd4b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0caa20 tx=0x7fd23c007550 comp rx=0 tx=0).stop 2023-10-06T21:09:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 -- 172.21.15.26:0/610437819 shutdown_connections 2023-10-06T21:09:53.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd4be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/610437819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd4b430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 -- 172.21.15.26:0/610437819 >> 172.21.15.26:0/610437819 conn(0x564efccbe210 msgr2=0x564efcd450f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 -- 172.21.15.26:0/610437819 shutdown_connections 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.728+0000 7fd2653d5b80 1 -- 172.21.15.26:0/610437819 wait complete. 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 -- start start 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 Processor -- start 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcc9cde0 con 0x564efcd4a210 2023-10-06T21:09:53.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcc396c0 con 0x564efcd4ba10 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcc39360 con 0x564efcd4ab70 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.729+0000 7fd2653d5b80 10 client.0 _ll_register_callbacks cb 0x564efca967b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49698/0 (socket says 172.21.15.26:49698) 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 -- 172.21.15.26:0/2946125187 learned_addr learned my addr 172.21.15.26:0/2946125187 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 -- 172.21.15.26:0/2946125187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 msgr2=0x564efcd54510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd54510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 -- 172.21.15.26:0/2946125187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 msgr2=0x564efcd5b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd5b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 -- 172.21.15.26:0/2946125187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c1a14b0 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd24affd700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009230 tx=0x7fd23c0a26d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24a6f3aa071da7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd22affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c09e030 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd23c0071c0 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.730+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564efcd3cec0 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23c09e190 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 10 client.15108 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564efcab0c10 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c0b6080 con 0x564efcd4a210 2023-10-06T21:09:53.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd23c0a2140 con 0x564efcd4a210 2023-10-06T21:09:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:53.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 10 client.15108 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 20 client.15108 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcd5f7f0 0x564efcd5e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 4 client.15108 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564efcc95dc0 con 0x564efcd5f7f0 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.731+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcd5f7f0 0x564efcd5e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.732+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcd5f7f0 0x564efcd5e480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f011ab4aefc6ad11 server_cookie=8548d66f4ccb1d58 in_seq=0 out_seq=0 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.732+0000 7fd22affd700 10 client.15108 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.732+0000 7fd22affd700 1 -- 172.21.15.26:0/2946125187 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564efcc95dc0 con 0x564efcd5f7f0 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd22affd700 10 client.15108 handle_command_reply: tid=0 2023-10-06T21:09:53.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 client.15108 shutdown 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 msgr2=0x564efcd53fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009230 tx=0x7fd23c0a26d0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 shutdown_connections 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd5b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd53fd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcd5f7f0 0x564efcd5e480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/2946125187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd54510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd22affd700 0 client.15108 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 >> 172.21.15.26:0/2946125187 conn(0x564efccbe210 msgr2=0x564efcd44f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 shutdown_connections 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 1 -- 172.21.15.26:0/2946125187 wait complete. 2023-10-06T21:09:53.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.733+0000 7fd2653d5b80 20 client.15108 trim_cache size 0 max 16384 2023-10-06T21:09:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.751+0000 7fd2653d5b80 1 Processor -- start 2023-10-06T21:09:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.751+0000 7fd2653d5b80 1 -- start start 2023-10-06T21:09:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcb1d8d0 con 0x564efcd4a210 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcc9e070 con 0x564efcd4ba10 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcc434d0 con 0x564efcd4ab70 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49714/0 (socket says 172.21.15.26:49714) 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 -- 172.21.15.26:0/3947888784 learned_addr learned my addr 172.21.15.26:0/3947888784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 -- 172.21.15.26:0/3947888784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 msgr2=0x564efcd5b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd5b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 -- 172.21.15.26:0/3947888784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 msgr2=0x564efcd52c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd52c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24affd700 1 -- 172.21.15.26:0/3947888784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564efcaee200 con 0x564efcd4a210 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.752+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd24affd700 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005950 tx=0x7fd23c0a2660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562f66432b0ac15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2327fc700 1 -- 172.21.15.26:0/3947888784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c099730 con 0x564efcd4a210 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2327fc700 1 -- 172.21.15.26:0/3947888784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564efcd3cec0 con 0x564efcd4a210 2023-10-06T21:09:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2327fc700 1 -- 172.21.15.26:0/3947888784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c09a4a0 con 0x564efcd4a210 2023-10-06T21:09:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 -- 172.21.15.26:0/3947888784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 msgr2=0x564efcd43b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005950 tx=0x7fd23c0a2660 comp rx=0 tx=0).stop 2023-10-06T21:09:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 -- 172.21.15.26:0/3947888784 shutdown_connections 2023-10-06T21:09:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ba10 0x564efcd52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4a210 0x564efcd43b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/3947888784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4ab70 0x564efcd5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 -- 172.21.15.26:0/3947888784 >> 172.21.15.26:0/3947888784 conn(0x564efccbe210 msgr2=0x564efcd68150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.753+0000 7fd2653d5b80 1 -- 172.21.15.26:0/3947888784 shutdown_connections 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.754+0000 7fd2653d5b80 1 -- 172.21.15.26:0/3947888784 wait complete. 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.754+0000 7fd2653d5b80 1 -- start start 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.754+0000 7fd2653d5b80 1 Processor -- start 2023-10-06T21:09:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.754+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.754+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd2653d5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcd5b150 con 0x564efcd4ba10 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcd5b2c0 con 0x564efcd4ab70 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd2653d5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564efcd5b480 con 0x564efcd4a210 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd2653d5b80 10 client.0 _ll_register_callbacks cb 0x7fd240009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49718/0 (socket says 172.21.15.26:49718) 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 -- 172.21.15.26:0/768073236 learned_addr learned my addr 172.21.15.26:0/768073236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 -- 172.21.15.26:0/768073236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 msgr2=0x564efcd576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 -- 172.21.15.26:0/768073236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 msgr2=0x564efcd57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24b7fe700 1 -- 172.21.15.26:0/768073236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564efcab0c10 con 0x564efcd4ba10 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24affd700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.755+0000 7fd24a7fc700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd24b7fe700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b0a0 tx=0x7fd2440099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7352e5bb4187193d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd22affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24400e070 con 0x564efcd4ba10 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd23c09a940 con 0x564efcd4ba10 2023-10-06T21:09:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd24402f690 con 0x564efcd4ba10 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd24400be80 con 0x564efcd4ba10 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd2653d5b80 10 client.15114 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564efcac6260 con 0x564efcd4ba10 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.756+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c09a940 con 0x564efcd4ba10 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd244036060 con 0x564efcd4ba10 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 10 client.15114 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 20 client.15114 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcdb7f50 0x564efcdb43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 4 client.15114 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564efcc95ae0 con 0x564efcdb7f50 2023-10-06T21:09:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd24affd700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcdb7f50 0x564efcdb43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd22affd700 10 client.15114 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.757+0000 7fd24affd700 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcdb7f50 0x564efcdb43f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8168a136f2bfcd89 server_cookie=7d35b6862def5f0f in_seq=0 out_seq=0 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd22affd700 1 -- 172.21.15.26:0/768073236 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564efcc95ae0 con 0x564efcdb7f50 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd22affd700 10 client.15114 handle_command_reply: tid=0 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 client.15114 shutdown 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 msgr2=0x564efcd5ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd24400b0a0 tx=0x7fd2440099e0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 shutdown_connections 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564efcd4ba10 0x564efcd5ac10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x564efcdb7f50 0x564efcdb43f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564efcd4a210 0x564efcd576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 --2- 172.21.15.26:0/768073236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564efcd4ab70 0x564efcd57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd22affd700 0 client.15114 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 >> 172.21.15.26:0/768073236 conn(0x564efccbe210 msgr2=0x564efcd43b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 shutdown_connections 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 1 -- 172.21.15.26:0/768073236 wait complete. 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.758+0000 7fd2653d5b80 20 client.15114 trim_cache size 0 max 16384 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:53.767 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24559, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 93.922213201999995, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 105.974465456 2023-10-06T21:09:53.768 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 msgr2=0x7fd2340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 0x7fd2340a39c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0752c0 tx=0x7fd24400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 msgr2=0x7fd24c074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e970 tx=0x7fd23c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 shutdown_connections 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd24c102a30 0x7fd24c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd2340a1500 0x7fd2340a39c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd24c100e90 0x7fd24c074c60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 --2- 172.21.15.26:0/4038823154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24c102070 0x7fd24c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 >> 172.21.15.26:0/4038823154 conn(0x7fd24c0fc430 msgr2=0x7fd24c1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 shutdown_connections 2023-10-06T21:09:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.759+0000 7fd2519ff700 1 -- 172.21.15.26:0/4038823154 wait complete. 2023-10-06T21:09:53.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:53 smithi026 ceph-mon[98490]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:53.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:53 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/566378350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:53 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/487820128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T21:09:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:53 smithi167 ceph-mon[134494]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:53 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/566378350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:53 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/487820128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:53 smithi078 ceph-mon[134896]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:53 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/566378350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:53 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/487820128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 -- 172.21.15.26:0/2090010168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd080106c00 msgr2=0x7fd080109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 --2- 172.21.15.26:0/2090010168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd080106c00 0x7fd080109010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd078009f90 tx=0x7fd07800b330 comp rx=0 tx=0).stop 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 -- 172.21.15.26:0/2090010168 shutdown_connections 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 --2- 172.21.15.26:0/2090010168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd080106c00 0x7fd080109010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 --2- 172.21.15.26:0/2090010168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0801019a0 0x7fd080103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 --2- 172.21.15.26:0/2090010168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801042d0 0x7fd0801066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 -- 172.21.15.26:0/2090010168 >> 172.21.15.26:0/2090010168 conn(0x7fd0800fb330 msgr2=0x7fd0800fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 -- 172.21.15.26:0/2090010168 shutdown_connections 2023-10-06T21:09:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.990+0000 7fd086fff700 1 -- 172.21.15.26:0/2090010168 wait complete. 2023-10-06T21:09:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 Processor -- start 2023-10-06T21:09:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 -- start start 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801019a0 0x7fd080101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd080106c00 0x7fd0800ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080100630 con 0x7fd0801042d0 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801007a0 con 0x7fd080106c00 2023-10-06T21:09:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.992+0000 7fd086fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801a0d70 con 0x7fd0801019a0 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49732/0 (socket says 172.21.15.26:49732) 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 -- 172.21.15.26:0/4018639585 learned_addr learned my addr 172.21.15.26:0/4018639585 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd084d9b700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801019a0 0x7fd080101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd08559c700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd080106c00 0x7fd0800ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 -- 172.21.15.26:0/4018639585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801019a0 msgr2=0x7fd080101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801019a0 0x7fd080101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 -- 172.21.15.26:0/4018639585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd080106c00 msgr2=0x7fd0800ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd080106c00 0x7fd0800ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 -- 172.21.15.26:0/4018639585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074009520 con 0x7fd0801042d0 2023-10-06T21:09:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07ffff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9c0 tx=0x7fd06c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07dffb700 1 -- 172.21.15.26:0/4018639585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c004a30 con 0x7fd0801042d0 2023-10-06T21:09:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07dffb700 1 -- 172.21.15.26:0/4018639585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06c00cc80 con 0x7fd0801042d0 2023-10-06T21:09:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.993+0000 7fd07dffb700 1 -- 172.21.15.26:0/4018639585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0054e0 con 0x7fd0801042d0 2023-10-06T21:09:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.994+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd078009c40 con 0x7fd0801042d0 2023-10-06T21:09:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.994+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0801a12f0 con 0x7fd0801042d0 2023-10-06T21:09:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.997+0000 7fd07dffb700 1 -- 172.21.15.26:0/4018639585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd06c004b90 con 0x7fd0801042d0 2023-10-06T21:09:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.998+0000 7fd09a845b80 1 Processor -- start 2023-10-06T21:09:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 -- start start 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf619630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf61ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf581410 con 0x55c6cf619b70 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf572bd0 con 0x55c6cf61aa10 2023-10-06T21:09:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd09a845b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf5704f0 con 0x55c6cf619210 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd07dffb700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 0x7fd0700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd084d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf61ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd084d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49744/0 (socket says 172.21.15.26:49744) 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd084d9b700 1 -- 172.21.15.26:0/3390909872 learned_addr learned my addr 172.21.15.26:0/3390909872 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:53.999+0000 7fd08559c700 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf619630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd07dffb700 1 -- 172.21.15.26:0/4018639585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd06c014070 con 0x7fd0801042d0 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 -- 172.21.15.26:0/3390909872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 msgr2=0x55c6cf619630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf619630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd08559c700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 0x7fd0700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 -- 172.21.15.26:0/3390909872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 msgr2=0x55c6cf61ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf61ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 -- 172.21.15.26:0/3390909872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6cf396d90 con 0x55c6cf619b70 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd07ffff700 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf61ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd084d9b700 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd074009f10 tx=0x7fd074011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a181dbf4231a237 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.000+0000 7fd06affd700 1 -- 172.21.15.26:0/3390909872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074019630 con 0x55c6cf619b70 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd08559c700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 0x7fd0700a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c10 tx=0x7fd0780096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd06affd700 1 -- 172.21.15.26:0/3390909872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd074019c50 con 0x55c6cf619b70 2023-10-06T21:09:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd06affd700 1 -- 172.21.15.26:0/3390909872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07400ece0 con 0x55c6cf619b70 2023-10-06T21:09:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 -- 172.21.15.26:0/3390909872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 msgr2=0x55c6cf61a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd074009f10 tx=0x7fd074011720 comp rx=0 tx=0).stop 2023-10-06T21:09:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 -- 172.21.15.26:0/3390909872 shutdown_connections 2023-10-06T21:09:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf61ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf619630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 --2- 172.21.15.26:0/3390909872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf61a430 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 -- 172.21.15.26:0/3390909872 >> 172.21.15.26:0/3390909872 conn(0x55c6cf58d210 msgr2=0x55c6cf6140f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 -- 172.21.15.26:0/3390909872 shutdown_connections 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.001+0000 7fd09a845b80 1 -- 172.21.15.26:0/3390909872 wait complete. 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.002+0000 7fd09a845b80 1 -- start start 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.002+0000 7fd09a845b80 1 Processor -- start 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf622f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 0x55c6cf62a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf56bde0 con 0x55c6cf619b70 2023-10-06T21:09:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf5086c0 con 0x55c6cf619210 2023-10-06T21:09:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf508360 con 0x55c6cf61aa10 2023-10-06T21:09:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd09a845b80 10 client.0 _ll_register_callbacks cb 0x55c6cf3657b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd08559c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 0x55c6cf62a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49760/0 (socket says 172.21.15.26:49760) 2023-10-06T21:09:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 -- 172.21.15.26:0/118584776 learned_addr learned my addr 172.21.15.26:0/118584776 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:54.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd084d9b700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf622f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 -- 172.21.15.26:0/118584776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 msgr2=0x55c6cf62a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 0x55c6cf62a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 -- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 msgr2=0x55c6cf622f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf622f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd07ffff700 1 -- 172.21.15.26:0/118584776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd074009b70 con 0x55c6cf619b70 2023-10-06T21:09:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.003+0000 7fd08559c700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 0x55c6cf62a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd07ffff700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd06c020000 tx=0x7fd06c06a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc4c99f42103f07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd05ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00cdf0 con 0x55c6cf619b70 2023-10-06T21:09:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6cf60bec0 con 0x55c6cf619b70 2023-10-06T21:09:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd06c09a770 con 0x55c6cf619b70 2023-10-06T21:09:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c09a070 con 0x55c6cf619b70 2023-10-06T21:09:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd09a845b80 10 client.15126 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c6cf37fc10 con 0x55c6cf619b70 2023-10-06T21:09:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.004+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06c0b1080 con 0x55c6cf619b70 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd06c099560 con 0x55c6cf619b70 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 20 client.15126 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf62eb10 0x55c6cf62d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 4 client.15126 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6cf564dc0 con 0x55c6cf62eb10 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.005+0000 7fd084d9b700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf62eb10 0x55c6cf62d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.006+0000 7fd084d9b700 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf62eb10 0x55c6cf62d610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2df2179a2a44269 server_cookie=ba6a635e539ce139 in_seq=0 out_seq=0 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.006+0000 7fd05ffff700 10 client.15126 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.006+0000 7fd05ffff700 1 -- 172.21.15.26:0/118584776 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6cf564dc0 con 0x55c6cf62eb10 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.006+0000 7fd05ffff700 10 client.15126 handle_command_reply: tid=0 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 client.15126 shutdown 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 msgr2=0x55c6cf6234c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd06c020000 tx=0x7fd06c06a050 comp rx=0 tx=0).stop 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 shutdown_connections 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf61aa10 0x55c6cf62a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf62eb10 0x55c6cf62d610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf622f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 --2- 172.21.15.26:0/118584776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf6234c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd05ffff700 0 client.15126 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 >> 172.21.15.26:0/118584776 conn(0x55c6cf58d210 msgr2=0x55c6cf613f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 shutdown_connections 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 1 -- 172.21.15.26:0/118584776 wait complete. 2023-10-06T21:09:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.007+0000 7fd09a845b80 20 client.15126 trim_cache size 0 max 16384 2023-10-06T21:09:54.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.025+0000 7fd09a845b80 1 Processor -- start 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.025+0000 7fd09a845b80 1 -- start start 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf620d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619b70 0x55c6cf62a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf3ec8d0 con 0x55c6cf61aa10 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf567ed0 con 0x55c6cf619210 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd09a845b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf5124d0 con 0x55c6cf619b70 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd084d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf620d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49764/0 (socket says 172.21.15.26:49764) 2023-10-06T21:09:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 -- 172.21.15.26:0/2311675841 learned_addr learned my addr 172.21.15.26:0/2311675841 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:54.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd07ffff700 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619b70 0x55c6cf62a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 -- 172.21.15.26:0/2311675841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619b70 msgr2=0x55c6cf62a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619b70 0x55c6cf62a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 -- 172.21.15.26:0/2311675841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 msgr2=0x55c6cf620d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf620d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd08559c700 1 -- 172.21.15.26:0/2311675841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6cf3bd200 con 0x55c6cf61aa10 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.026+0000 7fd084d9b700 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf620d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.027+0000 7fd08559c700 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd078015000 tx=0x7fd078009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6017d1ddf785326 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.027+0000 7fd06b7fe700 1 -- 172.21.15.26:0/2311675841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078025040 con 0x55c6cf61aa10 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.027+0000 7fd06b7fe700 1 -- 172.21.15.26:0/2311675841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07800bd70 con 0x55c6cf61aa10 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.027+0000 7fd06b7fe700 1 -- 172.21.15.26:0/2311675841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800e040 con 0x55c6cf61aa10 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- 172.21.15.26:0/2311675841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 msgr2=0x55c6cf622f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd078015000 tx=0x7fd078009ac0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- 172.21.15.26:0/2311675841 shutdown_connections 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf61aa10 0x55c6cf622f80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf619210 0x55c6cf620d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 --2- 172.21.15.26:0/2311675841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619b70 0x55c6cf62a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- 172.21.15.26:0/2311675841 >> 172.21.15.26:0/2311675841 conn(0x55c6cf58d210 msgr2=0x55c6cf612b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- 172.21.15.26:0/2311675841 shutdown_connections 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- 172.21.15.26:0/2311675841 wait complete. 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 -- start start 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.028+0000 7fd09a845b80 1 Processor -- start 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf624de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf629de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf62a320 con 0x55c6cf619b70 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf62a490 con 0x55c6cf61aa10 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6cf62a650 con 0x55c6cf619210 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd09a845b80 10 client.0 _ll_register_callbacks cb 0x7fd06c099e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd084d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf624de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd084d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55158/0 (socket says 172.21.15.26:55158) 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd084d9b700 1 -- 172.21.15.26:0/200201711 learned_addr learned my addr 172.21.15.26:0/200201711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd08559c700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf629de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 -- 172.21.15.26:0/200201711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 msgr2=0x55c6cf6248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 -- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 msgr2=0x55c6cf629de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf629de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd07ffff700 1 -- 172.21.15.26:0/200201711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6cf396d90 con 0x55c6cf619b70 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd084d9b700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.029+0000 7fd08559c700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf629de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd07ffff700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf624de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0983c0 tx=0x7fd06c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab15ed5face4f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0a2400 con 0x55c6cf619b70 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd06c09a770 con 0x55c6cf619b70 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd074019c50 con 0x55c6cf619b70 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c09a070 con 0x55c6cf619b70 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-10-06T21:09:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd080109c00 con 0x55c6cf619b70 2023-10-06T21:09:54.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06c0c8080 con 0x55c6cf619b70 2023-10-06T21:09:54.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd06c065ce0 con 0x55c6cf619b70 2023-10-06T21:09:54.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.030+0000 7fd09a845b80 20 client.15132 populate_metadata read hostname 'smithi026' 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf686de0 0x55c6cf683340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd09a845b80 4 client.15132 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c6cf564ae0 con 0x55c6cf686de0 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd084d9b700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf686de0 0x55c6cf683340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd084d9b700 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf686de0 0x55c6cf683340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40de09915f15400c server_cookie=c73936d85d9dbf47 in_seq=0 out_seq=0 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.031+0000 7fd05ffff700 10 client.15132 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd05ffff700 1 -- 172.21.15.26:0/200201711 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c6cf564ae0 con 0x55c6cf686de0 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd05ffff700 10 client.15132 handle_command_reply: tid=0 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 client.15132 shutdown 2023-10-06T21:09:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 msgr2=0x55c6cf624de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf624de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0983c0 tx=0x7fd06c005850 comp rx=0 tx=0).stop 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 shutdown_connections 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6cf686de0 0x55c6cf683340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6cf61aa10 0x55c6cf629de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6cf619210 0x55c6cf6248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd05ffff700 0 client.15132 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 --2- 172.21.15.26:0/200201711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6cf619b70 0x55c6cf624de0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 >> 172.21.15.26:0/200201711 conn(0x55c6cf58d210 msgr2=0x55c6cf620d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 shutdown_connections 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 1 -- 172.21.15.26:0/200201711 wait complete. 2023-10-06T21:09:54.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.032+0000 7fd09a845b80 20 client.15132 trim_cache size 0 max 16384 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "4dcc50a6-648b-11ee-8db5-212e2dc638e7", 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24587, 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 97.502202268999994, 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 23, 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2023-10-06T21:09:54.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 85, 2023-10-06T21:09:54.042 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 103.90473285 2023-10-06T21:09:54.042 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:09:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 msgr2=0x7fd0700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 0x7fd0700a39c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c10 tx=0x7fd0780096c0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 msgr2=0x7fd0800ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9c0 tx=0x7fd06c00ecd0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 shutdown_connections 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd080106c00 0x7fd0800ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0801019a0 0x7fd080101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd0700a1500 0x7fd0700a39c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 --2- 172.21.15.26:0/4018639585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0801042d0 0x7fd0800ff950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 >> 172.21.15.26:0/4018639585 conn(0x7fd0800fb330 msgr2=0x7fd080102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 shutdown_connections 2023-10-06T21:09:54.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:54.033+0000 7fd086fff700 1 -- 172.21.15.26:0/4018639585 wait complete. 2023-10-06T21:09:54.053 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T21:09:54.078 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T21:09:54.078 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-06T21:09:54.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:54.078 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-10-06T21:09:54.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:54.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-10-06T21:09:54.235 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-10-06T21:09:54.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:09:54.238 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-10-06T21:09:54.239 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-06T21:09:54.239 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-10-06T21:09:54.239 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-10-06T21:09:54.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-06T21:09:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:09:55.763 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:55 smithi026 ceph-mon[98490]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:09:55.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:55 smithi167 ceph-mon[134494]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:09:55.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:55 smithi078 ceph-mon[134896]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 -- 172.21.15.26:0/4193608992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 msgr2=0x7f01b010d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 --2- 172.21.15.26:0/4193608992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b010d3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098f0 tx=0x7f01a0009c00 comp rx=0 tx=0).stop 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 -- 172.21.15.26:0/4193608992 shutdown_connections 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 --2- 172.21.15.26:0/4193608992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b010fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 --2- 172.21.15.26:0/4193608992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 0x7f01b00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 --2- 172.21.15.26:0/4193608992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b010d3f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 -- 172.21.15.26:0/4193608992 >> 172.21.15.26:0/4193608992 conn(0x7f01b00780a0 msgr2=0x7f01b00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 -- 172.21.15.26:0/4193608992 shutdown_connections 2023-10-06T21:09:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.384+0000 7f01b8113700 1 -- 172.21.15.26:0/4193608992 wait complete. 2023-10-06T21:09:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.385+0000 7f01b8113700 1 Processor -- start 2023-10-06T21:09:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 -- start start 2023-10-06T21:09:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 0x7f01b0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b019e250 con 0x7f01b0073a00 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b019e3c0 con 0x7f01b010d930 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.386+0000 7f01b8113700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b019e580 con 0x7f01b00730e0 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49790/0 (socket says 172.21.15.26:49790) 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b5eaf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 0x7f01b0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 -- 172.21.15.26:0/3157796031 learned_addr learned my addr 172.21.15.26:0/3157796031 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b66b0700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 -- 172.21.15.26:0/3157796031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 msgr2=0x7f01b0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 0x7f01b0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 -- 172.21.15.26:0/3157796031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 msgr2=0x7f01b019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:56.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 -- 172.21.15.26:0/3157796031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0095f0 con 0x7f01b0073a00 2023-10-06T21:09:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b66b0700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:09:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.387+0000 7f01b56ae700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009810 tx=0x7f01a000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.388+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a001c070 con 0x7f01b0073a00 2023-10-06T21:09:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.388+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0009520 con 0x7f01b0073a00 2023-10-06T21:09:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.388+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01a000bb20 con 0x7f01b0073a00 2023-10-06T21:09:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.388+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0004930 con 0x7f01b0073a00 2023-10-06T21:09:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.388+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b01a5ef0 con 0x7f01b0073a00 2023-10-06T21:09:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.389+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b004ebb0 con 0x7f01b0073a00 2023-10-06T21:09:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.391+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f01a0020070 con 0x7f01b0073a00 2023-10-06T21:09:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.392+0000 7f01a6ffd700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 0x7f019c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:09:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.393+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01a00cd970 con 0x7f01b0073a00 2023-10-06T21:09:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.393+0000 7f01b5eaf700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 0x7f019c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:09:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.393+0000 7f01b5eaf700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 0x7f019c0a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:09:56.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.396+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f01a0097020 con 0x7f01b0073a00 2023-10-06T21:09:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.653+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01b019a5f0 con 0x7f01b0073a00 2023-10-06T21:09:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.653+0000 7f01a6ffd700 1 -- 172.21.15.26:0/3157796031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f01a009a640 con 0x7f01b0073a00 2023-10-06T21:09:56.657 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T21:09:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.655+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 msgr2=0x7f019c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.655+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 0x7f019c0a39c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac009310 comp rx=0 tx=0).stop 2023-10-06T21:09:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.655+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 msgr2=0x7f01b01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:09:56.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.655+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009810 tx=0x7f01a000fa00 comp rx=0 tx=0).stop 2023-10-06T21:09:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 shutdown_connections 2023-10-06T21:09:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01b010d930 0x7f01b019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01b00730e0 0x7f01b0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f019c0a1500 0x7f019c0a39c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 --2- 172.21.15.26:0/3157796031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b0073a00 0x7f01b01997d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:09:56.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 >> 172.21.15.26:0/3157796031 conn(0x7f01b00780a0 msgr2=0x7f01b00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:09:56.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 shutdown_connections 2023-10-06T21:09:56.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:09:56.656+0000 7f01b8113700 1 -- 172.21.15.26:0/3157796031 wait complete. 2023-10-06T21:09:57.256 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T21:09:57.257 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-10-06T21:09:57.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:57.258 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-10-06T21:09:57.302 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T21:09:57.302 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T21:09:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T21:09:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:09:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-10-06T21:09:57.303 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:09:57.303 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-10-06T21:09:57.304 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41620sec preferred_lft 41620sec 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-10-06T21:09:57.305 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-10-06T21:09:57.306 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T21:09:57.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:57.306 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T21:09:57.307 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-10-06T21:09:57.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:57 smithi026 ceph-mon[98490]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:57.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:57 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3157796031' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:09:57.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:57 smithi167 ceph-mon[134494]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:57 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3157796031' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:09:57.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:57.813 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T21:09:57.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:57 smithi078 ceph-mon[134896]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-10-06T21:09:57.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:57 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3157796031' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-06T21:09:57.885 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-10-06T21:09:57.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:57.888 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-10-06T21:09:57.956 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-10-06T21:09:57.956 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T21:09:57.956 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T21:09:57.956 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T21:09:57.956 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T21:09:57.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T21:09:57.957 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-10-06T21:09:58.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:58.711 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-10-06T21:09:58.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:58.757 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2023-10-06T21:09:58.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-10-06T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-10-06T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--tmp-cephfs 2023-10-06T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-10-06T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-10-06T21:09:58.986 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:09:58 smithi026 ceph-mon[98490]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:09:59.030 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-10-06T21:09:59.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-10-06T21:09:59.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-10-06T21:09:59.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-10-06T21:09:59.031 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-10-06T21:09:59.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:09:58 smithi078 ceph-mon[134896]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:09:59.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:09:58 smithi167 ceph-mon[134494]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:09:59.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:59.315 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-10-06T21:09:59.315 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-10-06T21:09:59.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-10-06T21:09:59.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T21:09:59.315 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-10-06T21:09:59.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T21:09:59.493 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /tmp/cephfs 2023-10-06T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/tmp/cephfs' 2023-10-06T21:09:59.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T21:09:59.539 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /tmp/cephfs 2023-10-06T21:09:59.622 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-06T21:09:59.623 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-06T21:09:59.623 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/68c2ada3-e9f1-4f5f-952a-1c21257dff12 2023-10-06T21:09:59.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:09:59.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/68c2ada3-e9f1-4f5f-952a-1c21257dff12 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-10-06T21:09:59.783 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-06T21:09:59.783 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-06T21:10:00.229 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:00 smithi026 ceph-mon[98490]: overall HEALTH_OK 2023-10-06T21:10:00.261 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-06T21:10:00.261 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-06T21:10:00.261 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.78:6789,172.21.15.167:6789" 2023-10-06T21:10:00.261 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@4dcc50a6-648b-11ee-8db5-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/68c2ada3-e9f1-4f5f-952a-1c21257dff12 2023-10-06T21:10:00.261 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.26:6789/172.21.15.78:6789/172.21.15.167:6789" will pass to kernel 2023-10-06T21:10:00.268 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T21:10:00.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:10:00.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /tmp/cephfs 2023-10-06T21:10:00.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:10:00.342 DEBUG:teuthology.orchestra.run.smithi026:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T21:10:00.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:00 smithi078 ceph-mon[134896]: overall HEALTH_OK 2023-10-06T21:10:00.453 INFO:teuthology.orchestra.run.smithi026.stdout:client25199 2023-10-06T21:10:00.454 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:10:00.454 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:10:00.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:10:00.499 DEBUG:teuthology.orchestra.run.smithi026:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T21:10:00.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:00 smithi167 ceph-mon[134494]: overall HEALTH_OK 2023-10-06T21:10:00.627 INFO:teuthology.orchestra.run.smithi026.stdout:client25199 2023-10-06T21:10:00.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:10:00.627 DEBUG:teuthology.orchestra.run.smithi026:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-06T21:10:00.713 INFO:teuthology.orchestra.run.smithi026.stdout:client25199 2023-10-06T21:10:00.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:10:00.714 DEBUG:teuthology.orchestra.run.smithi026:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7.client25199/status') 2023-10-06T21:10:00.822 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.25199 (3)172.21.15.26:0/467980806 2023-10-06T21:10:00.822 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2023-10-06T21:10:00.823 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2023-10-06T21:10:00.823 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2023-10-06T21:10:00.823 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 5.5841e-05 s, 1.3 MB/s 2023-10-06T21:10:00.826 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-06T21:10:00.836 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-06T21:10:00.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:10:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 -- 172.21.15.26:0/1315215267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 msgr2=0x7f4cdc1066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/1315215267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc1066c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009990 tx=0x7f4cd0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 -- 172.21.15.26:0/1315215267 shutdown_connections 2023-10-06T21:10:01.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/1315215267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 0x7f4cdc109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/1315215267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/1315215267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc1066c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 -- 172.21.15.26:0/1315215267 >> 172.21.15.26:0/1315215267 conn(0x7f4cdc0fb3b0 msgr2=0x7f4cdc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 -- 172.21.15.26:0/1315215267 shutdown_connections 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.065+0000 7f4ce19ff700 1 -- 172.21.15.26:0/1315215267 wait complete. 2023-10-06T21:10:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.066+0000 7f4ce19ff700 1 Processor -- start 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.066+0000 7f4ce19ff700 1 -- start start 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 0x7f4cdc19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19e6c0 con 0x7f4cdc1042d0 2023-10-06T21:10:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19e800 con 0x7f4cdc1019a0 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4ce19ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19e990 con 0x7f4cdc106c00 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51830/0 (socket says 172.21.15.26:51830) 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 -- 172.21.15.26:0/2814375442 learned_addr learned my addr 172.21.15.26:0/2814375442 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdb7fe700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 0x7f4cdc19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 -- 172.21.15.26:0/2814375442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 msgr2=0x7f4cdc19e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 0x7f4cdc19e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 -- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 msgr2=0x7f4cdc197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.067+0000 7f4cdaffd700 1 -- 172.21.15.26:0/2814375442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00095f0 con 0x7f4cdc1019a0 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4cdaffd700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e9a0 tx=0x7f4ccc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc009790 con 0x7f4cdc1019a0 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ccc010430 con 0x7f4cdc1019a0 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc19ec70 con 0x7f4cdc1019a0 2023-10-06T21:10:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc018680 con 0x7f4cdc1019a0 2023-10-06T21:10:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.068+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc19f1d0 con 0x7f4cdc1019a0 2023-10-06T21:10:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.069+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc0fcf30 con 0x7f4cdc1019a0 2023-10-06T21:10:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.071+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4ccc0105a0 con 0x7f4cdc1019a0 2023-10-06T21:10:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.072+0000 7f4ce09fd700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 0x7f4cc40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.072+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4ccc014070 con 0x7f4cdc1019a0 2023-10-06T21:10:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.073+0000 7f4cda7fc700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 0x7f4cc40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.073+0000 7f4cda7fc700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 0x7f4cc40a3a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b2e0 tx=0x7f4cd0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.075+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ccc096520 con 0x7f4cdc1019a0 2023-10-06T21:10:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.290+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cdc02d1c0 con 0x7f4cdc1019a0 2023-10-06T21:10:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.292+0000 7f4ce09fd700 1 -- 172.21.15.26:0/2814375442 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f4ccc099b40 con 0x7f4cdc1019a0 2023-10-06T21:10:01.295 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:01.296 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 msgr2=0x7f4cc40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 0x7f4cc40a3a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b2e0 tx=0x7f4cd0005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 msgr2=0x7f4cdc196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e9a0 tx=0x7f4ccc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 shutdown_connections 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc106c00 0x7f4cdc19e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cdc1019a0 0x7f4cdc196b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4cc40a1550 0x7f4cc40a3a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 --2- 172.21.15.26:0/2814375442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cdc1042d0 0x7f4cdc197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 >> 172.21.15.26:0/2814375442 conn(0x7f4cdc0fb3b0 msgr2=0x7f4cdc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 shutdown_connections 2023-10-06T21:10:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.295+0000 7f4ce19ff700 1 -- 172.21.15.26:0/2814375442 wait complete. 2023-10-06T21:10:01.314 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:01 smithi026 ceph-mon[98490]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-06T21:10:01.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:10:01.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:10:01.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:01 smithi078 ceph-mon[134896]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-06T21:10:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 -- 172.21.15.26:0/4186429643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8100e90 msgr2=0x7f58a81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/4186429643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8100e90 0x7f58a81012b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5898009990 tx=0x7f5898009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 -- 172.21.15.26:0/4186429643 shutdown_connections 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/4186429643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 0x7f58a8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/4186429643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8100e90 0x7f58a81012b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/4186429643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8102070 0x7f58a81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 -- 172.21.15.26:0/4186429643 >> 172.21.15.26:0/4186429643 conn(0x7f58a80fc430 msgr2=0x7f58a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 -- 172.21.15.26:0/4186429643 shutdown_connections 2023-10-06T21:10:01.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.521+0000 7f58ad1ff700 1 -- 172.21.15.26:0/4186429643 wait complete. 2023-10-06T21:10:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.522+0000 7f58ad1ff700 1 Processor -- start 2023-10-06T21:10:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.522+0000 7f58ad1ff700 1 -- start start 2023-10-06T21:10:01.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.522+0000 7f58ad1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8100e90 0x7f58a8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58ad1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58ad1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 0x7f58a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58ad1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8073f80 con 0x7f58a8102070 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58ad1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80740f0 con 0x7f58a8102a30 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58ad1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a81a0c90 con 0x7f58a8100e90 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38812/0 (socket says 172.21.15.26:38812) 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 -- 172.21.15.26:0/1419667564 learned_addr learned my addr 172.21.15.26:0/1419667564 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a759e700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 0x7f58a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a6d9d700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8100e90 0x7f58a8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 -- 172.21.15.26:0/1419667564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8100e90 msgr2=0x7f58a8074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8100e90 0x7f58a8074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 -- 172.21.15.26:0/1419667564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 msgr2=0x7f58a80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 0x7f58a80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.523+0000 7f58a659c700 1 -- 172.21.15.26:0/1419667564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980095f0 con 0x7f58a8102070 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f58a659c700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e970 tx=0x7f589c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c009790 con 0x7f58a8102070 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a81a0f70 con 0x7f58a8102070 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c00c7e0 con 0x7f58a8102070 2023-10-06T21:10:01.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c003c70 con 0x7f58a8102070 2023-10-06T21:10:01.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.524+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a81a1440 con 0x7f58a8102070 2023-10-06T21:10:01.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.525+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8107e90 con 0x7f58a8102070 2023-10-06T21:10:01.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.527+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f589c00cc50 con 0x7f58a8102070 2023-10-06T21:10:01.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.528+0000 7f588ffff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 0x7f58900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.528+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f589c014070 con 0x7f58a8102070 2023-10-06T21:10:01.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.529+0000 7f58a6d9d700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 0x7f58900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.529+0000 7f58a6d9d700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 0x7f58900a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f589800b2e0 tx=0x7f5898005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:01 smithi167 ceph-mon[134494]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-06T21:10:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.533+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f589c096510 con 0x7f58a8102070 2023-10-06T21:10:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.743+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58a802d270 con 0x7f58a8102070 2023-10-06T21:10:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.743+0000 7f588ffff700 1 -- 172.21.15.26:0/1419667564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f589c003dd0 con 0x7f58a8102070 2023-10-06T21:10:01.746 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:10:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 msgr2=0x7f58900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 0x7f58900a3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f589800b2e0 tx=0x7f5898005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 msgr2=0x7f58a80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e970 tx=0x7f589c00ec80 comp rx=0 tx=0).stop 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 shutdown_connections 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58a8102a30 0x7f58a80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58a8100e90 0x7f58a8074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f58900a1480 0x7f58900a3940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 --2- 172.21.15.26:0/1419667564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58a8102070 0x7f58a80732a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 >> 172.21.15.26:0/1419667564 conn(0x7f58a80fc430 msgr2=0x7f58a80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 shutdown_connections 2023-10-06T21:10:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.746+0000 7f58ad1ff700 1 -- 172.21.15.26:0/1419667564 wait complete. 2023-10-06T21:10:01.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:10:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.972+0000 7fd8203ff700 1 -- 172.21.15.26:0/1811325785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 msgr2=0x7fd818109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.972+0000 7fd8203ff700 1 --2- 172.21.15.26:0/1811325785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd818109010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b0a0 tx=0x7fd81000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 -- 172.21.15.26:0/1811325785 shutdown_connections 2023-10-06T21:10:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 --2- 172.21.15.26:0/1811325785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd818109010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 --2- 172.21.15.26:0/1811325785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818101940 0x7fd818103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 --2- 172.21.15.26:0/1811325785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818104270 0x7fd818106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 -- 172.21.15.26:0/1811325785 >> 172.21.15.26:0/1811325785 conn(0x7fd8180fb330 msgr2=0x7fd8180fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 -- 172.21.15.26:0/1811325785 shutdown_connections 2023-10-06T21:10:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.973+0000 7fd8203ff700 1 -- 172.21.15.26:0/1811325785 wait complete. 2023-10-06T21:10:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 Processor -- start 2023-10-06T21:10:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 -- start start 2023-10-06T21:10:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101940 0x7fd818198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 0x7fd8181993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a0a30 con 0x7fd818106ba0 2023-10-06T21:10:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a0b70 con 0x7fd818104270 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd8203ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8181a0d00 con 0x7fd818101940 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd81e99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd81e99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38838/0 (socket says 172.21.15.26:38838) 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.974+0000 7fd81e99c700 1 -- 172.21.15.26:0/4002664367 learned_addr learned my addr 172.21.15.26:0/4002664367 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81d99a700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 0x7fd8181993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e19b700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101940 0x7fd818198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 -- 172.21.15.26:0/4002664367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101940 msgr2=0x7fd818198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101940 0x7fd818198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 -- 172.21.15.26:0/4002664367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 msgr2=0x7fd8181993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 0x7fd8181993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 -- 172.21.15.26:0/4002664367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009520 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81d99a700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 0x7fd8181993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd81e99c700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b780 tx=0x7fd81000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.975+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000e040 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.976+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd810009d00 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.976+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd810004c50 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.976+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810005280 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.976+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8181a1210 con 0x7fd818106ba0 2023-10-06T21:10:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.977+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd818108720 con 0x7fd818106ba0 2023-10-06T21:10:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.979+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd81001d470 con 0x7fd818106ba0 2023-10-06T21:10:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.980+0000 7fd80f7fe700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 0x7fd8040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.980+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8100ce1d0 con 0x7fd818106ba0 2023-10-06T21:10:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.980+0000 7fd81e19b700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 0x7fd8040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.981+0000 7fd81e19b700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 0x7fd8040a3940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd814009df0 tx=0x7fd814009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:01.984+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd8100978c0 con 0x7fd818106ba0 2023-10-06T21:10:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.194+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd81802d270 con 0x7fd818106ba0 2023-10-06T21:10:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.195+0000 7fd80f7fe700 1 -- 172.21.15.26:0/4002664367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd81009aee0 con 0x7fd818106ba0 2023-10-06T21:10:02.198 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 msgr2=0x7fd8040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 0x7fd8040a3940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd814009df0 tx=0x7fd814009260 comp rx=0 tx=0).stop 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 msgr2=0x7fd8181a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b780 tx=0x7fd81000bab0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 shutdown_connections 2023-10-06T21:10:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8040a1480 0x7fd8040a3940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd818106ba0 0x7fd8181a0460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101940 0x7fd818198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 --2- 172.21.15.26:0/4002664367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd818104270 0x7fd8181993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 >> 172.21.15.26:0/4002664367 conn(0x7fd8180fb330 msgr2=0x7fd818100040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 shutdown_connections 2023-10-06T21:10:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.197+0000 7fd8203ff700 1 -- 172.21.15.26:0/4002664367 wait complete. 2023-10-06T21:10:02.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:10:02.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:02 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2814375442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:02.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:02 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1419667564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:02.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:02 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2814375442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:02.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:02 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1419667564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:02.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 -- 172.21.15.26:0/1979523856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50100e90 msgr2=0x7f8f501012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1979523856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50100e90 0x7f8f501012b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009990 tx=0x7f8f44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 -- 172.21.15.26:0/1979523856 shutdown_connections 2023-10-06T21:10:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1979523856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 0x7f8f50109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1979523856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50100e90 0x7f8f501012b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1979523856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50102070 0x7f8f501024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 -- 172.21.15.26:0/1979523856 >> 172.21.15.26:0/1979523856 conn(0x7f8f500fc430 msgr2=0x7f8f500fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 -- 172.21.15.26:0/1979523856 shutdown_connections 2023-10-06T21:10:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.423+0000 7f8f56bff700 1 -- 172.21.15.26:0/1979523856 wait complete. 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.424+0000 7f8f56bff700 1 Processor -- start 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.424+0000 7f8f56bff700 1 -- start start 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 0x7f8f50074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 0x7f8f500737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50073f80 con 0x7f8f50102070 2023-10-06T21:10:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500740f0 con 0x7f8f50102a30 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f56bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f501a0c90 con 0x7f8f50100e90 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f5499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 0x7f8f50074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f4ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38852/0 (socket says 172.21.15.26:38852) 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f4ffff700 1 -- 172.21.15.26:0/1490302155 learned_addr learned my addr 172.21.15.26:0/1490302155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.425+0000 7f8f5519c700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 0x7f8f500737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 -- 172.21.15.26:0/1490302155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 msgr2=0x7f8f50074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 0x7f8f50074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 -- 172.21.15.26:0/1490302155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 msgr2=0x7f8f500737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 0x7f8f500737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 -- 172.21.15.26:0/1490302155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440095f0 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f5499b700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 0x7f8f50074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4ffff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d980 tx=0x7f8f3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c004020 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f3c00f690 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f501a0f70 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003a60 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.426+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f501a13f0 con 0x7f8f50102070 2023-10-06T21:10:02.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.427+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f50107e90 con 0x7f8f50102070 2023-10-06T21:10:02.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.429+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8f3c00fb00 con 0x7f8f50102070 2023-10-06T21:10:02.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.431+0000 7f8f4dffb700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 0x7f8f400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.431+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8f3c0cc8d0 con 0x7f8f50102070 2023-10-06T21:10:02.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.431+0000 7f8f5499b700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 0x7f8f400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.432+0000 7f8f5499b700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 0x7f8f400a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b2e0 tx=0x7f8f44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:02.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.435+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8f3c095ef0 con 0x7f8f50102070 2023-10-06T21:10:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:02 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2814375442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:02.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:02 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1419667564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.645+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f5002d270 con 0x7f8f50102070 2023-10-06T21:10:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.646+0000 7f8f4dffb700 1 -- 172.21.15.26:0/1490302155 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f3c099510 con 0x7f8f50102070 2023-10-06T21:10:02.648 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 msgr2=0x7f8f400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 0x7f8f400a3940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b2e0 tx=0x7f8f44005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 msgr2=0x7f8f500732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d980 tx=0x7f8f3c00dc90 comp rx=0 tx=0).stop 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 shutdown_connections 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f50102a30 0x7f8f500737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f50100e90 0x7f8f50074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f8f400a1480 0x7f8f400a3940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 --2- 172.21.15.26:0/1490302155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f50102070 0x7f8f500732a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 >> 172.21.15.26:0/1490302155 conn(0x7f8f500fc430 msgr2=0x7f8f500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 shutdown_connections 2023-10-06T21:10:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.648+0000 7f8f56bff700 1 -- 172.21.15.26:0/1490302155 wait complete. 2023-10-06T21:10:02.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:10:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 -- 172.21.15.26:0/3556274460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324073910 msgr2=0x7f6324073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 --2- 172.21.15.26:0/3556274460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324073910 0x7f6324073d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6318009990 tx=0x7f6318009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 -- 172.21.15.26:0/3556274460 shutdown_connections 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 --2- 172.21.15.26:0/3556274460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324108d30 0x7f632410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 --2- 172.21.15.26:0/3556274460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324074ef0 0x7f6324073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 --2- 172.21.15.26:0/3556274460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324073910 0x7f6324073d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.872+0000 7f6328bff700 1 -- 172.21.15.26:0/3556274460 >> 172.21.15.26:0/3556274460 conn(0x7f63240fc3c0 msgr2=0x7f63240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.873+0000 7f6328bff700 1 -- 172.21.15.26:0/3556274460 shutdown_connections 2023-10-06T21:10:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.873+0000 7f6328bff700 1 -- 172.21.15.26:0/3556274460 wait complete. 2023-10-06T21:10:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.873+0000 7f6328bff700 1 Processor -- start 2023-10-06T21:10:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 -- start start 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 0x7f6324198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324108d30 0x7f63241a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a08f0 con 0x7f6324074ef0 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0a60 con 0x7f6324108d30 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6328bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0c20 con 0x7f6324073910 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6321d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f632259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 0x7f6324198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6321d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38872/0 (socket says 172.21.15.26:38872) 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6321d9b700 1 -- 172.21.15.26:0/1778520129 learned_addr learned my addr 172.21.15.26:0/1778520129 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:02.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.874+0000 7f6322d9d700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324108d30 0x7f63241a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 -- 172.21.15.26:0/1778520129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 msgr2=0x7f6324198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 0x7f6324198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 -- 172.21.15.26:0/1778520129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324108d30 msgr2=0x7f63241a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324108d30 0x7f63241a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 -- 172.21.15.26:0/1778520129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c009c40 con 0x7f6324074ef0 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f632259c700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 0x7f6324198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6321d9b700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6318005720 tx=0x7f6318009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631801d070 con 0x7f6324074ef0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6318021410 con 0x7f6324074ef0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63180095f0 con 0x7f6324074ef0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.875+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318017650 con 0x7f6324074ef0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.876+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63241a1130 con 0x7f6324074ef0 2023-10-06T21:10:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.877+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63241073e0 con 0x7f6324074ef0 2023-10-06T21:10:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.878+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6318021580 con 0x7f6324074ef0 2023-10-06T21:10:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.880+0000 7f63137fe700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 0x7f630c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.880+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63180cdf00 con 0x7f6324074ef0 2023-10-06T21:10:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.880+0000 7f632259c700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 0x7f630c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.881+0000 7f632259c700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 0x7f630c0a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f632419a280 tx=0x7f6314009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:02.884+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f63180975b0 con 0x7f6324074ef0 2023-10-06T21:10:03.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.101+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f632410e8b0 con 0x7f6324074ef0 2023-10-06T21:10:03.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.103+0000 7f63137fe700 1 -- 172.21.15.26:0/1778520129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f631809abd0 con 0x7f6324074ef0 2023-10-06T21:10:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:03.106 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 msgr2=0x7f630c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 0x7f630c0a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f632419a280 tx=0x7f6314009260 comp rx=0 tx=0).stop 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 msgr2=0x7f63241992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6318005720 tx=0x7f6318009580 comp rx=0 tx=0).stop 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 shutdown_connections 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6324108d30 0x7f63241a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324073910 0x7f6324198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f630c0a14d0 0x7f630c0a3990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 --2- 172.21.15.26:0/1778520129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6324074ef0 0x7f63241992f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 >> 172.21.15.26:0/1778520129 conn(0x7f63240fc3c0 msgr2=0x7f63240fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 shutdown_connections 2023-10-06T21:10:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.105+0000 7f6328bff700 1 -- 172.21.15.26:0/1778520129 wait complete. 2023-10-06T21:10:03.127 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:10:03.128 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:10:03.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:03 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4002664367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:03 smithi026 ceph-mon[98490]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:10:03.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:03 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1490302155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 -- 172.21.15.26:0/3798999009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 msgr2=0x7fd8d01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/3798999009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01024f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009990 tx=0x7fd8c0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 -- 172.21.15.26:0/3798999009 shutdown_connections 2023-10-06T21:10:03.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/3798999009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/3798999009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 0x7fd8d01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/3798999009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01024f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 -- 172.21.15.26:0/3798999009 >> 172.21.15.26:0/3798999009 conn(0x7fd8d00fc430 msgr2=0x7fd8d00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:03.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 -- 172.21.15.26:0/3798999009 shutdown_connections 2023-10-06T21:10:03.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.331+0000 7fd8d87ff700 1 -- 172.21.15.26:0/3798999009 wait complete. 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.332+0000 7fd8d87ff700 1 Processor -- start 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 -- start start 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 0x7fd8d0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a09f0 con 0x7fd8d0102070 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a0b30 con 0x7fd8d0100e90 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d87ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a0cc0 con 0x7fd8d0102a30 2023-10-06T21:10:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d5d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d5d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38904/0 (socket says 172.21.15.26:38904) 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d5d9a700 1 -- 172.21.15.26:0/315235720 learned_addr learned my addr 172.21.15.26:0/315235720 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.333+0000 7fd8d659b700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 0x7fd8d0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d6d9c700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 -- 172.21.15.26:0/315235720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 msgr2=0x7fd8d01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 -- 172.21.15.26:0/315235720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 msgr2=0x7fd8d0198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 0x7fd8d0198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 -- 172.21.15.26:0/315235720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00095f0 con 0x7fd8d0102070 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d6d9c700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d5d9a700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0005720 tx=0x7fd8c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:03.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c001d070 con 0x7fd8d0102070 2023-10-06T21:10:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c0021410 con 0x7fd8d0102070 2023-10-06T21:10:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d01a0f40 con 0x7fd8d0102070 2023-10-06T21:10:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0017650 con 0x7fd8d0102070 2023-10-06T21:10:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.334+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01a1300 con 0x7fd8d0102070 2023-10-06T21:10:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.335+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d010c750 con 0x7fd8d0102070 2023-10-06T21:10:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.337+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd8c0021580 con 0x7fd8d0102070 2023-10-06T21:10:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.339+0000 7fd8c77fe700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 0x7fd8bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.339+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8c00ce380 con 0x7fd8d0102070 2023-10-06T21:10:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.339+0000 7fd8d659b700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 0x7fd8bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.340+0000 7fd8d659b700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 0x7fd8bc0a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0101be0 tx=0x7fd8cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:03.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.341+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd8c0097a30 con 0x7fd8d0102070 2023-10-06T21:10:03.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:03 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4002664367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:03 smithi078 ceph-mon[134896]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:10:03.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:03 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1490302155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:03 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4002664367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:03 smithi167 ceph-mon[134494]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:10:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:03 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1490302155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.549+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8d01a1490 con 0x7fd8d0102070 2023-10-06T21:10:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.549+0000 7fd8c77fe700 1 -- 172.21.15.26:0/315235720 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8c0026020 con 0x7fd8d0102070 2023-10-06T21:10:03.552 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 msgr2=0x7fd8bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 0x7fd8bc0a39c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0101be0 tx=0x7fd8cc009260 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 msgr2=0x7fd8d01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0005720 tx=0x7fd8c0009580 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 shutdown_connections 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8d0102a30 0x7fd8d01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d0100e90 0x7fd8d0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fd8bc0a1500 0x7fd8bc0a39c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 --2- 172.21.15.26:0/315235720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d0102070 0x7fd8d01993b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 >> 172.21.15.26:0/315235720 conn(0x7fd8d00fc430 msgr2=0x7fd8d00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 shutdown_connections 2023-10-06T21:10:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.552+0000 7fd8d87ff700 1 -- 172.21.15.26:0/315235720 wait complete. 2023-10-06T21:10:03.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 -- 172.21.15.26:0/903063012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf01042b0 msgr2=0x7f7cf01066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/903063012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf01042b0 0x7f7cf01066a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009990 tx=0x7f7ce4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 -- 172.21.15.26:0/903063012 shutdown_connections 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/903063012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0106be0 0x7f7cf0109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/903063012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf0101980 0x7f7cf0103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/903063012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf01042b0 0x7f7cf01066a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 -- 172.21.15.26:0/903063012 >> 172.21.15.26:0/903063012 conn(0x7f7cf00fb370 msgr2=0x7f7cf00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 -- 172.21.15.26:0/903063012 shutdown_connections 2023-10-06T21:10:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.778+0000 7f7cf4bff700 1 -- 172.21.15.26:0/903063012 wait complete. 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.779+0000 7f7cf4bff700 1 Processor -- start 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 -- start start 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0101980 0x7f7cf0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf01042b0 0x7f7cf0197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019e760 con 0x7f7cf0106be0 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019e8d0 con 0x7f7cf0101980 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cf4bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf019ea90 con 0x7f7cf01042b0 2023-10-06T21:10:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7ceed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7ceed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38922/0 (socket says 172.21.15.26:38922) 2023-10-06T21:10:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7ceed9d700 1 -- 172.21.15.26:0/2588701062 learned_addr learned my addr 172.21.15.26:0/2588701062 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cee59c700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0101980 0x7f7cf0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.780+0000 7f7cedd9b700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf01042b0 0x7f7cf0197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 -- 172.21.15.26:0/2588701062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf01042b0 msgr2=0x7f7cf0197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf01042b0 0x7f7cf0197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 -- 172.21.15.26:0/2588701062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0101980 msgr2=0x7f7cf0196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0101980 0x7f7cf0196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 -- 172.21.15.26:0/2588701062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40095f0 con 0x7f7cf0106be0 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7ceed9d700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ef30 tx=0x7f7ce800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8018070 con 0x7f7cf0106be0 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf019ed10 con 0x7f7cf0106be0 2023-10-06T21:10:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ce800f040 con 0x7f7cf0106be0 2023-10-06T21:10:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800cd00 con 0x7f7cf0106be0 2023-10-06T21:10:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.781+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf019f1d0 con 0x7f7cf0106be0 2023-10-06T21:10:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.782+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf010c740 con 0x7f7cf0106be0 2023-10-06T21:10:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.785+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7ce8003a00 con 0x7f7cf0106be0 2023-10-06T21:10:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.786+0000 7f7cdf7fe700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 0x7f7cd80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.786+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ce80cd1b0 con 0x7f7cf0106be0 2023-10-06T21:10:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.786+0000 7f7cee59c700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 0x7f7cd80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.787+0000 7f7cee59c700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 0x7f7cd80a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7cf01980f0 tx=0x7f7ce000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:03.788+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7ce80968a0 con 0x7f7cf0106be0 2023-10-06T21:10:04.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.000+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cf010c900 con 0x7f7cf0106be0 2023-10-06T21:10:04.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.001+0000 7f7cdf7fe700 1 -- 172.21.15.26:0/2588701062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ce8096070 con 0x7f7cf0106be0 2023-10-06T21:10:04.004 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:10:04.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.003+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 msgr2=0x7f7cd80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.003+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 0x7f7cd80a3940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7cf01980f0 tx=0x7f7ce000b410 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.003+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 msgr2=0x7f7cf019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.003+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ef30 tx=0x7f7ce800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 shutdown_connections 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f7cd80a1480 0x7f7cd80a3940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf0106be0 0x7f7cf019e190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf0101980 0x7f7cf0196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 --2- 172.21.15.26:0/2588701062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cf01042b0 0x7f7cf0197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 >> 172.21.15.26:0/2588701062 conn(0x7f7cf00fb370 msgr2=0x7f7cf0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 shutdown_connections 2023-10-06T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.004+0000 7f7cf4bff700 1 -- 172.21.15.26:0/2588701062 wait complete. 2023-10-06T21:10:04.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:10:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 -- 172.21.15.26:0/2176305749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618102070 msgr2=0x7f76181024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 --2- 172.21.15.26:0/2176305749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618102070 0x7f76181024f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7600009990 tx=0x7f7600009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 -- 172.21.15.26:0/2176305749 shutdown_connections 2023-10-06T21:10:04.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 --2- 172.21.15.26:0/2176305749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 0x7f7618109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 --2- 172.21.15.26:0/2176305749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618100e90 0x7f76181012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 --2- 172.21.15.26:0/2176305749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618102070 0x7f76181024f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 -- 172.21.15.26:0/2176305749 >> 172.21.15.26:0/2176305749 conn(0x7f76180fc430 msgr2=0x7f76180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 -- 172.21.15.26:0/2176305749 shutdown_connections 2023-10-06T21:10:04.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.230+0000 7f761cdff700 1 -- 172.21.15.26:0/2176305749 wait complete. 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.231+0000 7f761cdff700 1 Processor -- start 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.231+0000 7f761cdff700 1 -- start start 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618102070 0x7f7618197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 0x7f761819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761819e770 con 0x7f7618100e90 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761819e8e0 con 0x7f7618102070 2023-10-06T21:10:04.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761cdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761819eaa0 con 0x7f7618102a30 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38938/0 (socket says 172.21.15.26:38938) 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 -- 172.21.15.26:0/2960973772 learned_addr learned my addr 172.21.15.26:0/2960973772 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f7616d9d700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 0x7f761819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f761659c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618102070 0x7f7618197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 -- 172.21.15.26:0/2960973772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 msgr2=0x7f761819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 0x7f761819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 -- 172.21.15.26:0/2960973772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618102070 msgr2=0x7f7618197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618102070 0x7f7618197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.232+0000 7f760e59c700 1 -- 172.21.15.26:0/2960973772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000095f0 con 0x7f7618100e90 2023-10-06T21:10:04.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f760e59c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f760800e970 tx=0x7f760800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608009790 con 0x7f7618100e90 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761819ed80 con 0x7f7618100e90 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f760800c7e0 con 0x7f7618100e90 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608003c70 con 0x7f7618100e90 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.233+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761819f220 con 0x7f7618100e90 2023-10-06T21:10:04.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.234+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7618107e90 con 0x7f7618100e90 2023-10-06T21:10:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.236+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f760800cc50 con 0x7f7618100e90 2023-10-06T21:10:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.237+0000 7f760ffff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 0x7f76040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.238+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7608014070 con 0x7f7618100e90 2023-10-06T21:10:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.238+0000 7f761659c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 0x7f76040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.238+0000 7f761659c700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 0x7f76040a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f760000b2e0 tx=0x7f7600005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:04.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.241+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7608096510 con 0x7f7618100e90 2023-10-06T21:10:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:04 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1778520129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:04 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/315235720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:04 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2588701062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:04 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1778520129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:04 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/315235720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:04 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2588701062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.455+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f761802d1a0 con 0x7f7618100e90 2023-10-06T21:10:04.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.456+0000 7f760ffff700 1 -- 172.21.15.26:0/2960973772 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7608003dd0 con 0x7f7618100e90 2023-10-06T21:10:04.459 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.458+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 msgr2=0x7f76040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.458+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 0x7f76040a3940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f760000b2e0 tx=0x7f7600005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 msgr2=0x7f7618196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f760800e970 tx=0x7f760800ec80 comp rx=0 tx=0).stop 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 shutdown_connections 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7618102a30 0x7f761819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f76040a1480 0x7f76040a3940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7618100e90 0x7f7618196c30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 --2- 172.21.15.26:0/2960973772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7618102070 0x7f7618197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 >> 172.21.15.26:0/2960973772 conn(0x7f76180fc430 msgr2=0x7f76180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 shutdown_connections 2023-10-06T21:10:04.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.459+0000 7f761cdff700 1 -- 172.21.15.26:0/2960973772 wait complete. 2023-10-06T21:10:04.480 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-06T21:10:04.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:10:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:04 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1778520129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:04 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/315235720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:04 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2588701062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 -- 172.21.15.26:0/3936598584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac100ea0 msgr2=0x7f30ac1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 --2- 172.21.15.26:0/3936598584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac100ea0 0x7f30ac1012c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009990 tx=0x7f30a0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 -- 172.21.15.26:0/3936598584 shutdown_connections 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 --2- 172.21.15.26:0/3936598584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102a40 0x7f30ac109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 --2- 172.21.15.26:0/3936598584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac100ea0 0x7f30ac1012c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 --2- 172.21.15.26:0/3936598584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac102080 0x7f30ac102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 -- 172.21.15.26:0/3936598584 >> 172.21.15.26:0/3936598584 conn(0x7f30ac0fc420 msgr2=0x7f30ac0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 -- 172.21.15.26:0/3936598584 shutdown_connections 2023-10-06T21:10:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.682+0000 7f30b2dff700 1 -- 172.21.15.26:0/3936598584 wait complete. 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.683+0000 7f30b2dff700 1 Processor -- start 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.683+0000 7f30b2dff700 1 -- start start 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 0x7f30ac197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac102a40 0x7f30ac19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19e770 con 0x7f30ac100ea0 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19e8b0 con 0x7f30ac102a40 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b2dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19ea40 con 0x7f30ac102080 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38958/0 (socket says 172.21.15.26:38958) 2023-10-06T21:10:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 -- 172.21.15.26:0/1894509305 learned_addr learned my addr 172.21.15.26:0/1894509305 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30abfff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 0x7f30ac197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b139c700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac102a40 0x7f30ac19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 -- 172.21.15.26:0/1894509305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 msgr2=0x7f30ac197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 0x7f30ac197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 -- 172.21.15.26:0/1894509305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac102a40 msgr2=0x7f30ac19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac102a40 0x7f30ac19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.684+0000 7f30b0b9b700 1 -- 172.21.15.26:0/1894509305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a00095f0 con 0x7f30ac100ea0 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30abfff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 0x7f30ac197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30b0b9b700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005720 tx=0x7f30a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a001d070 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac19ecc0 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a0021410 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0017650 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.685+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac19f0f0 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.686+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac1084c0 con 0x7f30ac100ea0 2023-10-06T21:10:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.688+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f30a0021580 con 0x7f30ac100ea0 2023-10-06T21:10:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.691+0000 7f30a9ffb700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 0x7f309c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.691+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30a00cdee0 con 0x7f30ac100ea0 2023-10-06T21:10:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.691+0000 7f30abfff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 0x7f309c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.692+0000 7f30abfff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 0x7f309c0a3990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f30ac100bc0 tx=0x7f3098009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.695+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f30a0097590 con 0x7f30ac100ea0 2023-10-06T21:10:04.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.952+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30ac10c760 con 0x7f30ac100ea0 2023-10-06T21:10:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.953+0000 7f30a9ffb700 1 -- 172.21.15.26:0/1894509305 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f30a0095e60 con 0x7f30ac100ea0 2023-10-06T21:10:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:04.956 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:10:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 msgr2=0x7f309c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 0x7f309c0a3990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f30ac100bc0 tx=0x7f3098009260 comp rx=0 tx=0).stop 2023-10-06T21:10:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 msgr2=0x7f30ac196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005720 tx=0x7f30a0009580 comp rx=0 tx=0).stop 2023-10-06T21:10:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 shutdown_connections 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30ac102a40 0x7f30ac19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f309c0a14d0 0x7f309c0a3990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac100ea0 0x7f30ac196bf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 --2- 172.21.15.26:0/1894509305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30ac102080 0x7f30ac197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.955+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 >> 172.21.15.26:0/1894509305 conn(0x7f30ac0fc420 msgr2=0x7f30ac105600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.956+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 shutdown_connections 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:04.956+0000 7f30b2dff700 1 -- 172.21.15.26:0/1894509305 wait complete. 2023-10-06T21:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:10:04.977 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-06T21:10:04.977 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-06T21:10:04.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:10:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 -- 172.21.15.26:0/2651165259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 msgr2=0x7f96dc109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 --2- 172.21.15.26:0/2651165259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc109010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f96d400b0a0 tx=0x7f96d400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 -- 172.21.15.26:0/2651165259 shutdown_connections 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 --2- 172.21.15.26:0/2651165259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc109010 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 --2- 172.21.15.26:0/2651165259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1019a0 0x7f96dc103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 --2- 172.21.15.26:0/2651165259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1042d0 0x7f96dc1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 -- 172.21.15.26:0/2651165259 >> 172.21.15.26:0/2651165259 conn(0x7f96dc0fb310 msgr2=0x7f96dc0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.179+0000 7f96e13ff700 1 -- 172.21.15.26:0/2651165259 shutdown_connections 2023-10-06T21:10:05.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.180+0000 7f96e13ff700 1 -- 172.21.15.26:0/2651165259 wait complete. 2023-10-06T21:10:05.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.180+0000 7f96e13ff700 1 Processor -- start 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 -- start start 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1019a0 0x7f96dc101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc100630 con 0x7f96dc106c00 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1007a0 con 0x7f96dc1019a0 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96e13ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1a0c80 con 0x7f96dc1042d0 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96da7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96da7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59996/0 (socket says 172.21.15.26:59996) 2023-10-06T21:10:05.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96da7fc700 1 -- 172.21.15.26:0/3118485028 learned_addr learned my addr 172.21.15.26:0/3118485028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.181+0000 7f96daffd700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1019a0 0x7f96dc101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 -- 172.21.15.26:0/3118485028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1019a0 msgr2=0x7f96dc101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1019a0 0x7f96dc101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 -- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 msgr2=0x7f96dc0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 -- 172.21.15.26:0/3118485028 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009d00 con 0x7f96dc1042d0 2023-10-06T21:10:05.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96da7fc700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e9a0 tx=0x7f96d000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0009790 con 0x7f96dc1042d0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.182+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d0010430 con 0x7f96dc1042d0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.183+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc1a0f60 con 0x7f96dc1042d0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.183+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018680 con 0x7f96dc1042d0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.183+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc1a13e0 con 0x7f96dc1042d0 2023-10-06T21:10:05.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.184+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0fce90 con 0x7f96dc1042d0 2023-10-06T21:10:05.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.186+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f96d00105a0 con 0x7f96dc1042d0 2023-10-06T21:10:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.187+0000 7f96c3fff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 0x7f96c40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.187+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96d0014070 con 0x7f96dc1042d0 2023-10-06T21:10:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.187+0000 7f96daffd700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 0x7f96c40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.188+0000 7f96daffd700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 0x7f96c40a3990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009c60 tx=0x7f96cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:05.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.189+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96d0096520 con 0x7f96dc1042d0 2023-10-06T21:10:05.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:05 smithi026 ceph-mon[98490]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:10:05.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2960973772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:05.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:05 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1894509305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:05.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:05 smithi078 ceph-mon[134896]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:10:05.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2960973772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:05.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:05 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1894509305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.397+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96dc02d2b0 con 0x7f96c40a14d0 2023-10-06T21:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.408+0000 7f96c3fff700 1 -- 172.21.15.26:0/3118485028 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345760 (secure 0 0 0) 0x7f96dc02d2b0 con 0x7f96c40a14d0 2023-10-06T21:10:05.416 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 msgr2=0x7f96c40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 0x7f96c40a3990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009c60 tx=0x7f96cc009260 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 msgr2=0x7f96dc0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e9a0 tx=0x7f96d000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 shutdown_connections 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f96c40a14d0 0x7f96c40a3990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96dc106c00 0x7f96dc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96dc1019a0 0x7f96dc101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 --2- 172.21.15.26:0/3118485028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96dc1042d0 0x7f96dc0ff950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 >> 172.21.15.26:0/3118485028 conn(0x7f96dc0fb310 msgr2=0x7f96dc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 shutdown_connections 2023-10-06T21:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.413+0000 7f96e13ff700 1 -- 172.21.15.26:0/3118485028 wait complete. 2023-10-06T21:10:05.420 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:10:05.435 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-06T21:10:04.399860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655011,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":759,"num_read_kb":494,"num_write":344,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":3009370,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337448,"kb_used_data":13256,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604376,"statfs":{"total":1151940427776,"available":1151594881024,"internally_reserved":0,"allocated":13574144,"data_stored":7436474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":41,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002567"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427530+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:09:44.427530+0000","last_peered":"2023-10-06T21:09:44.427530+0000","last_clean":"2023-10-06T21:09:44.427530+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:09:44.427530+0000","last_undegraded":"2023-10-06T21:09:44.427530+0000","last_fullsized":"2023-10-06T21:09:44.427530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415677+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:09:44.415677+0000","last_peered":"2023-10-06T21:09:44.415677+0000","last_clean":"2023-10-06T21:09:44.415677+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:09:44.415677+0000","last_undegraded":"2023-10-06T21:09:44.415677+0000","last_fullsized":"2023-10-06T21:09:44.415677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421162+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:09:44.421162+0000","last_peered":"2023-10-06T21:09:44.421162+0000","last_clean":"2023-10-06T21:09:44.421162+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:09:44.421162+0000","last_undegraded":"2023-10-06T21:09:44.421162+0000","last_fullsized":"2023-10-06T21:09:44.421162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417045+0000","last_change":"2023-10-06T21:08:45.451527+0000","last_active":"2023-10-06T21:09:44.417045+0000","last_peered":"2023-10-06T21:09:44.417045+0000","last_clean":"2023-10-06T21:09:44.417045+0000","last_became_active":"2023-10-06T21:08:45.451219+0000","last_became_peered":"2023-10-06T21:08:45.451219+0000","last_unstale":"2023-10-06T21:09:44.417045+0000","last_undegraded":"2023-10-06T21:09:44.417045+0000","last_fullsized":"2023-10-06T21:09:44.417045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:03:42.551771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413140+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:09:44.413140+0000","last_peered":"2023-10-06T21:09:44.413140+0000","last_clean":"2023-10-06T21:09:44.413140+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:09:44.413140+0000","last_undegraded":"2023-10-06T21:09:44.413140+0000","last_fullsized":"2023-10-06T21:09:44.413140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417327+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:09:44.417327+0000","last_peered":"2023-10-06T21:09:44.417327+0000","last_clean":"2023-10-06T21:09:44.417327+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:09:44.417327+0000","last_undegraded":"2023-10-06T21:09:44.417327+0000","last_fullsized":"2023-10-06T21:09:44.417327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416179+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:09:44.416179+0000","last_peered":"2023-10-06T21:09:44.416179+0000","last_clean":"2023-10-06T21:09:44.416179+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:09:44.416179+0000","last_undegraded":"2023-10-06T21:09:44.416179+0000","last_fullsized":"2023-10-06T21:09:44.416179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417668+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:09:44.417668+0000","last_peered":"2023-10-06T21:09:44.417668+0000","last_clean":"2023-10-06T21:09:44.417668+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:09:44.417668+0000","last_undegraded":"2023-10-06T21:09:44.417668+0000","last_fullsized":"2023-10-06T21:09:44.417668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418346+0000","last_change":"2023-10-06T21:08:45.451859+0000","last_active":"2023-10-06T21:09:44.418346+0000","last_peered":"2023-10-06T21:09:44.418346+0000","last_clean":"2023-10-06T21:09:44.418346+0000","last_became_active":"2023-10-06T21:08:45.451525+0000","last_became_peered":"2023-10-06T21:08:45.451525+0000","last_unstale":"2023-10-06T21:09:44.418346+0000","last_undegraded":"2023-10-06T21:09:44.418346+0000","last_fullsized":"2023-10-06T21:09:44.418346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:52.350672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411831+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:09:44.411831+0000","last_peered":"2023-10-06T21:09:44.411831+0000","last_clean":"2023-10-06T21:09:44.411831+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:09:44.411831+0000","last_undegraded":"2023-10-06T21:09:44.411831+0000","last_fullsized":"2023-10-06T21:09:44.411831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419428+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:09:44.419428+0000","last_peered":"2023-10-06T21:09:44.419428+0000","last_clean":"2023-10-06T21:09:44.419428+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:09:44.419428+0000","last_undegraded":"2023-10-06T21:09:44.419428+0000","last_fullsized":"2023-10-06T21:09:44.419428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410519+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:09:44.410519+0000","last_peered":"2023-10-06T21:09:44.410519+0000","last_clean":"2023-10-06T21:09:44.410519+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:09:44.410519+0000","last_undegraded":"2023-10-06T21:09:44.410519+0000","last_fullsized":"2023-10-06T21:09:44.410519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421301+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:09:44.421301+0000","last_peered":"2023-10-06T21:09:44.421301+0000","last_clean":"2023-10-06T21:09:44.421301+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:09:44.421301+0000","last_undegraded":"2023-10-06T21:09:44.421301+0000","last_fullsized":"2023-10-06T21:09:44.421301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420910+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:09:44.420910+0000","last_peered":"2023-10-06T21:09:44.420910+0000","last_clean":"2023-10-06T21:09:44.420910+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:09:44.420910+0000","last_undegraded":"2023-10-06T21:09:44.420910+0000","last_fullsized":"2023-10-06T21:09:44.420910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418350+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:09:44.418350+0000","last_peered":"2023-10-06T21:09:44.418350+0000","last_clean":"2023-10-06T21:09:44.418350+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:09:44.418350+0000","last_undegraded":"2023-10-06T21:09:44.418350+0000","last_fullsized":"2023-10-06T21:09:44.418350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425058+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:09:44.425058+0000","last_peered":"2023-10-06T21:09:44.425058+0000","last_clean":"2023-10-06T21:09:44.425058+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:09:44.425058+0000","last_undegraded":"2023-10-06T21:09:44.425058+0000","last_fullsized":"2023-10-06T21:09:44.425058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427610+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:09:44.427610+0000","last_peered":"2023-10-06T21:09:44.427610+0000","last_clean":"2023-10-06T21:09:44.427610+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:09:44.427610+0000","last_undegraded":"2023-10-06T21:09:44.427610+0000","last_fullsized":"2023-10-06T21:09:44.427610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.424873+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:09:44.424873+0000","last_peered":"2023-10-06T21:09:44.424873+0000","last_clean":"2023-10-06T21:09:44.424873+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:09:44.424873+0000","last_undegraded":"2023-10-06T21:09:44.424873+0000","last_fullsized":"2023-10-06T21:09:44.424873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413246+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:09:44.413246+0000","last_peered":"2023-10-06T21:09:44.413246+0000","last_clean":"2023-10-06T21:09:44.413246+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:09:44.413246+0000","last_undegraded":"2023-10-06T21:09:44.413246+0000","last_fullsized":"2023-10-06T21:09:44.413246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439306+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:44.439306+0000","last_peered":"2023-10-06T21:08:44.439306+0000","last_clean":"2023-10-06T21:08:44.439306+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:44.439306+0000","last_undegraded":"2023-10-06T21:08:44.439306+0000","last_fullsized":"2023-10-06T21:08:44.439306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416382+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:09:44.416382+0000","last_peered":"2023-10-06T21:09:44.416382+0000","last_clean":"2023-10-06T21:09:44.416382+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:09:44.416382+0000","last_undegraded":"2023-10-06T21:09:44.416382+0000","last_fullsized":"2023-10-06T21:09:44.416382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435884+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:44.435884+0000","last_peered":"2023-10-06T21:08:44.435884+0000","last_clean":"2023-10-06T21:08:44.435884+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:44.435884+0000","last_undegraded":"2023-10-06T21:08:44.435884+0000","last_fullsized":"2023-10-06T21:08:44.435884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429639+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:09:44.429639+0000","last_peered":"2023-10-06T21:09:44.429639+0000","last_clean":"2023-10-06T21:09:44.429639+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:09:44.429639+0000","last_undegraded":"2023-10-06T21:09:44.429639+0000","last_fullsized":"2023-10-06T21:09:44.429639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440673+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:44.440673+0000","last_peered":"2023-10-06T21:08:44.440673+0000","last_clean":"2023-10-06T21:08:44.440673+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:44.440673+0000","last_undegraded":"2023-10-06T21:08:44.440673+0000","last_fullsized":"2023-10-06T21:08:44.440673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420872+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:09:44.420872+0000","last_peered":"2023-10-06T21:09:44.420872+0000","last_clean":"2023-10-06T21:09:44.420872+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:09:44.420872+0000","last_undegraded":"2023-10-06T21:09:44.420872+0000","last_fullsized":"2023-10-06T21:09:44.420872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441604+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:44.441604+0000","last_peered":"2023-10-06T21:08:44.441604+0000","last_clean":"2023-10-06T21:08:44.441604+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:44.441604+0000","last_undegraded":"2023-10-06T21:08:44.441604+0000","last_fullsized":"2023-10-06T21:08:44.441604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412714+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:09:44.412714+0000","last_peered":"2023-10-06T21:09:44.412714+0000","last_clean":"2023-10-06T21:09:44.412714+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:09:44.412714+0000","last_undegraded":"2023-10-06T21:09:44.412714+0000","last_fullsized":"2023-10-06T21:09:44.412714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":127,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.013129+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:10:01.013129+0000","last_peered":"2023-10-06T21:10:01.013129+0000","last_clean":"2023-10-06T21:10:01.013129+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:10:01.013129+0000","last_undegraded":"2023-10-06T21:10:01.013129+0000","last_fullsized":"2023-10-06T21:10:01.013129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415642+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:09:44.415642+0000","last_peered":"2023-10-06T21:09:44.415642+0000","last_clean":"2023-10-06T21:09:44.415642+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:09:44.415642+0000","last_undegraded":"2023-10-06T21:09:44.415642+0000","last_fullsized":"2023-10-06T21:09:44.415642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443912+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:44.443912+0000","last_peered":"2023-10-06T21:08:44.443912+0000","last_clean":"2023-10-06T21:08:44.443912+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:44.443912+0000","last_undegraded":"2023-10-06T21:08:44.443912+0000","last_fullsized":"2023-10-06T21:08:44.443912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418357+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:09:44.418357+0000","last_peered":"2023-10-06T21:09:44.418357+0000","last_clean":"2023-10-06T21:09:44.418357+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:09:44.418357+0000","last_undegraded":"2023-10-06T21:09:44.418357+0000","last_fullsized":"2023-10-06T21:09:44.418357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436810+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:44.436810+0000","last_peered":"2023-10-06T21:08:44.436810+0000","last_clean":"2023-10-06T21:08:44.436810+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:44.436810+0000","last_undegraded":"2023-10-06T21:08:44.436810+0000","last_fullsized":"2023-10-06T21:08:44.436810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423370+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:09:44.423370+0000","last_peered":"2023-10-06T21:09:44.423370+0000","last_clean":"2023-10-06T21:09:44.423370+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:09:44.423370+0000","last_undegraded":"2023-10-06T21:09:44.423370+0000","last_fullsized":"2023-10-06T21:09:44.423370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425550+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:09:44.425550+0000","last_peered":"2023-10-06T21:09:44.425550+0000","last_clean":"2023-10-06T21:09:44.425550+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:09:44.425550+0000","last_undegraded":"2023-10-06T21:09:44.425550+0000","last_fullsized":"2023-10-06T21:09:44.425550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012541+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:10:01.012541+0000","last_peered":"2023-10-06T21:10:01.012541+0000","last_clean":"2023-10-06T21:10:01.012541+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:10:01.012541+0000","last_undegraded":"2023-10-06T21:10:01.012541+0000","last_fullsized":"2023-10-06T21:10:01.012541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.428824+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:09:44.428824+0000","last_peered":"2023-10-06T21:09:44.428824+0000","last_clean":"2023-10-06T21:09:44.428824+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:09:44.428824+0000","last_undegraded":"2023-10-06T21:09:44.428824+0000","last_fullsized":"2023-10-06T21:09:44.428824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441919+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:44.441919+0000","last_peered":"2023-10-06T21:08:44.441919+0000","last_clean":"2023-10-06T21:08:44.441919+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:44.441919+0000","last_undegraded":"2023-10-06T21:08:44.441919+0000","last_fullsized":"2023-10-06T21:08:44.441919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429382+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:09:44.429382+0000","last_peered":"2023-10-06T21:09:44.429382+0000","last_clean":"2023-10-06T21:09:44.429382+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:09:44.429382+0000","last_undegraded":"2023-10-06T21:09:44.429382+0000","last_fullsized":"2023-10-06T21:09:44.429382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438756+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:44.438756+0000","last_peered":"2023-10-06T21:08:44.438756+0000","last_clean":"2023-10-06T21:08:44.438756+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:44.438756+0000","last_undegraded":"2023-10-06T21:08:44.438756+0000","last_fullsized":"2023-10-06T21:08:44.438756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417432+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:09:44.417432+0000","last_peered":"2023-10-06T21:09:44.417432+0000","last_clean":"2023-10-06T21:09:44.417432+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:09:44.417432+0000","last_undegraded":"2023-10-06T21:09:44.417432+0000","last_fullsized":"2023-10-06T21:09:44.417432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":277,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.744268+0000","last_change":"2023-10-06T21:08:45.622755+0000","last_active":"2023-10-06T21:10:01.744268+0000","last_peered":"2023-10-06T21:10:01.744268+0000","last_clean":"2023-10-06T21:10:01.744268+0000","last_became_active":"2023-10-06T21:08:45.614102+0000","last_became_peered":"2023-10-06T21:08:45.614102+0000","last_unstale":"2023-10-06T21:10:01.744268+0000","last_undegraded":"2023-10-06T21:10:01.744268+0000","last_fullsized":"2023-10-06T21:10:01.744268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:17:14.262961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59725,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":155,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57508,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.405781+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:09:44.405781+0000","last_peered":"2023-10-06T21:09:44.405781+0000","last_clean":"2023-10-06T21:09:44.405781+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:09:44.405781+0000","last_undegraded":"2023-10-06T21:09:44.405781+0000","last_fullsized":"2023-10-06T21:09:44.405781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443552+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:44.443552+0000","last_peered":"2023-10-06T21:08:44.443552+0000","last_clean":"2023-10-06T21:08:44.443552+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:44.443552+0000","last_undegraded":"2023-10-06T21:08:44.443552+0000","last_fullsized":"2023-10-06T21:08:44.443552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422900+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:09:44.422900+0000","last_peered":"2023-10-06T21:09:44.422900+0000","last_clean":"2023-10-06T21:09:44.422900+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:09:44.422900+0000","last_undegraded":"2023-10-06T21:09:44.422900+0000","last_fullsized":"2023-10-06T21:09:44.422900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438078+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:44.438078+0000","last_peered":"2023-10-06T21:08:44.438078+0000","last_clean":"2023-10-06T21:08:44.438078+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:44.438078+0000","last_undegraded":"2023-10-06T21:08:44.438078+0000","last_fullsized":"2023-10-06T21:08:44.438078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423646+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:09:44.423646+0000","last_peered":"2023-10-06T21:09:44.423646+0000","last_clean":"2023-10-06T21:09:44.423646+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:09:44.423646+0000","last_undegraded":"2023-10-06T21:09:44.423646+0000","last_fullsized":"2023-10-06T21:09:44.423646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012372+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:10:01.012372+0000","last_peered":"2023-10-06T21:10:01.012372+0000","last_clean":"2023-10-06T21:10:01.012372+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:10:01.012372+0000","last_undegraded":"2023-10-06T21:10:01.012372+0000","last_fullsized":"2023-10-06T21:10:01.012372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410348+0000","last_change":"2023-10-06T21:08:45.447758+0000","last_active":"2023-10-06T21:09:44.410348+0000","last_peered":"2023-10-06T21:09:44.410348+0000","last_clean":"2023-10-06T21:09:44.410348+0000","last_became_active":"2023-10-06T21:08:45.447337+0000","last_became_peered":"2023-10-06T21:08:45.447337+0000","last_unstale":"2023-10-06T21:09:44.410348+0000","last_undegraded":"2023-10-06T21:09:44.410348+0000","last_fullsized":"2023-10-06T21:09:44.410348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:48:29.806108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439954+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:44.439954+0000","last_peered":"2023-10-06T21:08:44.439954+0000","last_clean":"2023-10-06T21:08:44.439954+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:44.439954+0000","last_undegraded":"2023-10-06T21:08:44.439954+0000","last_fullsized":"2023-10-06T21:08:44.439954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.402236+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:09:44.402236+0000","last_peered":"2023-10-06T21:09:44.402236+0000","last_clean":"2023-10-06T21:09:44.402236+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:09:44.402236+0000","last_undegraded":"2023-10-06T21:09:44.402236+0000","last_fullsized":"2023-10-06T21:09:44.402236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440701+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:44.440701+0000","last_peered":"2023-10-06T21:08:44.440701+0000","last_clean":"2023-10-06T21:08:44.440701+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:44.440701+0000","last_undegraded":"2023-10-06T21:08:44.440701+0000","last_fullsized":"2023-10-06T21:08:44.440701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409776+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:09:44.409776+0000","last_peered":"2023-10-06T21:09:44.409776+0000","last_clean":"2023-10-06T21:09:44.409776+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:09:44.409776+0000","last_undegraded":"2023-10-06T21:09:44.409776+0000","last_fullsized":"2023-10-06T21:09:44.409776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440260+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:44.440260+0000","last_peered":"2023-10-06T21:08:44.440260+0000","last_clean":"2023-10-06T21:08:44.440260+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:44.440260+0000","last_undegraded":"2023-10-06T21:08:44.440260+0000","last_fullsized":"2023-10-06T21:08:44.440260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422707+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:09:44.422707+0000","last_peered":"2023-10-06T21:09:44.422707+0000","last_clean":"2023-10-06T21:09:44.422707+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:09:44.422707+0000","last_undegraded":"2023-10-06T21:09:44.422707+0000","last_fullsized":"2023-10-06T21:09:44.422707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441637+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:44.441637+0000","last_peered":"2023-10-06T21:08:44.441637+0000","last_clean":"2023-10-06T21:08:44.441637+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:44.441637+0000","last_undegraded":"2023-10-06T21:08:44.441637+0000","last_fullsized":"2023-10-06T21:08:44.441637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423917+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:09:44.423917+0000","last_peered":"2023-10-06T21:09:44.423917+0000","last_clean":"2023-10-06T21:09:44.423917+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:09:44.423917+0000","last_undegraded":"2023-10-06T21:09:44.423917+0000","last_fullsized":"2023-10-06T21:09:44.423917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407154+0000","last_change":"2023-10-06T21:08:45.453278+0000","last_active":"2023-10-06T21:09:44.407154+0000","last_peered":"2023-10-06T21:09:44.407154+0000","last_clean":"2023-10-06T21:09:44.407154+0000","last_became_active":"2023-10-06T21:08:45.452892+0000","last_became_peered":"2023-10-06T21:08:45.452892+0000","last_unstale":"2023-10-06T21:09:44.407154+0000","last_undegraded":"2023-10-06T21:09:44.407154+0000","last_fullsized":"2023-10-06T21:09:44.407154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:54.604308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:58.006776+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:09:58.006776+0000","last_peered":"2023-10-06T21:09:58.006776+0000","last_clean":"2023-10-06T21:09:58.006776+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:09:58.006776+0000","last_undegraded":"2023-10-06T21:09:58.006776+0000","last_fullsized":"2023-10-06T21:09:58.006776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423839+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:09:44.423839+0000","last_peered":"2023-10-06T21:09:44.423839+0000","last_clean":"2023-10-06T21:09:44.423839+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:09:44.423839+0000","last_undegraded":"2023-10-06T21:09:44.423839+0000","last_fullsized":"2023-10-06T21:09:44.423839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436314+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:44.436314+0000","last_peered":"2023-10-06T21:08:44.436314+0000","last_clean":"2023-10-06T21:08:44.436314+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:44.436314+0000","last_undegraded":"2023-10-06T21:08:44.436314+0000","last_fullsized":"2023-10-06T21:08:44.436314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.446003+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:45.446003+0000","last_peered":"2023-10-06T21:08:45.446003+0000","last_clean":"2023-10-06T21:08:45.446003+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:45.446003+0000","last_undegraded":"2023-10-06T21:08:45.446003+0000","last_fullsized":"2023-10-06T21:08:45.446003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415596+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:09:44.415596+0000","last_peered":"2023-10-06T21:09:44.415596+0000","last_clean":"2023-10-06T21:09:44.415596+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:09:44.415596+0000","last_undegraded":"2023-10-06T21:09:44.415596+0000","last_fullsized":"2023-10-06T21:09:44.415596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442183+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:44.442183+0000","last_peered":"2023-10-06T21:08:44.442183+0000","last_clean":"2023-10-06T21:08:44.442183+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:44.442183+0000","last_undegraded":"2023-10-06T21:08:44.442183+0000","last_fullsized":"2023-10-06T21:08:44.442183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.454229+0000","last_change":"2023-10-06T21:08:45.453710+0000","last_active":"2023-10-06T21:08:45.454229+0000","last_peered":"2023-10-06T21:08:45.454229+0000","last_clean":"2023-10-06T21:08:45.454229+0000","last_became_active":"2023-10-06T21:08:45.447386+0000","last_became_peered":"2023-10-06T21:08:45.447386+0000","last_unstale":"2023-10-06T21:08:45.454229+0000","last_undegraded":"2023-10-06T21:08:45.454229+0000","last_fullsized":"2023-10-06T21:08:45.454229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:27.659601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.426691+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:09:44.426691+0000","last_peered":"2023-10-06T21:09:44.426691+0000","last_clean":"2023-10-06T21:09:44.426691+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:09:44.426691+0000","last_undegraded":"2023-10-06T21:09:44.426691+0000","last_fullsized":"2023-10-06T21:09:44.426691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.445149+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:45.445149+0000","last_peered":"2023-10-06T21:08:45.445149+0000","last_clean":"2023-10-06T21:08:45.445149+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:45.445149+0000","last_undegraded":"2023-10-06T21:08:45.445149+0000","last_fullsized":"2023-10-06T21:08:45.445149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417794+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:09:44.417794+0000","last_peered":"2023-10-06T21:09:44.417794+0000","last_clean":"2023-10-06T21:09:44.417794+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:09:44.417794+0000","last_undegraded":"2023-10-06T21:09:44.417794+0000","last_fullsized":"2023-10-06T21:09:44.417794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436021+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:44.436021+0000","last_peered":"2023-10-06T21:08:44.436021+0000","last_clean":"2023-10-06T21:08:44.436021+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:44.436021+0000","last_undegraded":"2023-10-06T21:08:44.436021+0000","last_fullsized":"2023-10-06T21:08:44.436021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418795+0000","last_change":"2023-10-06T21:08:45.613497+0000","last_active":"2023-10-06T21:09:44.418795+0000","last_peered":"2023-10-06T21:09:44.418795+0000","last_clean":"2023-10-06T21:09:44.418795+0000","last_became_active":"2023-10-06T21:08:45.613012+0000","last_became_peered":"2023-10-06T21:08:45.613012+0000","last_unstale":"2023-10-06T21:09:44.418795+0000","last_undegraded":"2023-10-06T21:09:44.418795+0000","last_fullsized":"2023-10-06T21:09:44.418795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:04:42.351299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443069+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:44.443069+0000","last_peered":"2023-10-06T21:08:44.443069+0000","last_clean":"2023-10-06T21:08:44.443069+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:44.443069+0000","last_undegraded":"2023-10-06T21:08:44.443069+0000","last_fullsized":"2023-10-06T21:08:44.443069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.384523+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:09:44.384523+0000","last_peered":"2023-10-06T21:09:44.384523+0000","last_clean":"2023-10-06T21:09:44.384523+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:09:44.384523+0000","last_undegraded":"2023-10-06T21:09:44.384523+0000","last_fullsized":"2023-10-06T21:09:44.384523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434716+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:44.434716+0000","last_peered":"2023-10-06T21:08:44.434716+0000","last_clean":"2023-10-06T21:08:44.434716+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:44.434716+0000","last_undegraded":"2023-10-06T21:08:44.434716+0000","last_fullsized":"2023-10-06T21:08:44.434716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423534+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:09:44.423534+0000","last_peered":"2023-10-06T21:09:44.423534+0000","last_clean":"2023-10-06T21:09:44.423534+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:09:44.423534+0000","last_undegraded":"2023-10-06T21:09:44.423534+0000","last_fullsized":"2023-10-06T21:09:44.423534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.457037+0000","last_change":"2023-10-06T21:08:45.456629+0000","last_active":"2023-10-06T21:08:45.457037+0000","last_peered":"2023-10-06T21:08:45.457037+0000","last_clean":"2023-10-06T21:08:45.457037+0000","last_became_active":"2023-10-06T21:08:45.449565+0000","last_became_peered":"2023-10-06T21:08:45.449565+0000","last_unstale":"2023-10-06T21:08:45.457037+0000","last_undegraded":"2023-10-06T21:08:45.457037+0000","last_fullsized":"2023-10-06T21:08:45.457037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:23:54.912161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422471+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:09:44.422471+0000","last_peered":"2023-10-06T21:09:44.422471+0000","last_clean":"2023-10-06T21:09:44.422471+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:09:44.422471+0000","last_undegraded":"2023-10-06T21:09:44.422471+0000","last_fullsized":"2023-10-06T21:09:44.422471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440584+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:44.440584+0000","last_peered":"2023-10-06T21:08:44.440584+0000","last_clean":"2023-10-06T21:08:44.440584+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:44.440584+0000","last_undegraded":"2023-10-06T21:08:44.440584+0000","last_fullsized":"2023-10-06T21:08:44.440584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422433+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:09:44.422433+0000","last_peered":"2023-10-06T21:09:44.422433+0000","last_clean":"2023-10-06T21:09:44.422433+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:09:44.422433+0000","last_undegraded":"2023-10-06T21:09:44.422433+0000","last_fullsized":"2023-10-06T21:09:44.422433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444225+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:44.444225+0000","last_peered":"2023-10-06T21:08:44.444225+0000","last_clean":"2023-10-06T21:08:44.444225+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:44.444225+0000","last_undegraded":"2023-10-06T21:08:44.444225+0000","last_fullsized":"2023-10-06T21:08:44.444225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419710+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:09:44.419710+0000","last_peered":"2023-10-06T21:09:44.419710+0000","last_clean":"2023-10-06T21:09:44.419710+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:09:44.419710+0000","last_undegraded":"2023-10-06T21:09:44.419710+0000","last_fullsized":"2023-10-06T21:09:44.419710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438547+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:44.438547+0000","last_peered":"2023-10-06T21:08:44.438547+0000","last_clean":"2023-10-06T21:08:44.438547+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:44.438547+0000","last_undegraded":"2023-10-06T21:08:44.438547+0000","last_fullsized":"2023-10-06T21:08:44.438547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419495+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:09:44.419495+0000","last_peered":"2023-10-06T21:09:44.419495+0000","last_clean":"2023-10-06T21:09:44.419495+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:09:44.419495+0000","last_undegraded":"2023-10-06T21:09:44.419495+0000","last_fullsized":"2023-10-06T21:09:44.419495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437500+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:44.437500+0000","last_peered":"2023-10-06T21:08:44.437500+0000","last_clean":"2023-10-06T21:08:44.437500+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:44.437500+0000","last_undegraded":"2023-10-06T21:08:44.437500+0000","last_fullsized":"2023-10-06T21:08:44.437500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415401+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:09:44.415401+0000","last_peered":"2023-10-06T21:09:44.415401+0000","last_clean":"2023-10-06T21:09:44.415401+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:09:44.415401+0000","last_undegraded":"2023-10-06T21:09:44.415401+0000","last_fullsized":"2023-10-06T21:09:44.415401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411059+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:09:44.411059+0000","last_peered":"2023-10-06T21:09:44.411059+0000","last_clean":"2023-10-06T21:09:44.411059+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:09:44.411059+0000","last_undegraded":"2023-10-06T21:09:44.411059+0000","last_fullsized":"2023-10-06T21:09:44.411059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412660+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:09:44.412660+0000","last_peered":"2023-10-06T21:09:44.412660+0000","last_clean":"2023-10-06T21:09:44.412660+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:09:44.412660+0000","last_undegraded":"2023-10-06T21:09:44.412660+0000","last_fullsized":"2023-10-06T21:09:44.412660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419648+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:09:44.419648+0000","last_peered":"2023-10-06T21:09:44.419648+0000","last_clean":"2023-10-06T21:09:44.419648+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:09:44.419648+0000","last_undegraded":"2023-10-06T21:09:44.419648+0000","last_fullsized":"2023-10-06T21:09:44.419648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422939+0000","last_change":"2023-10-06T21:08:45.448913+0000","last_active":"2023-10-06T21:09:44.422939+0000","last_peered":"2023-10-06T21:09:44.422939+0000","last_clean":"2023-10-06T21:09:44.422939+0000","last_became_active":"2023-10-06T21:08:45.448485+0000","last_became_peered":"2023-10-06T21:08:45.448485+0000","last_unstale":"2023-10-06T21:09:44.422939+0000","last_undegraded":"2023-10-06T21:09:44.422939+0000","last_fullsized":"2023-10-06T21:09:44.422939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:42:07.732211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422296+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:09:44.422296+0000","last_peered":"2023-10-06T21:09:44.422296+0000","last_clean":"2023-10-06T21:09:44.422296+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:09:44.422296+0000","last_undegraded":"2023-10-06T21:09:44.422296+0000","last_fullsized":"2023-10-06T21:09:44.422296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407447+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:09:44.407447+0000","last_peered":"2023-10-06T21:09:44.407447+0000","last_clean":"2023-10-06T21:09:44.407447+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:09:44.407447+0000","last_undegraded":"2023-10-06T21:09:44.407447+0000","last_fullsized":"2023-10-06T21:09:44.407447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.414676+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:09:44.414676+0000","last_peered":"2023-10-06T21:09:44.414676+0000","last_clean":"2023-10-06T21:09:44.414676+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:09:44.414676+0000","last_undegraded":"2023-10-06T21:09:44.414676+0000","last_fullsized":"2023-10-06T21:09:44.414676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416177+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:09:44.416177+0000","last_peered":"2023-10-06T21:09:44.416177+0000","last_clean":"2023-10-06T21:09:44.416177+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:09:44.416177+0000","last_undegraded":"2023-10-06T21:09:44.416177+0000","last_fullsized":"2023-10-06T21:09:44.416177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418196+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:09:44.418196+0000","last_peered":"2023-10-06T21:09:44.418196+0000","last_clean":"2023-10-06T21:09:44.418196+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:09:44.418196+0000","last_undegraded":"2023-10-06T21:09:44.418196+0000","last_fullsized":"2023-10-06T21:09:44.418196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417342+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:09:44.417342+0000","last_peered":"2023-10-06T21:09:44.417342+0000","last_clean":"2023-10-06T21:09:44.417342+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:09:44.417342+0000","last_undegraded":"2023-10-06T21:09:44.417342+0000","last_fullsized":"2023-10-06T21:09:44.417342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423644+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:09:44.423644+0000","last_peered":"2023-10-06T21:09:44.423644+0000","last_clean":"2023-10-06T21:09:44.423644+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:09:44.423644+0000","last_undegraded":"2023-10-06T21:09:44.423644+0000","last_fullsized":"2023-10-06T21:09:44.423644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423037+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:09:44.423037+0000","last_peered":"2023-10-06T21:09:44.423037+0000","last_clean":"2023-10-06T21:09:44.423037+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:09:44.423037+0000","last_undegraded":"2023-10-06T21:09:44.423037+0000","last_fullsized":"2023-10-06T21:09:44.423037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409451+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:09:44.409451+0000","last_peered":"2023-10-06T21:09:44.409451+0000","last_clean":"2023-10-06T21:09:44.409451+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:09:44.409451+0000","last_undegraded":"2023-10-06T21:09:44.409451+0000","last_fullsized":"2023-10-06T21:09:44.409451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64397,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":623,"num_read_kb":375,"num_write":161,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":57530,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.40699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.623}]},{"osd":3,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.47399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77200000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91900000000000004,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.92800000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.875},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.83399999999999996,"15min":0.81699999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.80800000000000005},"min":{"1min":0.63400000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.96899999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.90300000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.499}]},{"osd":2,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":1.089,"15min":1.089},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.82099999999999995,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.873,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.85199999999999998,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84899999999999998,"5min":1.26,"15min":1.26},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.048,"15min":1.048},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.56899999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.90900000000000003,"5min":1,"15min":1},"last":0.76400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":456799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51100000000000001,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.441},{"interface":"front","average":{"1min":0.44,"5min":0.56299999999999994,"15min":0.58699999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":1.014,"15min":1.014},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.083,"5min":1.347,"15min":1.347},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.78700000000000003,"15min":0.82499999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.79500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1240000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68200000000000005,"15min":0.70099999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.501},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81399999999999995,"15min":0.83799999999999997},"min":{"1min":0.499,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.085,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.93600000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85599999999999998,"15min":0.88700000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0089999999999999,"5min":1.226,"15min":1.226},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":1.0069999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2490000000000001,"5min":1.542,"15min":1.542},"last":0.89600000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":0.86699999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.125,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.056,"5min":1.365,"15min":1.365},"last":0.98899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.95799999999999996,"15min":0.94999999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1359999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Fri Oct 6 21:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.68899999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71999999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.53400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47999999999999998},"min":{"1min":0.39200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92900000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63},"min":{"1min":0.60399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93200000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.46300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93100000000000005,"5min":1.087,"15min":1.087},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.59399999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":1.038,"15min":1.038},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.65300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0009999999999999,"5min":1.155,"15min":1.155},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.71299999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.93899999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.63100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77600000000000002,"15min":0.754},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97299999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95399999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1049326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.443,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.752},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.76500000000000001,"5min":1.002,"15min":1.002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82299999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.60899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.48299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.877,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.012,"15min":1.012},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.55600000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.879},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.84699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.46,"5min":5.8620000000000001,"15min":5.8620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.93400000000000005,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.56699999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98299999999999998,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.58099999999999996,"5min":0.38,"15min":0.38},"max":{"1min":1.1679999999999999,"5min":1.948,"15min":1.948},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2749999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3879999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":5.9290000000000003,"15min":5.9290000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.355,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.53800000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.036,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.85099999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4790000000000001,"5min":5.0199999999999996,"15min":5.0199999999999996},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.497,"15min":0.497},"max":{"1min":4.9420000000000002,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":0.98499999999999999},"min":{"1min":0.76800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.127,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.622,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.206,"5min":2.258,"15min":2.258},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86899999999999999,"15min":0.83399999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.8979999999999997,"5min":4.8979999999999997,"15min":4.8979999999999997},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.29599999999999999},"max":{"1min":0.76900000000000002,"5min":6.8949999999999996,"15min":6.8949999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32000000000000001},"max":{"1min":0.81000000000000005,"5min":1.105,"15min":1.105},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:09:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.318,"15min":0.318},"max":{"1min":0.81599999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.48599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.877,"15min":0.877},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.59399999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.64100000000000001,"15min":0.60899999999999999},"min":{"1min":0.48099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78800000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64800000000000002},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":2.714,"15min":2.714},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.498},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.71399999999999997,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.999,"5min":1.022,"15min":1.022},"last":0.82699999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.622},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.63300000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.67200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.64500000000000002,"15min":0.58099999999999996},"min":{"1min":0.45400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92400000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.77800000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97399999999999998,"5min":1.216,"15min":1.216},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.145},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.238,"5min":1.397,"15min":1.397},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68200000000000005,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.58099999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.27,"5min":1.375,"15min":1.375},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.745},{"interface":"front","average":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":1.1100000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1000000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80700000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.45800000000000002,"15min":0.45000000000000001},"min":{"1min":0.39800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.502,"15min":0.48699999999999999},"min":{"1min":0.38700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.65400000000000003},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98799999999999999,"5min":1.139,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95099999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.58299999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47499999999999998,"15min":0.432},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.55300000000000005},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98199999999999998,"5min":1.002,"15min":1.002},"last":0.748},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69499999999999995,"15min":0.64900000000000002},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93600000000000005,"5min":1.103,"15min":1.103},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72099999999999997,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":1.129,"15min":1.129},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.71299999999999997},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1579999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.92400000000000004,"15min":0.96399999999999997},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0860000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.98099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.395,"15min":1.395},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.129,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78600000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":516881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.36099999999999999},"min":{"1min":0.32100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53700000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.36599999999999999},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.56100000000000005},"min":{"1min":0.45800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.371,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.435}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.58099999999999996},"min":{"1min":0.57499999999999996,"5min":0.37,"15min":0.37},"max":{"1min":0.89800000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.66100000000000003,"15min":0.60499999999999998},"min":{"1min":0.56499999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.94399999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.90100000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.752,"15min":0.745},"min":{"1min":0.47399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1048148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.38900000000000001},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.876,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92800000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77000000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90000000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.876},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:10:05.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:10:05.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:05 smithi167 ceph-mon[134494]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-06T21:10:05.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2960973772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:05.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:05 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1894509305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:05.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/570781360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8100ea0 msgr2=0x7f4dc81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/570781360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8100ea0 0x7f4dc81012c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009990 tx=0x7f4dc4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/570781360 shutdown_connections 2023-10-06T21:10:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/570781360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 0x7f4dc8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/570781360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8100ea0 0x7f4dc81012c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/570781360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8102080 0x7f4dc8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.676+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/570781360 >> 172.21.15.26:0/570781360 conn(0x7f4dc80fc420 msgr2=0x7f4dc80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.677+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/570781360 shutdown_connections 2023-10-06T21:10:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.677+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/570781360 wait complete. 2023-10-06T21:10:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 Processor -- start 2023-10-06T21:10:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 -- start start 2023-10-06T21:10:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8100ea0 0x7f4dc8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 0x7f4dc819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819e780 con 0x7f4dc8102080 2023-10-06T21:10:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819e8f0 con 0x7f4dc8100ea0 2023-10-06T21:10:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcf5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819eab0 con 0x7f4dc8102a40 2023-10-06T21:10:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dccb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dccb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38994/0 (socket says 172.21.15.26:38994) 2023-10-06T21:10:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dccb9a700 1 -- 172.21.15.26:0/3490243651 learned_addr learned my addr 172.21.15.26:0/3490243651 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcd39b700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8100ea0 0x7f4dc8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.678+0000 7f4dcdb9c700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 0x7f4dc819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 -- 172.21.15.26:0/3490243651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 msgr2=0x7f4dc819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 0x7f4dc819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 -- 172.21.15.26:0/3490243651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8100ea0 msgr2=0x7f4dc8196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8100ea0 0x7f4dc8196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 -- 172.21.15.26:0/3490243651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009520 con 0x7f4dc8102080 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dccb9a700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4db800ea50 tx=0x7f4db800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800cbd0 con 0x7f4dc8102080 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc40095f0 con 0x7f4dc8102080 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4db800cd30 con 0x7f4dc8102080 2023-10-06T21:10:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.679+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8018680 con 0x7f4dc8102080 2023-10-06T21:10:05.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.680+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc819f110 con 0x7f4dc8102080 2023-10-06T21:10:05.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.680+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc810c760 con 0x7f4dc8102080 2023-10-06T21:10:05.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.682+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4db8010430 con 0x7f4dc8102080 2023-10-06T21:10:05.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.685+0000 7f4dbe7fc700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 0x7f4db40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:05.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.685+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4db8014070 con 0x7f4dc8102080 2023-10-06T21:10:05.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.685+0000 7f4dcd39b700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 0x7f4db40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:05.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.685+0000 7f4dcd39b700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 0x7f4db40a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4000c00 tx=0x7f4dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:05.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.689+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4db80967f0 con 0x7f4dc8102080 2023-10-06T21:10:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.895+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4dc80612f0 con 0x7f4db40a14d0 2023-10-06T21:10:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.906+0000 7f4dbe7fc700 1 -- 172.21.15.26:0/3490243651 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345760 (secure 0 0 0) 0x7f4dc80612f0 con 0x7f4db40a14d0 2023-10-06T21:10:05.913 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 msgr2=0x7f4db40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 0x7f4db40a3990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4000c00 tx=0x7f4dc4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 msgr2=0x7f4dc8197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4db800ea50 tx=0x7f4db800ed60 comp rx=0 tx=0).stop 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 shutdown_connections 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dc8102a40 0x7f4dc819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc8100ea0 0x7f4dc8196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f4db40a14d0 0x7f4db40a3990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 --2- 172.21.15.26:0/3490243651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dc8102080 0x7f4dc8197180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 >> 172.21.15.26:0/3490243651 conn(0x7f4dc80fc420 msgr2=0x7f4dc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 shutdown_connections 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:05.911+0000 7f4dcf5ff700 1 -- 172.21.15.26:0/3490243651 wait complete. 2023-10-06T21:10:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:10:05.933 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-06T21:10:04.399860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655011,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":759,"num_read_kb":494,"num_write":344,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":3009370,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337448,"kb_used_data":13256,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604376,"statfs":{"total":1151940427776,"available":1151594881024,"internally_reserved":0,"allocated":13574144,"data_stored":7436474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":41,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002567"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427530+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:09:44.427530+0000","last_peered":"2023-10-06T21:09:44.427530+0000","last_clean":"2023-10-06T21:09:44.427530+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:09:44.427530+0000","last_undegraded":"2023-10-06T21:09:44.427530+0000","last_fullsized":"2023-10-06T21:09:44.427530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415677+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:09:44.415677+0000","last_peered":"2023-10-06T21:09:44.415677+0000","last_clean":"2023-10-06T21:09:44.415677+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:09:44.415677+0000","last_undegraded":"2023-10-06T21:09:44.415677+0000","last_fullsized":"2023-10-06T21:09:44.415677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421162+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:09:44.421162+0000","last_peered":"2023-10-06T21:09:44.421162+0000","last_clean":"2023-10-06T21:09:44.421162+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:09:44.421162+0000","last_undegraded":"2023-10-06T21:09:44.421162+0000","last_fullsized":"2023-10-06T21:09:44.421162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417045+0000","last_change":"2023-10-06T21:08:45.451527+0000","last_active":"2023-10-06T21:09:44.417045+0000","last_peered":"2023-10-06T21:09:44.417045+0000","last_clean":"2023-10-06T21:09:44.417045+0000","last_became_active":"2023-10-06T21:08:45.451219+0000","last_became_peered":"2023-10-06T21:08:45.451219+0000","last_unstale":"2023-10-06T21:09:44.417045+0000","last_undegraded":"2023-10-06T21:09:44.417045+0000","last_fullsized":"2023-10-06T21:09:44.417045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:03:42.551771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413140+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:09:44.413140+0000","last_peered":"2023-10-06T21:09:44.413140+0000","last_clean":"2023-10-06T21:09:44.413140+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:09:44.413140+0000","last_undegraded":"2023-10-06T21:09:44.413140+0000","last_fullsized":"2023-10-06T21:09:44.413140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417327+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:09:44.417327+0000","last_peered":"2023-10-06T21:09:44.417327+0000","last_clean":"2023-10-06T21:09:44.417327+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:09:44.417327+0000","last_undegraded":"2023-10-06T21:09:44.417327+0000","last_fullsized":"2023-10-06T21:09:44.417327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416179+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:09:44.416179+0000","last_peered":"2023-10-06T21:09:44.416179+0000","last_clean":"2023-10-06T21:09:44.416179+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:09:44.416179+0000","last_undegraded":"2023-10-06T21:09:44.416179+0000","last_fullsized":"2023-10-06T21:09:44.416179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417668+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:09:44.417668+0000","last_peered":"2023-10-06T21:09:44.417668+0000","last_clean":"2023-10-06T21:09:44.417668+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:09:44.417668+0000","last_undegraded":"2023-10-06T21:09:44.417668+0000","last_fullsized":"2023-10-06T21:09:44.417668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418346+0000","last_change":"2023-10-06T21:08:45.451859+0000","last_active":"2023-10-06T21:09:44.418346+0000","last_peered":"2023-10-06T21:09:44.418346+0000","last_clean":"2023-10-06T21:09:44.418346+0000","last_became_active":"2023-10-06T21:08:45.451525+0000","last_became_peered":"2023-10-06T21:08:45.451525+0000","last_unstale":"2023-10-06T21:09:44.418346+0000","last_undegraded":"2023-10-06T21:09:44.418346+0000","last_fullsized":"2023-10-06T21:09:44.418346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:52.350672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411831+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:09:44.411831+0000","last_peered":"2023-10-06T21:09:44.411831+0000","last_clean":"2023-10-06T21:09:44.411831+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:09:44.411831+0000","last_undegraded":"2023-10-06T21:09:44.411831+0000","last_fullsized":"2023-10-06T21:09:44.411831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419428+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:09:44.419428+0000","last_peered":"2023-10-06T21:09:44.419428+0000","last_clean":"2023-10-06T21:09:44.419428+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:09:44.419428+0000","last_undegraded":"2023-10-06T21:09:44.419428+0000","last_fullsized":"2023-10-06T21:09:44.419428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410519+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:09:44.410519+0000","last_peered":"2023-10-06T21:09:44.410519+0000","last_clean":"2023-10-06T21:09:44.410519+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:09:44.410519+0000","last_undegraded":"2023-10-06T21:09:44.410519+0000","last_fullsized":"2023-10-06T21:09:44.410519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421301+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:09:44.421301+0000","last_peered":"2023-10-06T21:09:44.421301+0000","last_clean":"2023-10-06T21:09:44.421301+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:09:44.421301+0000","last_undegraded":"2023-10-06T21:09:44.421301+0000","last_fullsized":"2023-10-06T21:09:44.421301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420910+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:09:44.420910+0000","last_peered":"2023-10-06T21:09:44.420910+0000","last_clean":"2023-10-06T21:09:44.420910+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:09:44.420910+0000","last_undegraded":"2023-10-06T21:09:44.420910+0000","last_fullsized":"2023-10-06T21:09:44.420910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418350+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:09:44.418350+0000","last_peered":"2023-10-06T21:09:44.418350+0000","last_clean":"2023-10-06T21:09:44.418350+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:09:44.418350+0000","last_undegraded":"2023-10-06T21:09:44.418350+0000","last_fullsized":"2023-10-06T21:09:44.418350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425058+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:09:44.425058+0000","last_peered":"2023-10-06T21:09:44.425058+0000","last_clean":"2023-10-06T21:09:44.425058+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:09:44.425058+0000","last_undegraded":"2023-10-06T21:09:44.425058+0000","last_fullsized":"2023-10-06T21:09:44.425058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427610+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:09:44.427610+0000","last_peered":"2023-10-06T21:09:44.427610+0000","last_clean":"2023-10-06T21:09:44.427610+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:09:44.427610+0000","last_undegraded":"2023-10-06T21:09:44.427610+0000","last_fullsized":"2023-10-06T21:09:44.427610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.424873+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:09:44.424873+0000","last_peered":"2023-10-06T21:09:44.424873+0000","last_clean":"2023-10-06T21:09:44.424873+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:09:44.424873+0000","last_undegraded":"2023-10-06T21:09:44.424873+0000","last_fullsized":"2023-10-06T21:09:44.424873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413246+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:09:44.413246+0000","last_peered":"2023-10-06T21:09:44.413246+0000","last_clean":"2023-10-06T21:09:44.413246+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:09:44.413246+0000","last_undegraded":"2023-10-06T21:09:44.413246+0000","last_fullsized":"2023-10-06T21:09:44.413246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439306+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:44.439306+0000","last_peered":"2023-10-06T21:08:44.439306+0000","last_clean":"2023-10-06T21:08:44.439306+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:44.439306+0000","last_undegraded":"2023-10-06T21:08:44.439306+0000","last_fullsized":"2023-10-06T21:08:44.439306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416382+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:09:44.416382+0000","last_peered":"2023-10-06T21:09:44.416382+0000","last_clean":"2023-10-06T21:09:44.416382+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:09:44.416382+0000","last_undegraded":"2023-10-06T21:09:44.416382+0000","last_fullsized":"2023-10-06T21:09:44.416382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435884+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:44.435884+0000","last_peered":"2023-10-06T21:08:44.435884+0000","last_clean":"2023-10-06T21:08:44.435884+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:44.435884+0000","last_undegraded":"2023-10-06T21:08:44.435884+0000","last_fullsized":"2023-10-06T21:08:44.435884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429639+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:09:44.429639+0000","last_peered":"2023-10-06T21:09:44.429639+0000","last_clean":"2023-10-06T21:09:44.429639+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:09:44.429639+0000","last_undegraded":"2023-10-06T21:09:44.429639+0000","last_fullsized":"2023-10-06T21:09:44.429639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440673+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:44.440673+0000","last_peered":"2023-10-06T21:08:44.440673+0000","last_clean":"2023-10-06T21:08:44.440673+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:44.440673+0000","last_undegraded":"2023-10-06T21:08:44.440673+0000","last_fullsized":"2023-10-06T21:08:44.440673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420872+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:09:44.420872+0000","last_peered":"2023-10-06T21:09:44.420872+0000","last_clean":"2023-10-06T21:09:44.420872+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:09:44.420872+0000","last_undegraded":"2023-10-06T21:09:44.420872+0000","last_fullsized":"2023-10-06T21:09:44.420872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441604+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:44.441604+0000","last_peered":"2023-10-06T21:08:44.441604+0000","last_clean":"2023-10-06T21:08:44.441604+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:44.441604+0000","last_undegraded":"2023-10-06T21:08:44.441604+0000","last_fullsized":"2023-10-06T21:08:44.441604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412714+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:09:44.412714+0000","last_peered":"2023-10-06T21:09:44.412714+0000","last_clean":"2023-10-06T21:09:44.412714+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:09:44.412714+0000","last_undegraded":"2023-10-06T21:09:44.412714+0000","last_fullsized":"2023-10-06T21:09:44.412714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":127,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.013129+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:10:01.013129+0000","last_peered":"2023-10-06T21:10:01.013129+0000","last_clean":"2023-10-06T21:10:01.013129+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:10:01.013129+0000","last_undegraded":"2023-10-06T21:10:01.013129+0000","last_fullsized":"2023-10-06T21:10:01.013129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415642+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:09:44.415642+0000","last_peered":"2023-10-06T21:09:44.415642+0000","last_clean":"2023-10-06T21:09:44.415642+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:09:44.415642+0000","last_undegraded":"2023-10-06T21:09:44.415642+0000","last_fullsized":"2023-10-06T21:09:44.415642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443912+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:44.443912+0000","last_peered":"2023-10-06T21:08:44.443912+0000","last_clean":"2023-10-06T21:08:44.443912+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:44.443912+0000","last_undegraded":"2023-10-06T21:08:44.443912+0000","last_fullsized":"2023-10-06T21:08:44.443912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418357+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:09:44.418357+0000","last_peered":"2023-10-06T21:09:44.418357+0000","last_clean":"2023-10-06T21:09:44.418357+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:09:44.418357+0000","last_undegraded":"2023-10-06T21:09:44.418357+0000","last_fullsized":"2023-10-06T21:09:44.418357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436810+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:44.436810+0000","last_peered":"2023-10-06T21:08:44.436810+0000","last_clean":"2023-10-06T21:08:44.436810+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:44.436810+0000","last_undegraded":"2023-10-06T21:08:44.436810+0000","last_fullsized":"2023-10-06T21:08:44.436810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423370+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:09:44.423370+0000","last_peered":"2023-10-06T21:09:44.423370+0000","last_clean":"2023-10-06T21:09:44.423370+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:09:44.423370+0000","last_undegraded":"2023-10-06T21:09:44.423370+0000","last_fullsized":"2023-10-06T21:09:44.423370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425550+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:09:44.425550+0000","last_peered":"2023-10-06T21:09:44.425550+0000","last_clean":"2023-10-06T21:09:44.425550+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:09:44.425550+0000","last_undegraded":"2023-10-06T21:09:44.425550+0000","last_fullsized":"2023-10-06T21:09:44.425550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012541+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:10:01.012541+0000","last_peered":"2023-10-06T21:10:01.012541+0000","last_clean":"2023-10-06T21:10:01.012541+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:10:01.012541+0000","last_undegraded":"2023-10-06T21:10:01.012541+0000","last_fullsized":"2023-10-06T21:10:01.012541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.428824+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:09:44.428824+0000","last_peered":"2023-10-06T21:09:44.428824+0000","last_clean":"2023-10-06T21:09:44.428824+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:09:44.428824+0000","last_undegraded":"2023-10-06T21:09:44.428824+0000","last_fullsized":"2023-10-06T21:09:44.428824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441919+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:44.441919+0000","last_peered":"2023-10-06T21:08:44.441919+0000","last_clean":"2023-10-06T21:08:44.441919+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:44.441919+0000","last_undegraded":"2023-10-06T21:08:44.441919+0000","last_fullsized":"2023-10-06T21:08:44.441919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429382+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:09:44.429382+0000","last_peered":"2023-10-06T21:09:44.429382+0000","last_clean":"2023-10-06T21:09:44.429382+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:09:44.429382+0000","last_undegraded":"2023-10-06T21:09:44.429382+0000","last_fullsized":"2023-10-06T21:09:44.429382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438756+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:44.438756+0000","last_peered":"2023-10-06T21:08:44.438756+0000","last_clean":"2023-10-06T21:08:44.438756+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:44.438756+0000","last_undegraded":"2023-10-06T21:08:44.438756+0000","last_fullsized":"2023-10-06T21:08:44.438756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417432+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:09:44.417432+0000","last_peered":"2023-10-06T21:09:44.417432+0000","last_clean":"2023-10-06T21:09:44.417432+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:09:44.417432+0000","last_undegraded":"2023-10-06T21:09:44.417432+0000","last_fullsized":"2023-10-06T21:09:44.417432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":277,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.744268+0000","last_change":"2023-10-06T21:08:45.622755+0000","last_active":"2023-10-06T21:10:01.744268+0000","last_peered":"2023-10-06T21:10:01.744268+0000","last_clean":"2023-10-06T21:10:01.744268+0000","last_became_active":"2023-10-06T21:08:45.614102+0000","last_became_peered":"2023-10-06T21:08:45.614102+0000","last_unstale":"2023-10-06T21:10:01.744268+0000","last_undegraded":"2023-10-06T21:10:01.744268+0000","last_fullsized":"2023-10-06T21:10:01.744268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:17:14.262961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59725,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":155,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57508,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.405781+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:09:44.405781+0000","last_peered":"2023-10-06T21:09:44.405781+0000","last_clean":"2023-10-06T21:09:44.405781+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:09:44.405781+0000","last_undegraded":"2023-10-06T21:09:44.405781+0000","last_fullsized":"2023-10-06T21:09:44.405781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443552+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:44.443552+0000","last_peered":"2023-10-06T21:08:44.443552+0000","last_clean":"2023-10-06T21:08:44.443552+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:44.443552+0000","last_undegraded":"2023-10-06T21:08:44.443552+0000","last_fullsized":"2023-10-06T21:08:44.443552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422900+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:09:44.422900+0000","last_peered":"2023-10-06T21:09:44.422900+0000","last_clean":"2023-10-06T21:09:44.422900+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:09:44.422900+0000","last_undegraded":"2023-10-06T21:09:44.422900+0000","last_fullsized":"2023-10-06T21:09:44.422900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438078+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:44.438078+0000","last_peered":"2023-10-06T21:08:44.438078+0000","last_clean":"2023-10-06T21:08:44.438078+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:44.438078+0000","last_undegraded":"2023-10-06T21:08:44.438078+0000","last_fullsized":"2023-10-06T21:08:44.438078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423646+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:09:44.423646+0000","last_peered":"2023-10-06T21:09:44.423646+0000","last_clean":"2023-10-06T21:09:44.423646+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:09:44.423646+0000","last_undegraded":"2023-10-06T21:09:44.423646+0000","last_fullsized":"2023-10-06T21:09:44.423646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012372+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:10:01.012372+0000","last_peered":"2023-10-06T21:10:01.012372+0000","last_clean":"2023-10-06T21:10:01.012372+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:10:01.012372+0000","last_undegraded":"2023-10-06T21:10:01.012372+0000","last_fullsized":"2023-10-06T21:10:01.012372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410348+0000","last_change":"2023-10-06T21:08:45.447758+0000","last_active":"2023-10-06T21:09:44.410348+0000","last_peered":"2023-10-06T21:09:44.410348+0000","last_clean":"2023-10-06T21:09:44.410348+0000","last_became_active":"2023-10-06T21:08:45.447337+0000","last_became_peered":"2023-10-06T21:08:45.447337+0000","last_unstale":"2023-10-06T21:09:44.410348+0000","last_undegraded":"2023-10-06T21:09:44.410348+0000","last_fullsized":"2023-10-06T21:09:44.410348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:48:29.806108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439954+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:44.439954+0000","last_peered":"2023-10-06T21:08:44.439954+0000","last_clean":"2023-10-06T21:08:44.439954+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:44.439954+0000","last_undegraded":"2023-10-06T21:08:44.439954+0000","last_fullsized":"2023-10-06T21:08:44.439954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.402236+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:09:44.402236+0000","last_peered":"2023-10-06T21:09:44.402236+0000","last_clean":"2023-10-06T21:09:44.402236+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:09:44.402236+0000","last_undegraded":"2023-10-06T21:09:44.402236+0000","last_fullsized":"2023-10-06T21:09:44.402236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440701+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:44.440701+0000","last_peered":"2023-10-06T21:08:44.440701+0000","last_clean":"2023-10-06T21:08:44.440701+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:44.440701+0000","last_undegraded":"2023-10-06T21:08:44.440701+0000","last_fullsized":"2023-10-06T21:08:44.440701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409776+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:09:44.409776+0000","last_peered":"2023-10-06T21:09:44.409776+0000","last_clean":"2023-10-06T21:09:44.409776+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:09:44.409776+0000","last_undegraded":"2023-10-06T21:09:44.409776+0000","last_fullsized":"2023-10-06T21:09:44.409776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440260+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:44.440260+0000","last_peered":"2023-10-06T21:08:44.440260+0000","last_clean":"2023-10-06T21:08:44.440260+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:44.440260+0000","last_undegraded":"2023-10-06T21:08:44.440260+0000","last_fullsized":"2023-10-06T21:08:44.440260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422707+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:09:44.422707+0000","last_peered":"2023-10-06T21:09:44.422707+0000","last_clean":"2023-10-06T21:09:44.422707+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:09:44.422707+0000","last_undegraded":"2023-10-06T21:09:44.422707+0000","last_fullsized":"2023-10-06T21:09:44.422707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441637+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:44.441637+0000","last_peered":"2023-10-06T21:08:44.441637+0000","last_clean":"2023-10-06T21:08:44.441637+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:44.441637+0000","last_undegraded":"2023-10-06T21:08:44.441637+0000","last_fullsized":"2023-10-06T21:08:44.441637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423917+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:09:44.423917+0000","last_peered":"2023-10-06T21:09:44.423917+0000","last_clean":"2023-10-06T21:09:44.423917+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:09:44.423917+0000","last_undegraded":"2023-10-06T21:09:44.423917+0000","last_fullsized":"2023-10-06T21:09:44.423917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407154+0000","last_change":"2023-10-06T21:08:45.453278+0000","last_active":"2023-10-06T21:09:44.407154+0000","last_peered":"2023-10-06T21:09:44.407154+0000","last_clean":"2023-10-06T21:09:44.407154+0000","last_became_active":"2023-10-06T21:08:45.452892+0000","last_became_peered":"2023-10-06T21:08:45.452892+0000","last_unstale":"2023-10-06T21:09:44.407154+0000","last_undegraded":"2023-10-06T21:09:44.407154+0000","last_fullsized":"2023-10-06T21:09:44.407154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:54.604308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:58.006776+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:09:58.006776+0000","last_peered":"2023-10-06T21:09:58.006776+0000","last_clean":"2023-10-06T21:09:58.006776+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:09:58.006776+0000","last_undegraded":"2023-10-06T21:09:58.006776+0000","last_fullsized":"2023-10-06T21:09:58.006776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423839+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:09:44.423839+0000","last_peered":"2023-10-06T21:09:44.423839+0000","last_clean":"2023-10-06T21:09:44.423839+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:09:44.423839+0000","last_undegraded":"2023-10-06T21:09:44.423839+0000","last_fullsized":"2023-10-06T21:09:44.423839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436314+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:44.436314+0000","last_peered":"2023-10-06T21:08:44.436314+0000","last_clean":"2023-10-06T21:08:44.436314+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:44.436314+0000","last_undegraded":"2023-10-06T21:08:44.436314+0000","last_fullsized":"2023-10-06T21:08:44.436314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.446003+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:45.446003+0000","last_peered":"2023-10-06T21:08:45.446003+0000","last_clean":"2023-10-06T21:08:45.446003+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:45.446003+0000","last_undegraded":"2023-10-06T21:08:45.446003+0000","last_fullsized":"2023-10-06T21:08:45.446003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415596+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:09:44.415596+0000","last_peered":"2023-10-06T21:09:44.415596+0000","last_clean":"2023-10-06T21:09:44.415596+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:09:44.415596+0000","last_undegraded":"2023-10-06T21:09:44.415596+0000","last_fullsized":"2023-10-06T21:09:44.415596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442183+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:44.442183+0000","last_peered":"2023-10-06T21:08:44.442183+0000","last_clean":"2023-10-06T21:08:44.442183+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:44.442183+0000","last_undegraded":"2023-10-06T21:08:44.442183+0000","last_fullsized":"2023-10-06T21:08:44.442183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.454229+0000","last_change":"2023-10-06T21:08:45.453710+0000","last_active":"2023-10-06T21:08:45.454229+0000","last_peered":"2023-10-06T21:08:45.454229+0000","last_clean":"2023-10-06T21:08:45.454229+0000","last_became_active":"2023-10-06T21:08:45.447386+0000","last_became_peered":"2023-10-06T21:08:45.447386+0000","last_unstale":"2023-10-06T21:08:45.454229+0000","last_undegraded":"2023-10-06T21:08:45.454229+0000","last_fullsized":"2023-10-06T21:08:45.454229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:27.659601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.426691+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:09:44.426691+0000","last_peered":"2023-10-06T21:09:44.426691+0000","last_clean":"2023-10-06T21:09:44.426691+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:09:44.426691+0000","last_undegraded":"2023-10-06T21:09:44.426691+0000","last_fullsized":"2023-10-06T21:09:44.426691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.445149+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:45.445149+0000","last_peered":"2023-10-06T21:08:45.445149+0000","last_clean":"2023-10-06T21:08:45.445149+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:45.445149+0000","last_undegraded":"2023-10-06T21:08:45.445149+0000","last_fullsized":"2023-10-06T21:08:45.445149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417794+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:09:44.417794+0000","last_peered":"2023-10-06T21:09:44.417794+0000","last_clean":"2023-10-06T21:09:44.417794+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:09:44.417794+0000","last_undegraded":"2023-10-06T21:09:44.417794+0000","last_fullsized":"2023-10-06T21:09:44.417794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436021+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:44.436021+0000","last_peered":"2023-10-06T21:08:44.436021+0000","last_clean":"2023-10-06T21:08:44.436021+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:44.436021+0000","last_undegraded":"2023-10-06T21:08:44.436021+0000","last_fullsized":"2023-10-06T21:08:44.436021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418795+0000","last_change":"2023-10-06T21:08:45.613497+0000","last_active":"2023-10-06T21:09:44.418795+0000","last_peered":"2023-10-06T21:09:44.418795+0000","last_clean":"2023-10-06T21:09:44.418795+0000","last_became_active":"2023-10-06T21:08:45.613012+0000","last_became_peered":"2023-10-06T21:08:45.613012+0000","last_unstale":"2023-10-06T21:09:44.418795+0000","last_undegraded":"2023-10-06T21:09:44.418795+0000","last_fullsized":"2023-10-06T21:09:44.418795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:04:42.351299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443069+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:44.443069+0000","last_peered":"2023-10-06T21:08:44.443069+0000","last_clean":"2023-10-06T21:08:44.443069+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:44.443069+0000","last_undegraded":"2023-10-06T21:08:44.443069+0000","last_fullsized":"2023-10-06T21:08:44.443069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.384523+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:09:44.384523+0000","last_peered":"2023-10-06T21:09:44.384523+0000","last_clean":"2023-10-06T21:09:44.384523+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:09:44.384523+0000","last_undegraded":"2023-10-06T21:09:44.384523+0000","last_fullsized":"2023-10-06T21:09:44.384523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434716+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:44.434716+0000","last_peered":"2023-10-06T21:08:44.434716+0000","last_clean":"2023-10-06T21:08:44.434716+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:44.434716+0000","last_undegraded":"2023-10-06T21:08:44.434716+0000","last_fullsized":"2023-10-06T21:08:44.434716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423534+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:09:44.423534+0000","last_peered":"2023-10-06T21:09:44.423534+0000","last_clean":"2023-10-06T21:09:44.423534+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:09:44.423534+0000","last_undegraded":"2023-10-06T21:09:44.423534+0000","last_fullsized":"2023-10-06T21:09:44.423534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.457037+0000","last_change":"2023-10-06T21:08:45.456629+0000","last_active":"2023-10-06T21:08:45.457037+0000","last_peered":"2023-10-06T21:08:45.457037+0000","last_clean":"2023-10-06T21:08:45.457037+0000","last_became_active":"2023-10-06T21:08:45.449565+0000","last_became_peered":"2023-10-06T21:08:45.449565+0000","last_unstale":"2023-10-06T21:08:45.457037+0000","last_undegraded":"2023-10-06T21:08:45.457037+0000","last_fullsized":"2023-10-06T21:08:45.457037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:23:54.912161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422471+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:09:44.422471+0000","last_peered":"2023-10-06T21:09:44.422471+0000","last_clean":"2023-10-06T21:09:44.422471+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:09:44.422471+0000","last_undegraded":"2023-10-06T21:09:44.422471+0000","last_fullsized":"2023-10-06T21:09:44.422471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440584+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:44.440584+0000","last_peered":"2023-10-06T21:08:44.440584+0000","last_clean":"2023-10-06T21:08:44.440584+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:44.440584+0000","last_undegraded":"2023-10-06T21:08:44.440584+0000","last_fullsized":"2023-10-06T21:08:44.440584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422433+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:09:44.422433+0000","last_peered":"2023-10-06T21:09:44.422433+0000","last_clean":"2023-10-06T21:09:44.422433+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:09:44.422433+0000","last_undegraded":"2023-10-06T21:09:44.422433+0000","last_fullsized":"2023-10-06T21:09:44.422433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444225+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:44.444225+0000","last_peered":"2023-10-06T21:08:44.444225+0000","last_clean":"2023-10-06T21:08:44.444225+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:44.444225+0000","last_undegraded":"2023-10-06T21:08:44.444225+0000","last_fullsized":"2023-10-06T21:08:44.444225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419710+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:09:44.419710+0000","last_peered":"2023-10-06T21:09:44.419710+0000","last_clean":"2023-10-06T21:09:44.419710+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:09:44.419710+0000","last_undegraded":"2023-10-06T21:09:44.419710+0000","last_fullsized":"2023-10-06T21:09:44.419710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438547+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:44.438547+0000","last_peered":"2023-10-06T21:08:44.438547+0000","last_clean":"2023-10-06T21:08:44.438547+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:44.438547+0000","last_undegraded":"2023-10-06T21:08:44.438547+0000","last_fullsized":"2023-10-06T21:08:44.438547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419495+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:09:44.419495+0000","last_peered":"2023-10-06T21:09:44.419495+0000","last_clean":"2023-10-06T21:09:44.419495+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:09:44.419495+0000","last_undegraded":"2023-10-06T21:09:44.419495+0000","last_fullsized":"2023-10-06T21:09:44.419495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437500+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:44.437500+0000","last_peered":"2023-10-06T21:08:44.437500+0000","last_clean":"2023-10-06T21:08:44.437500+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:44.437500+0000","last_undegraded":"2023-10-06T21:08:44.437500+0000","last_fullsized":"2023-10-06T21:08:44.437500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415401+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:09:44.415401+0000","last_peered":"2023-10-06T21:09:44.415401+0000","last_clean":"2023-10-06T21:09:44.415401+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:09:44.415401+0000","last_undegraded":"2023-10-06T21:09:44.415401+0000","last_fullsized":"2023-10-06T21:09:44.415401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411059+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:09:44.411059+0000","last_peered":"2023-10-06T21:09:44.411059+0000","last_clean":"2023-10-06T21:09:44.411059+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:09:44.411059+0000","last_undegraded":"2023-10-06T21:09:44.411059+0000","last_fullsized":"2023-10-06T21:09:44.411059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412660+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:09:44.412660+0000","last_peered":"2023-10-06T21:09:44.412660+0000","last_clean":"2023-10-06T21:09:44.412660+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:09:44.412660+0000","last_undegraded":"2023-10-06T21:09:44.412660+0000","last_fullsized":"2023-10-06T21:09:44.412660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419648+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:09:44.419648+0000","last_peered":"2023-10-06T21:09:44.419648+0000","last_clean":"2023-10-06T21:09:44.419648+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:09:44.419648+0000","last_undegraded":"2023-10-06T21:09:44.419648+0000","last_fullsized":"2023-10-06T21:09:44.419648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422939+0000","last_change":"2023-10-06T21:08:45.448913+0000","last_active":"2023-10-06T21:09:44.422939+0000","last_peered":"2023-10-06T21:09:44.422939+0000","last_clean":"2023-10-06T21:09:44.422939+0000","last_became_active":"2023-10-06T21:08:45.448485+0000","last_became_peered":"2023-10-06T21:08:45.448485+0000","last_unstale":"2023-10-06T21:09:44.422939+0000","last_undegraded":"2023-10-06T21:09:44.422939+0000","last_fullsized":"2023-10-06T21:09:44.422939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:42:07.732211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422296+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:09:44.422296+0000","last_peered":"2023-10-06T21:09:44.422296+0000","last_clean":"2023-10-06T21:09:44.422296+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:09:44.422296+0000","last_undegraded":"2023-10-06T21:09:44.422296+0000","last_fullsized":"2023-10-06T21:09:44.422296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407447+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:09:44.407447+0000","last_peered":"2023-10-06T21:09:44.407447+0000","last_clean":"2023-10-06T21:09:44.407447+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:09:44.407447+0000","last_undegraded":"2023-10-06T21:09:44.407447+0000","last_fullsized":"2023-10-06T21:09:44.407447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.414676+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:09:44.414676+0000","last_peered":"2023-10-06T21:09:44.414676+0000","last_clean":"2023-10-06T21:09:44.414676+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:09:44.414676+0000","last_undegraded":"2023-10-06T21:09:44.414676+0000","last_fullsized":"2023-10-06T21:09:44.414676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416177+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:09:44.416177+0000","last_peered":"2023-10-06T21:09:44.416177+0000","last_clean":"2023-10-06T21:09:44.416177+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:09:44.416177+0000","last_undegraded":"2023-10-06T21:09:44.416177+0000","last_fullsized":"2023-10-06T21:09:44.416177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418196+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:09:44.418196+0000","last_peered":"2023-10-06T21:09:44.418196+0000","last_clean":"2023-10-06T21:09:44.418196+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:09:44.418196+0000","last_undegraded":"2023-10-06T21:09:44.418196+0000","last_fullsized":"2023-10-06T21:09:44.418196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417342+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:09:44.417342+0000","last_peered":"2023-10-06T21:09:44.417342+0000","last_clean":"2023-10-06T21:09:44.417342+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:09:44.417342+0000","last_undegraded":"2023-10-06T21:09:44.417342+0000","last_fullsized":"2023-10-06T21:09:44.417342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423644+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:09:44.423644+0000","last_peered":"2023-10-06T21:09:44.423644+0000","last_clean":"2023-10-06T21:09:44.423644+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:09:44.423644+0000","last_undegraded":"2023-10-06T21:09:44.423644+0000","last_fullsized":"2023-10-06T21:09:44.423644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423037+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:09:44.423037+0000","last_peered":"2023-10-06T21:09:44.423037+0000","last_clean":"2023-10-06T21:09:44.423037+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:09:44.423037+0000","last_undegraded":"2023-10-06T21:09:44.423037+0000","last_fullsized":"2023-10-06T21:09:44.423037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409451+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:09:44.409451+0000","last_peered":"2023-10-06T21:09:44.409451+0000","last_clean":"2023-10-06T21:09:44.409451+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:09:44.409451+0000","last_undegraded":"2023-10-06T21:09:44.409451+0000","last_fullsized":"2023-10-06T21:09:44.409451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64397,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":623,"num_read_kb":375,"num_write":161,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":57530,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.40699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.623}]},{"osd":3,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.47399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77200000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91900000000000004,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.92800000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.875},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.83399999999999996,"15min":0.81699999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.80800000000000005},"min":{"1min":0.63400000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.96899999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.90300000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.499}]},{"osd":2,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":1.089,"15min":1.089},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.82099999999999995,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.873,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.85199999999999998,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84899999999999998,"5min":1.26,"15min":1.26},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.048,"15min":1.048},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.56899999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.90900000000000003,"5min":1,"15min":1},"last":0.76400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":456799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51100000000000001,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.441},{"interface":"front","average":{"1min":0.44,"5min":0.56299999999999994,"15min":0.58699999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":1.014,"15min":1.014},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.083,"5min":1.347,"15min":1.347},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.78700000000000003,"15min":0.82499999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.79500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1240000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68200000000000005,"15min":0.70099999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.501},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81399999999999995,"15min":0.83799999999999997},"min":{"1min":0.499,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.085,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.93600000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85599999999999998,"15min":0.88700000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0089999999999999,"5min":1.226,"15min":1.226},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":1.0069999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2490000000000001,"5min":1.542,"15min":1.542},"last":0.89600000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":0.86699999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.125,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.056,"5min":1.365,"15min":1.365},"last":0.98899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.95799999999999996,"15min":0.94999999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1359999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Fri Oct 6 21:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.68899999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71999999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.53400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47999999999999998},"min":{"1min":0.39200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92900000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63},"min":{"1min":0.60399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93200000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.46300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93100000000000005,"5min":1.087,"15min":1.087},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.59399999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":1.038,"15min":1.038},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.65300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0009999999999999,"5min":1.155,"15min":1.155},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.71299999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.93899999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.63100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77600000000000002,"15min":0.754},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97299999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95399999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1049326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.443,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.752},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.76500000000000001,"5min":1.002,"15min":1.002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82299999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.60899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.48299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.877,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.012,"15min":1.012},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.55600000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.879},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.84699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.46,"5min":5.8620000000000001,"15min":5.8620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.93400000000000005,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.56699999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98299999999999998,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.58099999999999996,"5min":0.38,"15min":0.38},"max":{"1min":1.1679999999999999,"5min":1.948,"15min":1.948},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2749999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3879999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":5.9290000000000003,"15min":5.9290000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.355,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.53800000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.036,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.85099999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4790000000000001,"5min":5.0199999999999996,"15min":5.0199999999999996},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.497,"15min":0.497},"max":{"1min":4.9420000000000002,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":0.98499999999999999},"min":{"1min":0.76800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.127,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.622,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.206,"5min":2.258,"15min":2.258},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86899999999999999,"15min":0.83399999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.8979999999999997,"5min":4.8979999999999997,"15min":4.8979999999999997},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.29599999999999999},"max":{"1min":0.76900000000000002,"5min":6.8949999999999996,"15min":6.8949999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32000000000000001},"max":{"1min":0.81000000000000005,"5min":1.105,"15min":1.105},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:09:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.318,"15min":0.318},"max":{"1min":0.81599999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.48599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.877,"15min":0.877},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.59399999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.64100000000000001,"15min":0.60899999999999999},"min":{"1min":0.48099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78800000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64800000000000002},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":2.714,"15min":2.714},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.498},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.71399999999999997,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.999,"5min":1.022,"15min":1.022},"last":0.82699999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.622},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.63300000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.67200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.64500000000000002,"15min":0.58099999999999996},"min":{"1min":0.45400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92400000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.77800000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97399999999999998,"5min":1.216,"15min":1.216},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.145},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.238,"5min":1.397,"15min":1.397},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68200000000000005,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.58099999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.27,"5min":1.375,"15min":1.375},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.745},{"interface":"front","average":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":1.1100000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1000000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80700000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.45800000000000002,"15min":0.45000000000000001},"min":{"1min":0.39800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.502,"15min":0.48699999999999999},"min":{"1min":0.38700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.65400000000000003},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98799999999999999,"5min":1.139,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95099999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.58299999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47499999999999998,"15min":0.432},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.55300000000000005},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98199999999999998,"5min":1.002,"15min":1.002},"last":0.748},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69499999999999995,"15min":0.64900000000000002},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93600000000000005,"5min":1.103,"15min":1.103},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72099999999999997,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":1.129,"15min":1.129},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.71299999999999997},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1579999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.92400000000000004,"15min":0.96399999999999997},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0860000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.98099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.395,"15min":1.395},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.129,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78600000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":516881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.36099999999999999},"min":{"1min":0.32100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53700000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.36599999999999999},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.56100000000000005},"min":{"1min":0.45800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.371,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.435}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.58099999999999996},"min":{"1min":0.57499999999999996,"5min":0.37,"15min":0.37},"max":{"1min":0.89800000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.66100000000000003,"15min":0.60499999999999998},"min":{"1min":0.56499999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.94399999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.90100000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.752,"15min":0.745},"min":{"1min":0.47399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1048148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.38900000000000001},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.876,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92800000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77000000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90000000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.876},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:10:05.942 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-06T21:10:05.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T21:10:06.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.176+0000 7fa73e3ff700 1 -- 172.21.15.26:0/2347239813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102080 msgr2=0x7fa738102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.176+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/2347239813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102080 0x7fa738102500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009990 tx=0x7fa72c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 -- 172.21.15.26:0/2347239813 shutdown_connections 2023-10-06T21:10:06.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/2347239813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102a40 0x7fa738109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/2347239813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa7381012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/2347239813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102080 0x7fa738102500 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 -- 172.21.15.26:0/2347239813 >> 172.21.15.26:0/2347239813 conn(0x7fa7380fc420 msgr2=0x7fa7380fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:06.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 -- 172.21.15.26:0/2347239813 shutdown_connections 2023-10-06T21:10:06.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.177+0000 7fa73e3ff700 1 -- 172.21.15.26:0/2347239813 wait complete. 2023-10-06T21:10:06.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 Processor -- start 2023-10-06T21:10:06.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 -- start start 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102080 0x7fa738197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102a40 0x7fa73819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73819e770 con 0x7fa738100ea0 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73819e8b0 con 0x7fa738102080 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa73e3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73819ea40 con 0x7fa738102a40 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.178+0000 7fa737fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39018/0 (socket says 172.21.15.26:39018) 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa73c99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102a40 0x7fa73819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 -- 172.21.15.26:0/3515951744 learned_addr learned my addr 172.21.15.26:0/3515951744 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa7377fe700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102080 0x7fa738197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 -- 172.21.15.26:0/3515951744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102a40 msgr2=0x7fa73819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102a40 0x7fa73819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 -- 172.21.15.26:0/3515951744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102080 msgr2=0x7fa738197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102080 0x7fa738197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 -- 172.21.15.26:0/3515951744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009c40 con 0x7fa738100ea0 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa737fff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e970 tx=0x7fa72800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:06.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728009790 con 0x7fa738100ea0 2023-10-06T21:10:06.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72800c7c0 con 0x7fa738100ea0 2023-10-06T21:10:06.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728003c20 con 0x7fa738100ea0 2023-10-06T21:10:06.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.179+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0095f0 con 0x7fa738100ea0 2023-10-06T21:10:06.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.180+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73819f050 con 0x7fa738100ea0 2023-10-06T21:10:06.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.181+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7381084c0 con 0x7fa738100ea0 2023-10-06T21:10:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.183+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa72800c930 con 0x7fa738100ea0 2023-10-06T21:10:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.184+0000 7fa7357fa700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 0x7fa7200a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.184+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa728014070 con 0x7fa738100ea0 2023-10-06T21:10:06.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.184+0000 7fa7377fe700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 0x7fa7200a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.185+0000 7fa7377fe700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 0x7fa7200a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00b2e0 tx=0x7fa72c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:06.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.186+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa7280966a0 con 0x7fa738100ea0 2023-10-06T21:10:06.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:06 smithi026 ceph-mon[98490]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:06.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:06 smithi078 ceph-mon[134896]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:06.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.395+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa73802d2f0 con 0x7fa7200a14d0 2023-10-06T21:10:06.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.407+0000 7fa7357fa700 1 -- 172.21.15.26:0/3515951744 <== mgr.14148 v2:172.21.15.26:6800/2830651730 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345760 (secure 0 0 0) 0x7fa73802d2f0 con 0x7fa7200a14d0 2023-10-06T21:10:06.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 msgr2=0x7fa7200a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 0x7fa7200a3990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00b2e0 tx=0x7fa72c005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 msgr2=0x7fa738196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e970 tx=0x7fa72800ec80 comp rx=0 tx=0).stop 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 shutdown_connections 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa738102a40 0x7fa73819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa7200a14d0 0x7fa7200a3990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa738100ea0 0x7fa738196bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 --2- 172.21.15.26:0/3515951744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa738102080 0x7fa738197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.411+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 >> 172.21.15.26:0/3515951744 conn(0x7fa7380fc420 msgr2=0x7fa738105600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.412+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 shutdown_connections 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.412+0000 7fa73e3ff700 1 -- 172.21.15.26:0/3515951744 wait complete. 2023-10-06T21:10:06.417 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-10-06T21:10:06.433 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-06T21:10:04.399860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655011,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":759,"num_read_kb":494,"num_write":344,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":3009370,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337448,"kb_used_data":13256,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604376,"statfs":{"total":1151940427776,"available":1151594881024,"internally_reserved":0,"allocated":13574144,"data_stored":7436474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":316,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":41,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002567"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427530+0000","last_change":"2023-10-06T21:08:16.519346+0000","last_active":"2023-10-06T21:09:44.427530+0000","last_peered":"2023-10-06T21:09:44.427530+0000","last_clean":"2023-10-06T21:09:44.427530+0000","last_became_active":"2023-10-06T21:08:16.519054+0000","last_became_peered":"2023-10-06T21:08:16.519054+0000","last_unstale":"2023-10-06T21:09:44.427530+0000","last_undegraded":"2023-10-06T21:09:44.427530+0000","last_fullsized":"2023-10-06T21:09:44.427530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:11.569420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415677+0000","last_change":"2023-10-06T21:08:16.523738+0000","last_active":"2023-10-06T21:09:44.415677+0000","last_peered":"2023-10-06T21:09:44.415677+0000","last_clean":"2023-10-06T21:09:44.415677+0000","last_became_active":"2023-10-06T21:08:16.523417+0000","last_became_peered":"2023-10-06T21:08:16.523417+0000","last_unstale":"2023-10-06T21:09:44.415677+0000","last_undegraded":"2023-10-06T21:09:44.415677+0000","last_fullsized":"2023-10-06T21:09:44.415677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:25:56.283599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421162+0000","last_change":"2023-10-06T21:08:16.518190+0000","last_active":"2023-10-06T21:09:44.421162+0000","last_peered":"2023-10-06T21:09:44.421162+0000","last_clean":"2023-10-06T21:09:44.421162+0000","last_became_active":"2023-10-06T21:08:16.517860+0000","last_became_peered":"2023-10-06T21:08:16.517860+0000","last_unstale":"2023-10-06T21:09:44.421162+0000","last_undegraded":"2023-10-06T21:09:44.421162+0000","last_fullsized":"2023-10-06T21:09:44.421162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:47:56.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417045+0000","last_change":"2023-10-06T21:08:45.451527+0000","last_active":"2023-10-06T21:09:44.417045+0000","last_peered":"2023-10-06T21:09:44.417045+0000","last_clean":"2023-10-06T21:09:44.417045+0000","last_became_active":"2023-10-06T21:08:45.451219+0000","last_became_peered":"2023-10-06T21:08:45.451219+0000","last_unstale":"2023-10-06T21:09:44.417045+0000","last_undegraded":"2023-10-06T21:09:44.417045+0000","last_fullsized":"2023-10-06T21:09:44.417045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T09:03:42.551771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413140+0000","last_change":"2023-10-06T21:08:16.522797+0000","last_active":"2023-10-06T21:09:44.413140+0000","last_peered":"2023-10-06T21:09:44.413140+0000","last_clean":"2023-10-06T21:09:44.413140+0000","last_became_active":"2023-10-06T21:08:16.522509+0000","last_became_peered":"2023-10-06T21:08:16.522509+0000","last_unstale":"2023-10-06T21:09:44.413140+0000","last_undegraded":"2023-10-06T21:09:44.413140+0000","last_fullsized":"2023-10-06T21:09:44.413140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:48:44.539936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417327+0000","last_change":"2023-10-06T21:08:16.513607+0000","last_active":"2023-10-06T21:09:44.417327+0000","last_peered":"2023-10-06T21:09:44.417327+0000","last_clean":"2023-10-06T21:09:44.417327+0000","last_became_active":"2023-10-06T21:08:16.513307+0000","last_became_peered":"2023-10-06T21:08:16.513307+0000","last_unstale":"2023-10-06T21:09:44.417327+0000","last_undegraded":"2023-10-06T21:09:44.417327+0000","last_fullsized":"2023-10-06T21:09:44.417327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:21:37.462496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416179+0000","last_change":"2023-10-06T21:08:16.529938+0000","last_active":"2023-10-06T21:09:44.416179+0000","last_peered":"2023-10-06T21:09:44.416179+0000","last_clean":"2023-10-06T21:09:44.416179+0000","last_became_active":"2023-10-06T21:08:16.529677+0000","last_became_peered":"2023-10-06T21:08:16.529677+0000","last_unstale":"2023-10-06T21:09:44.416179+0000","last_undegraded":"2023-10-06T21:09:44.416179+0000","last_fullsized":"2023-10-06T21:09:44.416179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:32:44.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417668+0000","last_change":"2023-10-06T21:08:16.521870+0000","last_active":"2023-10-06T21:09:44.417668+0000","last_peered":"2023-10-06T21:09:44.417668+0000","last_clean":"2023-10-06T21:09:44.417668+0000","last_became_active":"2023-10-06T21:08:16.521474+0000","last_became_peered":"2023-10-06T21:08:16.521474+0000","last_unstale":"2023-10-06T21:09:44.417668+0000","last_undegraded":"2023-10-06T21:09:44.417668+0000","last_fullsized":"2023-10-06T21:09:44.417668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:00:56.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418346+0000","last_change":"2023-10-06T21:08:45.451859+0000","last_active":"2023-10-06T21:09:44.418346+0000","last_peered":"2023-10-06T21:09:44.418346+0000","last_clean":"2023-10-06T21:09:44.418346+0000","last_became_active":"2023-10-06T21:08:45.451525+0000","last_became_peered":"2023-10-06T21:08:45.451525+0000","last_unstale":"2023-10-06T21:09:44.418346+0000","last_undegraded":"2023-10-06T21:09:44.418346+0000","last_fullsized":"2023-10-06T21:09:44.418346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:39:52.350672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411831+0000","last_change":"2023-10-06T21:08:16.518004+0000","last_active":"2023-10-06T21:09:44.411831+0000","last_peered":"2023-10-06T21:09:44.411831+0000","last_clean":"2023-10-06T21:09:44.411831+0000","last_became_active":"2023-10-06T21:08:16.517580+0000","last_became_peered":"2023-10-06T21:08:16.517580+0000","last_unstale":"2023-10-06T21:09:44.411831+0000","last_undegraded":"2023-10-06T21:09:44.411831+0000","last_fullsized":"2023-10-06T21:09:44.411831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:10:08.154902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419428+0000","last_change":"2023-10-06T21:08:16.519846+0000","last_active":"2023-10-06T21:09:44.419428+0000","last_peered":"2023-10-06T21:09:44.419428+0000","last_clean":"2023-10-06T21:09:44.419428+0000","last_became_active":"2023-10-06T21:08:16.519544+0000","last_became_peered":"2023-10-06T21:08:16.519544+0000","last_unstale":"2023-10-06T21:09:44.419428+0000","last_undegraded":"2023-10-06T21:09:44.419428+0000","last_fullsized":"2023-10-06T21:09:44.419428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:20:50.517104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410519+0000","last_change":"2023-10-06T21:08:16.515642+0000","last_active":"2023-10-06T21:09:44.410519+0000","last_peered":"2023-10-06T21:09:44.410519+0000","last_clean":"2023-10-06T21:09:44.410519+0000","last_became_active":"2023-10-06T21:08:16.515327+0000","last_became_peered":"2023-10-06T21:08:16.515327+0000","last_unstale":"2023-10-06T21:09:44.410519+0000","last_undegraded":"2023-10-06T21:09:44.410519+0000","last_fullsized":"2023-10-06T21:09:44.410519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:56:28.540188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.421301+0000","last_change":"2023-10-06T21:08:16.530100+0000","last_active":"2023-10-06T21:09:44.421301+0000","last_peered":"2023-10-06T21:09:44.421301+0000","last_clean":"2023-10-06T21:09:44.421301+0000","last_became_active":"2023-10-06T21:08:16.528026+0000","last_became_peered":"2023-10-06T21:08:16.528026+0000","last_unstale":"2023-10-06T21:09:44.421301+0000","last_undegraded":"2023-10-06T21:09:44.421301+0000","last_fullsized":"2023-10-06T21:09:44.421301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:22:20.738632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420910+0000","last_change":"2023-10-06T21:08:16.516232+0000","last_active":"2023-10-06T21:09:44.420910+0000","last_peered":"2023-10-06T21:09:44.420910+0000","last_clean":"2023-10-06T21:09:44.420910+0000","last_became_active":"2023-10-06T21:08:16.515904+0000","last_became_peered":"2023-10-06T21:08:16.515904+0000","last_unstale":"2023-10-06T21:09:44.420910+0000","last_undegraded":"2023-10-06T21:09:44.420910+0000","last_fullsized":"2023-10-06T21:09:44.420910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:21.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418350+0000","last_change":"2023-10-06T21:08:16.517077+0000","last_active":"2023-10-06T21:09:44.418350+0000","last_peered":"2023-10-06T21:09:44.418350+0000","last_clean":"2023-10-06T21:09:44.418350+0000","last_became_active":"2023-10-06T21:08:16.516606+0000","last_became_peered":"2023-10-06T21:08:16.516606+0000","last_unstale":"2023-10-06T21:09:44.418350+0000","last_undegraded":"2023-10-06T21:09:44.418350+0000","last_fullsized":"2023-10-06T21:09:44.418350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:17:49.182243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425058+0000","last_change":"2023-10-06T21:08:16.516448+0000","last_active":"2023-10-06T21:09:44.425058+0000","last_peered":"2023-10-06T21:09:44.425058+0000","last_clean":"2023-10-06T21:09:44.425058+0000","last_became_active":"2023-10-06T21:08:16.515898+0000","last_became_peered":"2023-10-06T21:08:16.515898+0000","last_unstale":"2023-10-06T21:09:44.425058+0000","last_undegraded":"2023-10-06T21:09:44.425058+0000","last_fullsized":"2023-10-06T21:09:44.425058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:22.306334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.427610+0000","last_change":"2023-10-06T21:08:16.519241+0000","last_active":"2023-10-06T21:09:44.427610+0000","last_peered":"2023-10-06T21:09:44.427610+0000","last_clean":"2023-10-06T21:09:44.427610+0000","last_became_active":"2023-10-06T21:08:16.518909+0000","last_became_peered":"2023-10-06T21:08:16.518909+0000","last_unstale":"2023-10-06T21:09:44.427610+0000","last_undegraded":"2023-10-06T21:09:44.427610+0000","last_fullsized":"2023-10-06T21:09:44.427610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:40:06.514281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.424873+0000","last_change":"2023-10-06T21:08:16.527363+0000","last_active":"2023-10-06T21:09:44.424873+0000","last_peered":"2023-10-06T21:09:44.424873+0000","last_clean":"2023-10-06T21:09:44.424873+0000","last_became_active":"2023-10-06T21:08:16.527067+0000","last_became_peered":"2023-10-06T21:08:16.527067+0000","last_unstale":"2023-10-06T21:09:44.424873+0000","last_undegraded":"2023-10-06T21:09:44.424873+0000","last_fullsized":"2023-10-06T21:09:44.424873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:49:41.258887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.413246+0000","last_change":"2023-10-06T21:08:16.522508+0000","last_active":"2023-10-06T21:09:44.413246+0000","last_peered":"2023-10-06T21:09:44.413246+0000","last_clean":"2023-10-06T21:09:44.413246+0000","last_became_active":"2023-10-06T21:08:16.522188+0000","last_became_peered":"2023-10-06T21:08:16.522188+0000","last_unstale":"2023-10-06T21:09:44.413246+0000","last_undegraded":"2023-10-06T21:09:44.413246+0000","last_fullsized":"2023-10-06T21:09:44.413246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.106397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439306+0000","last_change":"2023-10-06T21:08:15.516893+0000","last_active":"2023-10-06T21:08:44.439306+0000","last_peered":"2023-10-06T21:08:44.439306+0000","last_clean":"2023-10-06T21:08:44.439306+0000","last_became_active":"2023-10-06T21:08:15.516549+0000","last_became_peered":"2023-10-06T21:08:15.516549+0000","last_unstale":"2023-10-06T21:08:44.439306+0000","last_undegraded":"2023-10-06T21:08:44.439306+0000","last_fullsized":"2023-10-06T21:08:44.439306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:53:30.105621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416382+0000","last_change":"2023-10-06T21:08:16.520943+0000","last_active":"2023-10-06T21:09:44.416382+0000","last_peered":"2023-10-06T21:09:44.416382+0000","last_clean":"2023-10-06T21:09:44.416382+0000","last_became_active":"2023-10-06T21:08:16.520568+0000","last_became_peered":"2023-10-06T21:08:16.520568+0000","last_unstale":"2023-10-06T21:09:44.416382+0000","last_undegraded":"2023-10-06T21:09:44.416382+0000","last_fullsized":"2023-10-06T21:09:44.416382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:27.524545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.435884+0000","last_change":"2023-10-06T21:08:15.525625+0000","last_active":"2023-10-06T21:08:44.435884+0000","last_peered":"2023-10-06T21:08:44.435884+0000","last_clean":"2023-10-06T21:08:44.435884+0000","last_became_active":"2023-10-06T21:08:15.525296+0000","last_became_peered":"2023-10-06T21:08:15.525296+0000","last_unstale":"2023-10-06T21:08:44.435884+0000","last_undegraded":"2023-10-06T21:08:44.435884+0000","last_fullsized":"2023-10-06T21:08:44.435884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:08:27.910461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429639+0000","last_change":"2023-10-06T21:08:16.528193+0000","last_active":"2023-10-06T21:09:44.429639+0000","last_peered":"2023-10-06T21:09:44.429639+0000","last_clean":"2023-10-06T21:09:44.429639+0000","last_became_active":"2023-10-06T21:08:16.527872+0000","last_became_peered":"2023-10-06T21:08:16.527872+0000","last_unstale":"2023-10-06T21:09:44.429639+0000","last_undegraded":"2023-10-06T21:09:44.429639+0000","last_fullsized":"2023-10-06T21:09:44.429639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:04.078279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440673+0000","last_change":"2023-10-06T21:08:15.522871+0000","last_active":"2023-10-06T21:08:44.440673+0000","last_peered":"2023-10-06T21:08:44.440673+0000","last_clean":"2023-10-06T21:08:44.440673+0000","last_became_active":"2023-10-06T21:08:15.522571+0000","last_became_peered":"2023-10-06T21:08:15.522571+0000","last_unstale":"2023-10-06T21:08:44.440673+0000","last_undegraded":"2023-10-06T21:08:44.440673+0000","last_fullsized":"2023-10-06T21:08:44.440673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:50.837381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.420872+0000","last_change":"2023-10-06T21:08:16.524720+0000","last_active":"2023-10-06T21:09:44.420872+0000","last_peered":"2023-10-06T21:09:44.420872+0000","last_clean":"2023-10-06T21:09:44.420872+0000","last_became_active":"2023-10-06T21:08:16.524413+0000","last_became_peered":"2023-10-06T21:08:16.524413+0000","last_unstale":"2023-10-06T21:09:44.420872+0000","last_undegraded":"2023-10-06T21:09:44.420872+0000","last_fullsized":"2023-10-06T21:09:44.420872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:53:50.485419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441604+0000","last_change":"2023-10-06T21:08:15.516662+0000","last_active":"2023-10-06T21:08:44.441604+0000","last_peered":"2023-10-06T21:08:44.441604+0000","last_clean":"2023-10-06T21:08:44.441604+0000","last_became_active":"2023-10-06T21:08:15.516347+0000","last_became_peered":"2023-10-06T21:08:15.516347+0000","last_unstale":"2023-10-06T21:08:44.441604+0000","last_undegraded":"2023-10-06T21:08:44.441604+0000","last_fullsized":"2023-10-06T21:08:44.441604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:33:56.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412714+0000","last_change":"2023-10-06T21:08:16.511677+0000","last_active":"2023-10-06T21:09:44.412714+0000","last_peered":"2023-10-06T21:09:44.412714+0000","last_clean":"2023-10-06T21:09:44.412714+0000","last_became_active":"2023-10-06T21:08:16.511355+0000","last_became_peered":"2023-10-06T21:08:16.511355+0000","last_unstale":"2023-10-06T21:09:44.412714+0000","last_undegraded":"2023-10-06T21:09:44.412714+0000","last_fullsized":"2023-10-06T21:09:44.412714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:13.369899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":127,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.013129+0000","last_change":"2023-10-06T21:08:15.523947+0000","last_active":"2023-10-06T21:10:01.013129+0000","last_peered":"2023-10-06T21:10:01.013129+0000","last_clean":"2023-10-06T21:10:01.013129+0000","last_became_active":"2023-10-06T21:08:15.523642+0000","last_became_peered":"2023-10-06T21:08:15.523642+0000","last_unstale":"2023-10-06T21:10:01.013129+0000","last_undegraded":"2023-10-06T21:10:01.013129+0000","last_fullsized":"2023-10-06T21:10:01.013129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:17.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415642+0000","last_change":"2023-10-06T21:08:16.514907+0000","last_active":"2023-10-06T21:09:44.415642+0000","last_peered":"2023-10-06T21:09:44.415642+0000","last_clean":"2023-10-06T21:09:44.415642+0000","last_became_active":"2023-10-06T21:08:16.512560+0000","last_became_peered":"2023-10-06T21:08:16.512560+0000","last_unstale":"2023-10-06T21:09:44.415642+0000","last_undegraded":"2023-10-06T21:09:44.415642+0000","last_fullsized":"2023-10-06T21:09:44.415642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:02:18.211658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443912+0000","last_change":"2023-10-06T21:08:15.519993+0000","last_active":"2023-10-06T21:08:44.443912+0000","last_peered":"2023-10-06T21:08:44.443912+0000","last_clean":"2023-10-06T21:08:44.443912+0000","last_became_active":"2023-10-06T21:08:15.519664+0000","last_became_peered":"2023-10-06T21:08:15.519664+0000","last_unstale":"2023-10-06T21:08:44.443912+0000","last_undegraded":"2023-10-06T21:08:44.443912+0000","last_fullsized":"2023-10-06T21:08:44.443912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:09:10.380747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418357+0000","last_change":"2023-10-06T21:08:16.520491+0000","last_active":"2023-10-06T21:09:44.418357+0000","last_peered":"2023-10-06T21:09:44.418357+0000","last_clean":"2023-10-06T21:09:44.418357+0000","last_became_active":"2023-10-06T21:08:16.520181+0000","last_became_peered":"2023-10-06T21:08:16.520181+0000","last_unstale":"2023-10-06T21:09:44.418357+0000","last_undegraded":"2023-10-06T21:09:44.418357+0000","last_fullsized":"2023-10-06T21:09:44.418357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:04:54.024188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436810+0000","last_change":"2023-10-06T21:08:15.517852+0000","last_active":"2023-10-06T21:08:44.436810+0000","last_peered":"2023-10-06T21:08:44.436810+0000","last_clean":"2023-10-06T21:08:44.436810+0000","last_became_active":"2023-10-06T21:08:15.517452+0000","last_became_peered":"2023-10-06T21:08:15.517452+0000","last_unstale":"2023-10-06T21:08:44.436810+0000","last_undegraded":"2023-10-06T21:08:44.436810+0000","last_fullsized":"2023-10-06T21:08:44.436810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:11:05.332509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423370+0000","last_change":"2023-10-06T21:08:16.525265+0000","last_active":"2023-10-06T21:09:44.423370+0000","last_peered":"2023-10-06T21:09:44.423370+0000","last_clean":"2023-10-06T21:09:44.423370+0000","last_became_active":"2023-10-06T21:08:16.524879+0000","last_became_peered":"2023-10-06T21:08:16.524879+0000","last_unstale":"2023-10-06T21:09:44.423370+0000","last_undegraded":"2023-10-06T21:09:44.423370+0000","last_fullsized":"2023-10-06T21:09:44.423370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:43.422863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437995+0000","last_change":"2023-10-06T21:08:15.521432+0000","last_active":"2023-10-06T21:08:44.437995+0000","last_peered":"2023-10-06T21:08:44.437995+0000","last_clean":"2023-10-06T21:08:44.437995+0000","last_became_active":"2023-10-06T21:08:15.521134+0000","last_became_peered":"2023-10-06T21:08:15.521134+0000","last_unstale":"2023-10-06T21:08:44.437995+0000","last_undegraded":"2023-10-06T21:08:44.437995+0000","last_fullsized":"2023-10-06T21:08:44.437995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:44.165522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.425550+0000","last_change":"2023-10-06T21:08:16.528889+0000","last_active":"2023-10-06T21:09:44.425550+0000","last_peered":"2023-10-06T21:09:44.425550+0000","last_clean":"2023-10-06T21:09:44.425550+0000","last_became_active":"2023-10-06T21:08:16.528366+0000","last_became_peered":"2023-10-06T21:08:16.528366+0000","last_unstale":"2023-10-06T21:09:44.425550+0000","last_undegraded":"2023-10-06T21:09:44.425550+0000","last_fullsized":"2023-10-06T21:09:44.425550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:29:40.907845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012541+0000","last_change":"2023-10-06T21:08:15.520338+0000","last_active":"2023-10-06T21:10:01.012541+0000","last_peered":"2023-10-06T21:10:01.012541+0000","last_clean":"2023-10-06T21:10:01.012541+0000","last_became_active":"2023-10-06T21:08:15.520028+0000","last_became_peered":"2023-10-06T21:08:15.520028+0000","last_unstale":"2023-10-06T21:10:01.012541+0000","last_undegraded":"2023-10-06T21:10:01.012541+0000","last_fullsized":"2023-10-06T21:10:01.012541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:11:56.113407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.428824+0000","last_change":"2023-10-06T21:08:16.528256+0000","last_active":"2023-10-06T21:09:44.428824+0000","last_peered":"2023-10-06T21:09:44.428824+0000","last_clean":"2023-10-06T21:09:44.428824+0000","last_became_active":"2023-10-06T21:08:16.527913+0000","last_became_peered":"2023-10-06T21:08:16.527913+0000","last_unstale":"2023-10-06T21:09:44.428824+0000","last_undegraded":"2023-10-06T21:09:44.428824+0000","last_fullsized":"2023-10-06T21:09:44.428824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:29.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441919+0000","last_change":"2023-10-06T21:08:15.522217+0000","last_active":"2023-10-06T21:08:44.441919+0000","last_peered":"2023-10-06T21:08:44.441919+0000","last_clean":"2023-10-06T21:08:44.441919+0000","last_became_active":"2023-10-06T21:08:15.521177+0000","last_became_peered":"2023-10-06T21:08:15.521177+0000","last_unstale":"2023-10-06T21:08:44.441919+0000","last_undegraded":"2023-10-06T21:08:44.441919+0000","last_fullsized":"2023-10-06T21:08:44.441919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:53:38.006634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.429382+0000","last_change":"2023-10-06T21:08:16.516984+0000","last_active":"2023-10-06T21:09:44.429382+0000","last_peered":"2023-10-06T21:09:44.429382+0000","last_clean":"2023-10-06T21:09:44.429382+0000","last_became_active":"2023-10-06T21:08:16.516618+0000","last_became_peered":"2023-10-06T21:08:16.516618+0000","last_unstale":"2023-10-06T21:09:44.429382+0000","last_undegraded":"2023-10-06T21:09:44.429382+0000","last_fullsized":"2023-10-06T21:09:44.429382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:42:35.918049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438756+0000","last_change":"2023-10-06T21:08:15.526873+0000","last_active":"2023-10-06T21:08:44.438756+0000","last_peered":"2023-10-06T21:08:44.438756+0000","last_clean":"2023-10-06T21:08:44.438756+0000","last_became_active":"2023-10-06T21:08:15.526631+0000","last_became_peered":"2023-10-06T21:08:15.526631+0000","last_unstale":"2023-10-06T21:08:44.438756+0000","last_undegraded":"2023-10-06T21:08:44.438756+0000","last_fullsized":"2023-10-06T21:08:44.438756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:42:08.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417432+0000","last_change":"2023-10-06T21:08:16.519773+0000","last_active":"2023-10-06T21:09:44.417432+0000","last_peered":"2023-10-06T21:09:44.417432+0000","last_clean":"2023-10-06T21:09:44.417432+0000","last_became_active":"2023-10-06T21:08:16.519422+0000","last_became_peered":"2023-10-06T21:08:16.519422+0000","last_unstale":"2023-10-06T21:09:44.417432+0000","last_undegraded":"2023-10-06T21:09:44.417432+0000","last_fullsized":"2023-10-06T21:09:44.417432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:47:15.377892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":277,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.744268+0000","last_change":"2023-10-06T21:08:45.622755+0000","last_active":"2023-10-06T21:10:01.744268+0000","last_peered":"2023-10-06T21:10:01.744268+0000","last_clean":"2023-10-06T21:10:01.744268+0000","last_became_active":"2023-10-06T21:08:45.614102+0000","last_became_peered":"2023-10-06T21:08:45.614102+0000","last_unstale":"2023-10-06T21:10:01.744268+0000","last_undegraded":"2023-10-06T21:10:01.744268+0000","last_fullsized":"2023-10-06T21:10:01.744268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:17:14.262961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59725,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":155,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57508,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.405781+0000","last_change":"2023-10-06T21:08:16.522162+0000","last_active":"2023-10-06T21:09:44.405781+0000","last_peered":"2023-10-06T21:09:44.405781+0000","last_clean":"2023-10-06T21:09:44.405781+0000","last_became_active":"2023-10-06T21:08:16.521778+0000","last_became_peered":"2023-10-06T21:08:16.521778+0000","last_unstale":"2023-10-06T21:09:44.405781+0000","last_undegraded":"2023-10-06T21:09:44.405781+0000","last_fullsized":"2023-10-06T21:09:44.405781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:32:33.599380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443552+0000","last_change":"2023-10-06T21:08:15.521257+0000","last_active":"2023-10-06T21:08:44.443552+0000","last_peered":"2023-10-06T21:08:44.443552+0000","last_clean":"2023-10-06T21:08:44.443552+0000","last_became_active":"2023-10-06T21:08:15.520965+0000","last_became_peered":"2023-10-06T21:08:15.520965+0000","last_unstale":"2023-10-06T21:08:44.443552+0000","last_undegraded":"2023-10-06T21:08:44.443552+0000","last_fullsized":"2023-10-06T21:08:44.443552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:47:06.913577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422900+0000","last_change":"2023-10-06T21:08:16.530932+0000","last_active":"2023-10-06T21:09:44.422900+0000","last_peered":"2023-10-06T21:09:44.422900+0000","last_clean":"2023-10-06T21:09:44.422900+0000","last_became_active":"2023-10-06T21:08:16.530486+0000","last_became_peered":"2023-10-06T21:08:16.530486+0000","last_unstale":"2023-10-06T21:09:44.422900+0000","last_undegraded":"2023-10-06T21:09:44.422900+0000","last_fullsized":"2023-10-06T21:09:44.422900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:29:57.499791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438078+0000","last_change":"2023-10-06T21:08:15.519857+0000","last_active":"2023-10-06T21:08:44.438078+0000","last_peered":"2023-10-06T21:08:44.438078+0000","last_clean":"2023-10-06T21:08:44.438078+0000","last_became_active":"2023-10-06T21:08:15.519559+0000","last_became_peered":"2023-10-06T21:08:15.519559+0000","last_unstale":"2023-10-06T21:08:44.438078+0000","last_undegraded":"2023-10-06T21:08:44.438078+0000","last_fullsized":"2023-10-06T21:08:44.438078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:57.103871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423646+0000","last_change":"2023-10-06T21:08:16.524695+0000","last_active":"2023-10-06T21:09:44.423646+0000","last_peered":"2023-10-06T21:09:44.423646+0000","last_clean":"2023-10-06T21:09:44.423646+0000","last_became_active":"2023-10-06T21:08:16.524388+0000","last_became_peered":"2023-10-06T21:08:16.524388+0000","last_unstale":"2023-10-06T21:09:44.423646+0000","last_undegraded":"2023-10-06T21:09:44.423646+0000","last_fullsized":"2023-10-06T21:09:44.423646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:03:16.877796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":125,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:10:01.012372+0000","last_change":"2023-10-06T21:08:15.517681+0000","last_active":"2023-10-06T21:10:01.012372+0000","last_peered":"2023-10-06T21:10:01.012372+0000","last_clean":"2023-10-06T21:10:01.012372+0000","last_became_active":"2023-10-06T21:08:15.517340+0000","last_became_peered":"2023-10-06T21:08:15.517340+0000","last_unstale":"2023-10-06T21:10:01.012372+0000","last_undegraded":"2023-10-06T21:10:01.012372+0000","last_fullsized":"2023-10-06T21:10:01.012372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:05:40.243346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.410348+0000","last_change":"2023-10-06T21:08:45.447758+0000","last_active":"2023-10-06T21:09:44.410348+0000","last_peered":"2023-10-06T21:09:44.410348+0000","last_clean":"2023-10-06T21:09:44.410348+0000","last_became_active":"2023-10-06T21:08:45.447337+0000","last_became_peered":"2023-10-06T21:08:45.447337+0000","last_unstale":"2023-10-06T21:09:44.410348+0000","last_undegraded":"2023-10-06T21:09:44.410348+0000","last_fullsized":"2023-10-06T21:09:44.410348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:48:29.806108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.439954+0000","last_change":"2023-10-06T21:08:15.522962+0000","last_active":"2023-10-06T21:08:44.439954+0000","last_peered":"2023-10-06T21:08:44.439954+0000","last_clean":"2023-10-06T21:08:44.439954+0000","last_became_active":"2023-10-06T21:08:15.522539+0000","last_became_peered":"2023-10-06T21:08:15.522539+0000","last_unstale":"2023-10-06T21:08:44.439954+0000","last_undegraded":"2023-10-06T21:08:44.439954+0000","last_fullsized":"2023-10-06T21:08:44.439954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:46.296748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.402236+0000","last_change":"2023-10-06T21:08:16.514490+0000","last_active":"2023-10-06T21:09:44.402236+0000","last_peered":"2023-10-06T21:09:44.402236+0000","last_clean":"2023-10-06T21:09:44.402236+0000","last_became_active":"2023-10-06T21:08:16.514162+0000","last_became_peered":"2023-10-06T21:08:16.514162+0000","last_unstale":"2023-10-06T21:09:44.402236+0000","last_undegraded":"2023-10-06T21:09:44.402236+0000","last_fullsized":"2023-10-06T21:09:44.402236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.528750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440701+0000","last_change":"2023-10-06T21:08:15.517642+0000","last_active":"2023-10-06T21:08:44.440701+0000","last_peered":"2023-10-06T21:08:44.440701+0000","last_clean":"2023-10-06T21:08:44.440701+0000","last_became_active":"2023-10-06T21:08:15.517318+0000","last_became_peered":"2023-10-06T21:08:15.517318+0000","last_unstale":"2023-10-06T21:08:44.440701+0000","last_undegraded":"2023-10-06T21:08:44.440701+0000","last_fullsized":"2023-10-06T21:08:44.440701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:53:07.754599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409776+0000","last_change":"2023-10-06T21:08:16.516891+0000","last_active":"2023-10-06T21:09:44.409776+0000","last_peered":"2023-10-06T21:09:44.409776+0000","last_clean":"2023-10-06T21:09:44.409776+0000","last_became_active":"2023-10-06T21:08:16.515408+0000","last_became_peered":"2023-10-06T21:08:16.515408+0000","last_unstale":"2023-10-06T21:09:44.409776+0000","last_undegraded":"2023-10-06T21:09:44.409776+0000","last_fullsized":"2023-10-06T21:09:44.409776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:33:39.993989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440260+0000","last_change":"2023-10-06T21:08:15.523262+0000","last_active":"2023-10-06T21:08:44.440260+0000","last_peered":"2023-10-06T21:08:44.440260+0000","last_clean":"2023-10-06T21:08:44.440260+0000","last_became_active":"2023-10-06T21:08:15.522777+0000","last_became_peered":"2023-10-06T21:08:15.522777+0000","last_unstale":"2023-10-06T21:08:44.440260+0000","last_undegraded":"2023-10-06T21:08:44.440260+0000","last_fullsized":"2023-10-06T21:08:44.440260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:57:04.771361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422707+0000","last_change":"2023-10-06T21:08:16.524620+0000","last_active":"2023-10-06T21:09:44.422707+0000","last_peered":"2023-10-06T21:09:44.422707+0000","last_clean":"2023-10-06T21:09:44.422707+0000","last_became_active":"2023-10-06T21:08:16.524341+0000","last_became_peered":"2023-10-06T21:08:16.524341+0000","last_unstale":"2023-10-06T21:09:44.422707+0000","last_undegraded":"2023-10-06T21:09:44.422707+0000","last_fullsized":"2023-10-06T21:09:44.422707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:49:23.800109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.441637+0000","last_change":"2023-10-06T21:08:15.522640+0000","last_active":"2023-10-06T21:08:44.441637+0000","last_peered":"2023-10-06T21:08:44.441637+0000","last_clean":"2023-10-06T21:08:44.441637+0000","last_became_active":"2023-10-06T21:08:15.522165+0000","last_became_peered":"2023-10-06T21:08:15.522165+0000","last_unstale":"2023-10-06T21:08:44.441637+0000","last_undegraded":"2023-10-06T21:08:44.441637+0000","last_fullsized":"2023-10-06T21:08:44.441637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:02:29.019380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423917+0000","last_change":"2023-10-06T21:08:16.523483+0000","last_active":"2023-10-06T21:09:44.423917+0000","last_peered":"2023-10-06T21:09:44.423917+0000","last_clean":"2023-10-06T21:09:44.423917+0000","last_became_active":"2023-10-06T21:08:16.522824+0000","last_became_peered":"2023-10-06T21:08:16.522824+0000","last_unstale":"2023-10-06T21:09:44.423917+0000","last_undegraded":"2023-10-06T21:09:44.423917+0000","last_fullsized":"2023-10-06T21:09:44.423917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:27:22.307712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407154+0000","last_change":"2023-10-06T21:08:45.453278+0000","last_active":"2023-10-06T21:09:44.407154+0000","last_peered":"2023-10-06T21:09:44.407154+0000","last_clean":"2023-10-06T21:09:44.407154+0000","last_became_active":"2023-10-06T21:08:45.452892+0000","last_became_peered":"2023-10-06T21:08:45.452892+0000","last_unstale":"2023-10-06T21:09:44.407154+0000","last_undegraded":"2023-10-06T21:09:44.407154+0000","last_fullsized":"2023-10-06T21:09:44.407154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:09:54.604308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:58.006776+0000","last_change":"2023-10-06T21:08:15.526804+0000","last_active":"2023-10-06T21:09:58.006776+0000","last_peered":"2023-10-06T21:09:58.006776+0000","last_clean":"2023-10-06T21:09:58.006776+0000","last_became_active":"2023-10-06T21:08:15.526463+0000","last_became_peered":"2023-10-06T21:08:15.526463+0000","last_unstale":"2023-10-06T21:09:58.006776+0000","last_undegraded":"2023-10-06T21:09:58.006776+0000","last_fullsized":"2023-10-06T21:09:58.006776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:10.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423839+0000","last_change":"2023-10-06T21:08:16.520186+0000","last_active":"2023-10-06T21:09:44.423839+0000","last_peered":"2023-10-06T21:09:44.423839+0000","last_clean":"2023-10-06T21:09:44.423839+0000","last_became_active":"2023-10-06T21:08:16.519763+0000","last_became_peered":"2023-10-06T21:08:16.519763+0000","last_unstale":"2023-10-06T21:09:44.423839+0000","last_undegraded":"2023-10-06T21:09:44.423839+0000","last_fullsized":"2023-10-06T21:09:44.423839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:27:45.103865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436314+0000","last_change":"2023-10-06T21:08:15.516762+0000","last_active":"2023-10-06T21:08:44.436314+0000","last_peered":"2023-10-06T21:08:44.436314+0000","last_clean":"2023-10-06T21:08:44.436314+0000","last_became_active":"2023-10-06T21:08:15.516421+0000","last_became_peered":"2023-10-06T21:08:15.516421+0000","last_unstale":"2023-10-06T21:08:44.436314+0000","last_undegraded":"2023-10-06T21:08:44.436314+0000","last_fullsized":"2023-10-06T21:08:44.436314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:27.054255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.446003+0000","last_change":"2023-10-06T21:07:10.597834+0000","last_active":"2023-10-06T21:08:45.446003+0000","last_peered":"2023-10-06T21:08:45.446003+0000","last_clean":"2023-10-06T21:08:45.446003+0000","last_became_active":"2023-10-06T21:07:10.597242+0000","last_became_peered":"2023-10-06T21:07:10.597242+0000","last_unstale":"2023-10-06T21:08:45.446003+0000","last_undegraded":"2023-10-06T21:08:45.446003+0000","last_fullsized":"2023-10-06T21:08:45.446003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:04:03.340210+0000","last_clean_scrub_stamp":"2023-10-06T21:04:03.340210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:58:58.021171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415596+0000","last_change":"2023-10-06T21:08:16.524427+0000","last_active":"2023-10-06T21:09:44.415596+0000","last_peered":"2023-10-06T21:09:44.415596+0000","last_clean":"2023-10-06T21:09:44.415596+0000","last_became_active":"2023-10-06T21:08:16.523940+0000","last_became_peered":"2023-10-06T21:08:16.523940+0000","last_unstale":"2023-10-06T21:09:44.415596+0000","last_undegraded":"2023-10-06T21:09:44.415596+0000","last_fullsized":"2023-10-06T21:09:44.415596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:47:14.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.442183+0000","last_change":"2023-10-06T21:08:15.521930+0000","last_active":"2023-10-06T21:08:44.442183+0000","last_peered":"2023-10-06T21:08:44.442183+0000","last_clean":"2023-10-06T21:08:44.442183+0000","last_became_active":"2023-10-06T21:08:15.520334+0000","last_became_peered":"2023-10-06T21:08:15.520334+0000","last_unstale":"2023-10-06T21:08:44.442183+0000","last_undegraded":"2023-10-06T21:08:44.442183+0000","last_fullsized":"2023-10-06T21:08:44.442183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:19:09.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.454229+0000","last_change":"2023-10-06T21:08:45.453710+0000","last_active":"2023-10-06T21:08:45.454229+0000","last_peered":"2023-10-06T21:08:45.454229+0000","last_clean":"2023-10-06T21:08:45.454229+0000","last_became_active":"2023-10-06T21:08:45.447386+0000","last_became_peered":"2023-10-06T21:08:45.447386+0000","last_unstale":"2023-10-06T21:08:45.454229+0000","last_undegraded":"2023-10-06T21:08:45.454229+0000","last_fullsized":"2023-10-06T21:08:45.454229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:27.659601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.426691+0000","last_change":"2023-10-06T21:08:16.531833+0000","last_active":"2023-10-06T21:09:44.426691+0000","last_peered":"2023-10-06T21:09:44.426691+0000","last_clean":"2023-10-06T21:09:44.426691+0000","last_became_active":"2023-10-06T21:08:16.531463+0000","last_became_peered":"2023-10-06T21:08:16.531463+0000","last_unstale":"2023-10-06T21:09:44.426691+0000","last_undegraded":"2023-10-06T21:09:44.426691+0000","last_fullsized":"2023-10-06T21:09:44.426691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:14:22.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.445149+0000","last_change":"2023-10-06T21:08:15.518977+0000","last_active":"2023-10-06T21:08:45.445149+0000","last_peered":"2023-10-06T21:08:45.445149+0000","last_clean":"2023-10-06T21:08:45.445149+0000","last_became_active":"2023-10-06T21:08:15.518649+0000","last_became_peered":"2023-10-06T21:08:15.518649+0000","last_unstale":"2023-10-06T21:08:45.445149+0000","last_undegraded":"2023-10-06T21:08:45.445149+0000","last_fullsized":"2023-10-06T21:08:45.445149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:40:34.109690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417794+0000","last_change":"2023-10-06T21:08:16.523197+0000","last_active":"2023-10-06T21:09:44.417794+0000","last_peered":"2023-10-06T21:09:44.417794+0000","last_clean":"2023-10-06T21:09:44.417794+0000","last_became_active":"2023-10-06T21:08:16.522767+0000","last_became_peered":"2023-10-06T21:08:16.522767+0000","last_unstale":"2023-10-06T21:09:44.417794+0000","last_undegraded":"2023-10-06T21:09:44.417794+0000","last_fullsized":"2023-10-06T21:09:44.417794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:52:52.537180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.436021+0000","last_change":"2023-10-06T21:08:15.518254+0000","last_active":"2023-10-06T21:08:44.436021+0000","last_peered":"2023-10-06T21:08:44.436021+0000","last_clean":"2023-10-06T21:08:44.436021+0000","last_became_active":"2023-10-06T21:08:15.517847+0000","last_became_peered":"2023-10-06T21:08:15.517847+0000","last_unstale":"2023-10-06T21:08:44.436021+0000","last_undegraded":"2023-10-06T21:08:44.436021+0000","last_fullsized":"2023-10-06T21:08:44.436021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:36.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418795+0000","last_change":"2023-10-06T21:08:45.613497+0000","last_active":"2023-10-06T21:09:44.418795+0000","last_peered":"2023-10-06T21:09:44.418795+0000","last_clean":"2023-10-06T21:09:44.418795+0000","last_became_active":"2023-10-06T21:08:45.613012+0000","last_became_peered":"2023-10-06T21:08:45.613012+0000","last_unstale":"2023-10-06T21:09:44.418795+0000","last_undegraded":"2023-10-06T21:09:44.418795+0000","last_fullsized":"2023-10-06T21:09:44.418795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:04:42.351299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.443069+0000","last_change":"2023-10-06T21:08:15.524537+0000","last_active":"2023-10-06T21:08:44.443069+0000","last_peered":"2023-10-06T21:08:44.443069+0000","last_clean":"2023-10-06T21:08:44.443069+0000","last_became_active":"2023-10-06T21:08:15.524264+0000","last_became_peered":"2023-10-06T21:08:15.524264+0000","last_unstale":"2023-10-06T21:08:44.443069+0000","last_undegraded":"2023-10-06T21:08:44.443069+0000","last_fullsized":"2023-10-06T21:08:44.443069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:00:08.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.384523+0000","last_change":"2023-10-06T21:08:16.515391+0000","last_active":"2023-10-06T21:09:44.384523+0000","last_peered":"2023-10-06T21:09:44.384523+0000","last_clean":"2023-10-06T21:09:44.384523+0000","last_became_active":"2023-10-06T21:08:16.514115+0000","last_became_peered":"2023-10-06T21:08:16.514115+0000","last_unstale":"2023-10-06T21:09:44.384523+0000","last_undegraded":"2023-10-06T21:09:44.384523+0000","last_fullsized":"2023-10-06T21:09:44.384523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:40:33.514223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.434716+0000","last_change":"2023-10-06T21:08:15.526428+0000","last_active":"2023-10-06T21:08:44.434716+0000","last_peered":"2023-10-06T21:08:44.434716+0000","last_clean":"2023-10-06T21:08:44.434716+0000","last_became_active":"2023-10-06T21:08:15.525886+0000","last_became_peered":"2023-10-06T21:08:15.525886+0000","last_unstale":"2023-10-06T21:08:44.434716+0000","last_undegraded":"2023-10-06T21:08:44.434716+0000","last_fullsized":"2023-10-06T21:08:44.434716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:36.469482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423534+0000","last_change":"2023-10-06T21:08:16.517862+0000","last_active":"2023-10-06T21:09:44.423534+0000","last_peered":"2023-10-06T21:09:44.423534+0000","last_clean":"2023-10-06T21:09:44.423534+0000","last_became_active":"2023-10-06T21:08:16.517482+0000","last_became_peered":"2023-10-06T21:08:16.517482+0000","last_unstale":"2023-10-06T21:09:44.423534+0000","last_undegraded":"2023-10-06T21:09:44.423534+0000","last_fullsized":"2023-10-06T21:09:44.423534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:04:33.851746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:08:45.457037+0000","last_change":"2023-10-06T21:08:45.456629+0000","last_active":"2023-10-06T21:08:45.457037+0000","last_peered":"2023-10-06T21:08:45.457037+0000","last_clean":"2023-10-06T21:08:45.457037+0000","last_became_active":"2023-10-06T21:08:45.449565+0000","last_became_peered":"2023-10-06T21:08:45.449565+0000","last_unstale":"2023-10-06T21:08:45.457037+0000","last_undegraded":"2023-10-06T21:08:45.457037+0000","last_fullsized":"2023-10-06T21:08:45.457037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:23:54.912161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422471+0000","last_change":"2023-10-06T21:08:16.521341+0000","last_active":"2023-10-06T21:09:44.422471+0000","last_peered":"2023-10-06T21:09:44.422471+0000","last_clean":"2023-10-06T21:09:44.422471+0000","last_became_active":"2023-10-06T21:08:16.520956+0000","last_became_peered":"2023-10-06T21:08:16.520956+0000","last_unstale":"2023-10-06T21:09:44.422471+0000","last_undegraded":"2023-10-06T21:09:44.422471+0000","last_fullsized":"2023-10-06T21:09:44.422471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:37.209384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.440584+0000","last_change":"2023-10-06T21:08:15.523042+0000","last_active":"2023-10-06T21:08:44.440584+0000","last_peered":"2023-10-06T21:08:44.440584+0000","last_clean":"2023-10-06T21:08:44.440584+0000","last_became_active":"2023-10-06T21:08:15.522736+0000","last_became_peered":"2023-10-06T21:08:15.522736+0000","last_unstale":"2023-10-06T21:08:44.440584+0000","last_undegraded":"2023-10-06T21:08:44.440584+0000","last_fullsized":"2023-10-06T21:08:44.440584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:13:55.282085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422433+0000","last_change":"2023-10-06T21:08:16.524659+0000","last_active":"2023-10-06T21:09:44.422433+0000","last_peered":"2023-10-06T21:09:44.422433+0000","last_clean":"2023-10-06T21:09:44.422433+0000","last_became_active":"2023-10-06T21:08:16.524354+0000","last_became_peered":"2023-10-06T21:08:16.524354+0000","last_unstale":"2023-10-06T21:09:44.422433+0000","last_undegraded":"2023-10-06T21:09:44.422433+0000","last_fullsized":"2023-10-06T21:09:44.422433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:09:32.343671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.444225+0000","last_change":"2023-10-06T21:08:15.528067+0000","last_active":"2023-10-06T21:08:44.444225+0000","last_peered":"2023-10-06T21:08:44.444225+0000","last_clean":"2023-10-06T21:08:44.444225+0000","last_became_active":"2023-10-06T21:08:15.527793+0000","last_became_peered":"2023-10-06T21:08:15.527793+0000","last_unstale":"2023-10-06T21:08:44.444225+0000","last_undegraded":"2023-10-06T21:08:44.444225+0000","last_fullsized":"2023-10-06T21:08:44.444225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.330152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419710+0000","last_change":"2023-10-06T21:08:16.519153+0000","last_active":"2023-10-06T21:09:44.419710+0000","last_peered":"2023-10-06T21:09:44.419710+0000","last_clean":"2023-10-06T21:09:44.419710+0000","last_became_active":"2023-10-06T21:08:16.518779+0000","last_became_peered":"2023-10-06T21:08:16.518779+0000","last_unstale":"2023-10-06T21:09:44.419710+0000","last_undegraded":"2023-10-06T21:09:44.419710+0000","last_fullsized":"2023-10-06T21:09:44.419710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:37:56.618878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.438547+0000","last_change":"2023-10-06T21:08:15.526511+0000","last_active":"2023-10-06T21:08:44.438547+0000","last_peered":"2023-10-06T21:08:44.438547+0000","last_clean":"2023-10-06T21:08:44.438547+0000","last_became_active":"2023-10-06T21:08:15.526183+0000","last_became_peered":"2023-10-06T21:08:15.526183+0000","last_unstale":"2023-10-06T21:08:44.438547+0000","last_undegraded":"2023-10-06T21:08:44.438547+0000","last_fullsized":"2023-10-06T21:08:44.438547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:55:14.469197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419495+0000","last_change":"2023-10-06T21:08:16.522705+0000","last_active":"2023-10-06T21:09:44.419495+0000","last_peered":"2023-10-06T21:09:44.419495+0000","last_clean":"2023-10-06T21:09:44.419495+0000","last_became_active":"2023-10-06T21:08:16.522412+0000","last_became_peered":"2023-10-06T21:08:16.522412+0000","last_unstale":"2023-10-06T21:09:44.419495+0000","last_undegraded":"2023-10-06T21:09:44.419495+0000","last_fullsized":"2023-10-06T21:09:44.419495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:23.672247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-06T21:08:44.437500+0000","last_change":"2023-10-06T21:08:15.526073+0000","last_active":"2023-10-06T21:08:44.437500+0000","last_peered":"2023-10-06T21:08:44.437500+0000","last_clean":"2023-10-06T21:08:44.437500+0000","last_became_active":"2023-10-06T21:08:15.525659+0000","last_became_peered":"2023-10-06T21:08:15.525659+0000","last_unstale":"2023-10-06T21:08:44.437500+0000","last_undegraded":"2023-10-06T21:08:44.437500+0000","last_fullsized":"2023-10-06T21:08:44.437500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:14.493373+0000","last_clean_scrub_stamp":"2023-10-06T21:08:14.493373+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T05:41:37.603215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.415401+0000","last_change":"2023-10-06T21:08:16.519814+0000","last_active":"2023-10-06T21:09:44.415401+0000","last_peered":"2023-10-06T21:09:44.415401+0000","last_clean":"2023-10-06T21:09:44.415401+0000","last_became_active":"2023-10-06T21:08:16.519513+0000","last_became_peered":"2023-10-06T21:08:16.519513+0000","last_unstale":"2023-10-06T21:09:44.415401+0000","last_undegraded":"2023-10-06T21:09:44.415401+0000","last_fullsized":"2023-10-06T21:09:44.415401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:35:00.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.411059+0000","last_change":"2023-10-06T21:08:16.517706+0000","last_active":"2023-10-06T21:09:44.411059+0000","last_peered":"2023-10-06T21:09:44.411059+0000","last_clean":"2023-10-06T21:09:44.411059+0000","last_became_active":"2023-10-06T21:08:16.517132+0000","last_became_peered":"2023-10-06T21:08:16.517132+0000","last_unstale":"2023-10-06T21:09:44.411059+0000","last_undegraded":"2023-10-06T21:09:44.411059+0000","last_fullsized":"2023-10-06T21:09:44.411059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:58:36.016999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.412660+0000","last_change":"2023-10-06T21:08:16.515270+0000","last_active":"2023-10-06T21:09:44.412660+0000","last_peered":"2023-10-06T21:09:44.412660+0000","last_clean":"2023-10-06T21:09:44.412660+0000","last_became_active":"2023-10-06T21:08:16.514952+0000","last_became_peered":"2023-10-06T21:08:16.514952+0000","last_unstale":"2023-10-06T21:09:44.412660+0000","last_undegraded":"2023-10-06T21:09:44.412660+0000","last_fullsized":"2023-10-06T21:09:44.412660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:39:27.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.419648+0000","last_change":"2023-10-06T21:08:16.521556+0000","last_active":"2023-10-06T21:09:44.419648+0000","last_peered":"2023-10-06T21:09:44.419648+0000","last_clean":"2023-10-06T21:09:44.419648+0000","last_became_active":"2023-10-06T21:08:16.521304+0000","last_became_peered":"2023-10-06T21:08:16.521304+0000","last_unstale":"2023-10-06T21:09:44.419648+0000","last_undegraded":"2023-10-06T21:09:44.419648+0000","last_fullsized":"2023-10-06T21:09:44.419648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T02:57:34.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422939+0000","last_change":"2023-10-06T21:08:45.448913+0000","last_active":"2023-10-06T21:09:44.422939+0000","last_peered":"2023-10-06T21:09:44.422939+0000","last_clean":"2023-10-06T21:09:44.422939+0000","last_became_active":"2023-10-06T21:08:45.448485+0000","last_became_peered":"2023-10-06T21:08:45.448485+0000","last_unstale":"2023-10-06T21:09:44.422939+0000","last_undegraded":"2023-10-06T21:09:44.422939+0000","last_fullsized":"2023-10-06T21:09:44.422939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:42:07.732211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.422296+0000","last_change":"2023-10-06T21:08:16.531725+0000","last_active":"2023-10-06T21:09:44.422296+0000","last_peered":"2023-10-06T21:09:44.422296+0000","last_clean":"2023-10-06T21:09:44.422296+0000","last_became_active":"2023-10-06T21:08:16.531412+0000","last_became_peered":"2023-10-06T21:08:16.531412+0000","last_unstale":"2023-10-06T21:09:44.422296+0000","last_undegraded":"2023-10-06T21:09:44.422296+0000","last_fullsized":"2023-10-06T21:09:44.422296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T07:02:20.635944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.407447+0000","last_change":"2023-10-06T21:08:16.521912+0000","last_active":"2023-10-06T21:09:44.407447+0000","last_peered":"2023-10-06T21:09:44.407447+0000","last_clean":"2023-10-06T21:09:44.407447+0000","last_became_active":"2023-10-06T21:08:16.521621+0000","last_became_peered":"2023-10-06T21:08:16.521621+0000","last_unstale":"2023-10-06T21:09:44.407447+0000","last_undegraded":"2023-10-06T21:09:44.407447+0000","last_fullsized":"2023-10-06T21:09:44.407447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T08:24:12.483193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.414676+0000","last_change":"2023-10-06T21:08:16.519733+0000","last_active":"2023-10-06T21:09:44.414676+0000","last_peered":"2023-10-06T21:09:44.414676+0000","last_clean":"2023-10-06T21:09:44.414676+0000","last_became_active":"2023-10-06T21:08:16.519424+0000","last_became_peered":"2023-10-06T21:08:16.519424+0000","last_unstale":"2023-10-06T21:09:44.414676+0000","last_undegraded":"2023-10-06T21:09:44.414676+0000","last_fullsized":"2023-10-06T21:09:44.414676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:23:23.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.416177+0000","last_change":"2023-10-06T21:08:16.520477+0000","last_active":"2023-10-06T21:09:44.416177+0000","last_peered":"2023-10-06T21:09:44.416177+0000","last_clean":"2023-10-06T21:09:44.416177+0000","last_became_active":"2023-10-06T21:08:16.520049+0000","last_became_peered":"2023-10-06T21:08:16.520049+0000","last_unstale":"2023-10-06T21:09:44.416177+0000","last_undegraded":"2023-10-06T21:09:44.416177+0000","last_fullsized":"2023-10-06T21:09:44.416177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:11.520287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.418196+0000","last_change":"2023-10-06T21:08:16.520050+0000","last_active":"2023-10-06T21:09:44.418196+0000","last_peered":"2023-10-06T21:09:44.418196+0000","last_clean":"2023-10-06T21:09:44.418196+0000","last_became_active":"2023-10-06T21:08:16.519730+0000","last_became_peered":"2023-10-06T21:08:16.519730+0000","last_unstale":"2023-10-06T21:09:44.418196+0000","last_undegraded":"2023-10-06T21:09:44.418196+0000","last_fullsized":"2023-10-06T21:09:44.418196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T04:31:00.460529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.417342+0000","last_change":"2023-10-06T21:08:16.523871+0000","last_active":"2023-10-06T21:09:44.417342+0000","last_peered":"2023-10-06T21:09:44.417342+0000","last_clean":"2023-10-06T21:09:44.417342+0000","last_became_active":"2023-10-06T21:08:16.522082+0000","last_became_peered":"2023-10-06T21:08:16.522082+0000","last_unstale":"2023-10-06T21:09:44.417342+0000","last_undegraded":"2023-10-06T21:09:44.417342+0000","last_fullsized":"2023-10-06T21:09:44.417342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T03:00:45.073524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423644+0000","last_change":"2023-10-06T21:08:16.521349+0000","last_active":"2023-10-06T21:09:44.423644+0000","last_peered":"2023-10-06T21:09:44.423644+0000","last_clean":"2023-10-06T21:09:44.423644+0000","last_became_active":"2023-10-06T21:08:16.521035+0000","last_became_peered":"2023-10-06T21:08:16.521035+0000","last_unstale":"2023-10-06T21:09:44.423644+0000","last_undegraded":"2023-10-06T21:09:44.423644+0000","last_fullsized":"2023-10-06T21:09:44.423644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T06:33:57.104628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.423037+0000","last_change":"2023-10-06T21:08:16.525091+0000","last_active":"2023-10-06T21:09:44.423037+0000","last_peered":"2023-10-06T21:09:44.423037+0000","last_clean":"2023-10-06T21:09:44.423037+0000","last_became_active":"2023-10-06T21:08:16.524779+0000","last_became_peered":"2023-10-06T21:08:16.524779+0000","last_unstale":"2023-10-06T21:09:44.423037+0000","last_undegraded":"2023-10-06T21:09:44.423037+0000","last_fullsized":"2023-10-06T21:09:44.423037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:39:46.609465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-06T21:09:44.409451+0000","last_change":"2023-10-06T21:08:16.517127+0000","last_active":"2023-10-06T21:09:44.409451+0000","last_peered":"2023-10-06T21:09:44.409451+0000","last_clean":"2023-10-06T21:09:44.409451+0000","last_became_active":"2023-10-06T21:08:16.516043+0000","last_became_peered":"2023-10-06T21:08:16.516043+0000","last_unstale":"2023-10-06T21:09:44.409451+0000","last_undegraded":"2023-10-06T21:09:44.409451+0000","last_fullsized":"2023-10-06T21:09:44.409451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T21:08:15.495392+0000","last_clean_scrub_stamp":"2023-10-06T21:08:15.495392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T01:05:19.635125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64397,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":623,"num_read_kb":375,"num_write":161,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":57530,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.40699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77300000000000002,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.623}]},{"osd":3,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.47399999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67600000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.84499999999999997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 21:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77200000000000002,"15min":0.76700000000000002},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91900000000000004,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.85899999999999999}]},{"osd":8,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.81299999999999994},"min":{"1min":0.623,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.92800000000000005,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.875},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.83399999999999996,"15min":0.81699999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.80800000000000005},"min":{"1min":0.63400000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.96899999999999997,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.90300000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1108306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.499}]},{"osd":2,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.755,"5min":1.089,"15min":1.089},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83499999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.72399999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65200000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.55100000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Oct 6 21:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.63200000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.83299999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.81499999999999995,"15min":0.83899999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.82099999999999995,"5min":1.165,"15min":1.165},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83799999999999997,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.873,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.85199999999999998,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84899999999999998,"5min":1.26,"15min":1.26},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92500000000000004,"5min":1.048,"15min":1.048},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.56899999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.90900000000000003,"5min":1,"15min":1},"last":0.76400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":456799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51100000000000001,"15min":0.52300000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.441},{"interface":"front","average":{"1min":0.44,"5min":0.56299999999999994,"15min":0.58699999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":1.014,"15min":1.014},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.083,"5min":1.347,"15min":1.347},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.66600000000000004,"15min":0.67400000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97999999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.78700000000000003,"15min":0.82499999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1659999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.65400000000000003,"15min":0.67800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72399999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.79500000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1240000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68200000000000005,"15min":0.70099999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.501},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81399999999999995,"15min":0.83799999999999997},"min":{"1min":0.499,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.085,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.879},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.4329999999999998,"5min":3.4329999999999998,"15min":3.4329999999999998},"last":0.93600000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85599999999999998,"15min":0.88700000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0089999999999999,"5min":1.226,"15min":1.226},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":1.0069999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2490000000000001,"5min":1.542,"15min":1.542},"last":0.89600000000000002}]},{"osd":8,"last update":"Fri Oct 6 21:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":0.86699999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.125,"5min":2.9169999999999998,"15min":2.9169999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.95199999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.056,"5min":1.365,"15min":1.365},"last":0.98899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.95799999999999996,"15min":0.94999999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.9369999999999998,"5min":2.9369999999999998,"15min":2.9369999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1359999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.95599999999999996}]},{"osd":11,"last update":"Fri Oct 6 21:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.68899999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71999999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.46200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.53400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.51600000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76900000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48399999999999999,"15min":0.47999999999999998},"min":{"1min":0.39200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40500000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.63200000000000001},"min":{"1min":0.44900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.59499999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92900000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.65000000000000002,"15min":0.63},"min":{"1min":0.60399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.94799999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.93200000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.46300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93100000000000005,"5min":1.087,"15min":1.087},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.59399999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":1.038,"15min":1.038},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.65300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0009999999999999,"5min":1.155,"15min":1.155},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70899999999999996,"15min":0.69099999999999995},"min":{"1min":0.71299999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.93899999999999995}]},{"osd":9,"last update":"Fri Oct 6 21:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.77700000000000002},"min":{"1min":0.63100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77600000000000002,"15min":0.754},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.999}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.97299999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95399999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1049326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.40200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.28599999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.443,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.29199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46200000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.752},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.48899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93100000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.76500000000000001,"5min":1.002,"15min":1.002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82299999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.60899999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73999999999999999,"15min":0.746},"min":{"1min":0.57899999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.48299999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.877,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Oct 6 21:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91600000000000004,"5min":1.012,"15min":1.012},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.55600000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90200000000000002,"5min":1.038,"15min":1.038},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.879},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.84699999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.46,"5min":5.8620000000000001,"15min":5.8620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.93400000000000005,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.56699999999999995,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98299999999999998,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.58099999999999996,"5min":0.38,"15min":0.38},"max":{"1min":1.1679999999999999,"5min":1.948,"15min":1.948},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68500000000000005,"15min":0.64700000000000002},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2749999999999999,"5min":4.9809999999999999,"15min":4.9809999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73599999999999999,"15min":0.72099999999999997},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3879999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0089999999999999,"5min":5.9290000000000003,"15min":5.9290000000000003},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.355,"5min":4.8369999999999997,"15min":4.8369999999999997},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80300000000000005,"15min":0.79300000000000004},"min":{"1min":0.53800000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.036,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.80300000000000005,"15min":0.85099999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.056,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.59899999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4790000000000001,"5min":5.0199999999999996,"15min":5.0199999999999996},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.82899999999999996,"15min":0.84399999999999997},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90600000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.56899999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.86599999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.497,"15min":0.497},"max":{"1min":4.9420000000000002,"5min":4.9420000000000002,"15min":4.9420000000000002},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":0.98499999999999999},"min":{"1min":0.76800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.127,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.622,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.206,"5min":2.258,"15min":2.258},"last":0.71399999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:12 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86899999999999999,"15min":0.83399999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.8979999999999997,"5min":4.8979999999999997,"15min":4.8979999999999997},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":515706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.29599999999999999},"max":{"1min":0.76900000000000002,"5min":6.8949999999999996,"15min":6.8949999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32400000000000001,"15min":0.32000000000000001},"max":{"1min":0.81000000000000005,"5min":1.105,"15min":1.105},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Oct 6 21:09:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.318,"15min":0.318},"max":{"1min":0.81599999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.48599999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.877,"15min":0.877},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.59399999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.84399999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.64100000000000001,"15min":0.60899999999999999},"min":{"1min":0.48099999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Oct 6 21:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.57199999999999995,"15min":0.53800000000000003},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.57099999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78800000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64800000000000002},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95099999999999996,"5min":2.714,"15min":2.714},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.498},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.68899999999999995,"15min":0.66000000000000003},"min":{"1min":0.60399999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.71399999999999997,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.999,"5min":1.022,"15min":1.022},"last":0.82699999999999996}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82899999999999996,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.622},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83699999999999997,"15min":0.80500000000000005},"min":{"1min":0.63300000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73699999999999999,"15min":0.73399999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.048,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Oct 6 21:09:19 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.67200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":457459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.64500000000000002,"15min":0.58099999999999996},"min":{"1min":0.45400000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.2690000000000001,"5min":4.2690000000000001,"15min":4.2690000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92400000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.77800000000000002}]},{"osd":2,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.626},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97399999999999998,"5min":1.216,"15min":1.216},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64600000000000002,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Oct 6 21:06:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.145},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94199999999999995,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 21:09:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.238,"5min":1.397,"15min":1.397},"last":0.83099999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.67300000000000004,"15min":0.69699999999999995},"min":{"1min":0.39400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76800000000000002,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68200000000000005,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.79900000000000004},"min":{"1min":0.58099999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.79000000000000004,"15min":0.79200000000000004},"min":{"1min":0.60899999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.27,"5min":1.375,"15min":1.375},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Oct 6 21:09:53 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81999999999999995,"15min":0.84599999999999997},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.745},{"interface":"front","average":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":1.1100000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.64500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.92000000000000004,"15min":0.92800000000000005},"min":{"1min":0.61499999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0600000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:31 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.90400000000000003},"min":{"1min":0.71199999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.1000000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.84599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":457862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80700000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.502}]},{"osd":1,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.57899999999999996,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.60099999999999998},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.45800000000000002,"15min":0.45000000000000001},"min":{"1min":0.39800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.502,"15min":0.48699999999999999},"min":{"1min":0.38700000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46000000000000002}]},{"osd":6,"last update":"Fri Oct 6 21:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.65400000000000003},"min":{"1min":0.67200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98799999999999999,"5min":1.139,"15min":1.139},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93400000000000005,"5min":1,"15min":1},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 21:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Oct 6 21:09:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64400000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95099999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.58299999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.94599999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.76100000000000001,"15min":0.745},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.071,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.80400000000000005}]},{"osd":11,"last update":"Fri Oct 6 21:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71699999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47499999999999998,"15min":0.432},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.41899999999999998},"min":{"1min":0.38100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.55300000000000005},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.51600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98199999999999998,"5min":1.002,"15min":1.002},"last":0.748},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66700000000000004,"15min":0.61699999999999999},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 21:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69499999999999995,"15min":0.64900000000000002},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93600000000000005,"5min":1.103,"15min":1.103},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.68799999999999994,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Oct 6 21:09:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69299999999999995,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72099999999999997,"15min":0.68999999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85799999999999998,"5min":1.129,"15min":1.129},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.71299999999999997},"min":{"1min":0.56599999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Oct 6 21:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1579999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.92400000000000004,"15min":0.96399999999999997},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0860000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Oct 6 21:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.98099999999999998},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.129,"5min":1.395,"15min":1.395},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.85799999999999998,"15min":0.87},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.129,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78600000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":516881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.496,"15min":0.47699999999999998},"min":{"1min":0.436,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.48499999999999999},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.36099999999999999},"min":{"1min":0.32100000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53700000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.36599999999999999},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Oct 6 21:10:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.56100000000000005},"min":{"1min":0.45800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.371,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.435}]},{"osd":5,"last update":"Fri Oct 6 21:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.58099999999999996},"min":{"1min":0.57499999999999996,"5min":0.37,"15min":0.37},"max":{"1min":0.89800000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.66100000000000003,"15min":0.60499999999999998},"min":{"1min":0.56499999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 21:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.51900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88100000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.621,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 21:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Oct 6 21:09:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.501,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.86699999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Oct 6 21:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.628,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.94399999999999995,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92300000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.90100000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.752,"15min":0.745},"min":{"1min":0.47399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Oct 6 21:09:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1048148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76400000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.38900000000000001},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.48299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.876,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 21:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.64400000000000002},"min":{"1min":0.45500000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89600000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75600000000000001}]},{"osd":8,"last update":"Fri Oct 6 21:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92800000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77000000000000002}]},{"osd":9,"last update":"Fri Oct 6 21:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90000000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.745,"15min":0.73799999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79200000000000004}]},{"osd":10,"last update":"Fri Oct 6 21:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.876},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T21:10:06.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:10:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:06 smithi167 ceph-mon[134494]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 -- 172.21.15.26:0/3000169011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 msgr2=0x7faf68109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 --2- 172.21.15.26:0/3000169011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf68109070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faf6000b0a0 tx=0x7faf6000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 -- 172.21.15.26:0/3000169011 shutdown_connections 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 --2- 172.21.15.26:0/3000169011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf68109070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 --2- 172.21.15.26:0/3000169011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681019a0 0x7faf68103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 --2- 172.21.15.26:0/3000169011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681042d0 0x7faf681066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 -- 172.21.15.26:0/3000169011 >> 172.21.15.26:0/3000169011 conn(0x7faf680fb3b0 msgr2=0x7faf680fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 -- 172.21.15.26:0/3000169011 shutdown_connections 2023-10-06T21:10:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.675+0000 7faf6ddff700 1 -- 172.21.15.26:0/3000169011 wait complete. 2023-10-06T21:10:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 Processor -- start 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 -- start start 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681042d0 0x7faf680ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf680ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68100630 con 0x7faf68106c00 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681007a0 con 0x7faf681019a0 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.676+0000 7faf6ddff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681a0c30 con 0x7faf681042d0 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf680ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52002/0 (socket says 172.21.15.26:52002) 2023-10-06T21:10:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf677fe700 1 -- 172.21.15.26:0/4095191047 learned_addr learned my addr 172.21.15.26:0/4095191047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf66ffd700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681042d0 0x7faf680ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 -- 172.21.15.26:0/4095191047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681042d0 msgr2=0x7faf680ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681042d0 0x7faf680ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 -- 172.21.15.26:0/4095191047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 msgr2=0x7faf68101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 -- 172.21.15.26:0/4095191047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c009520 con 0x7faf68106c00 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf677fe700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.677+0000 7faf67fff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf680ffe90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faf60009fd0 tx=0x7faf60007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.678+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000e040 con 0x7faf68106c00 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.678+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60009d00 con 0x7faf68106c00 2023-10-06T21:10:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.678+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf60004c50 con 0x7faf68106c00 2023-10-06T21:10:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.678+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6001b6c0 con 0x7faf68106c00 2023-10-06T21:10:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.678+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681a11a0 con 0x7faf68106c00 2023-10-06T21:10:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.679+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf680fcf30 con 0x7faf68106c00 2023-10-06T21:10:06.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.681+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faf6001d470 con 0x7faf68106c00 2023-10-06T21:10:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.682+0000 7faf64ff9700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 0x7faf500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.682+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faf600cdd30 con 0x7faf68106c00 2023-10-06T21:10:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.683+0000 7faf677fe700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 0x7faf500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.683+0000 7faf677fe700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 0x7faf500a3990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf58011550 tx=0x7faf58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.686+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faf600973e0 con 0x7faf68106c00 2023-10-06T21:10:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.900+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf6810c750 con 0x7faf68106c00 2023-10-06T21:10:06.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.901+0000 7faf64ff9700 1 -- 172.21.15.26:0/4095191047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7faf6009aa00 con 0x7faf68106c00 2023-10-06T21:10:06.904 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:06.905 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:10:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 msgr2=0x7faf500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 0x7faf500a3990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf58011550 tx=0x7faf58009260 comp rx=0 tx=0).stop 2023-10-06T21:10:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 msgr2=0x7faf680ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf680ffe90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faf60009fd0 tx=0x7faf60007930 comp rx=0 tx=0).stop 2023-10-06T21:10:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 shutdown_connections 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7faf500a14d0 0x7faf500a3990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf68106c00 0x7faf680ffe90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf681019a0 0x7faf68101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 --2- 172.21.15.26:0/4095191047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf681042d0 0x7faf680ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 >> 172.21.15.26:0/4095191047 conn(0x7faf680fb3b0 msgr2=0x7faf68102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:06.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 shutdown_connections 2023-10-06T21:10:06.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:06.904+0000 7faf6ddff700 1 -- 172.21.15.26:0/4095191047 wait complete. 2023-10-06T21:10:06.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:10:06.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3332341204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 msgr2=0x7f92a4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3332341204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a4109370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b0a0 tx=0x7f929c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3332341204 shutdown_connections 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3332341204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a4109370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3332341204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3332341204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 0x7f92a4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3332341204 >> 172.21.15.26:0/3332341204 conn(0x7f92a40fc4d0 msgr2=0x7f92a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3332341204 shutdown_connections 2023-10-06T21:10:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.132+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3332341204 wait complete. 2023-10-06T21:10:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 Processor -- start 2023-10-06T21:10:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 -- start start 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 0x7f92a4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41a0a50 con 0x7f92a4102a50 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41a0b90 con 0x7f92a4100f40 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.134+0000 7f92ab7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41a0d20 con 0x7f92a4102090 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39042/0 (socket says 172.21.15.26:39042) 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52032/0 (socket says 172.21.15.26:52032) 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 -- 172.21.15.26:0/3438109094 learned_addr learned my addr 172.21.15.26:0/3438109094 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a8d9a700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 0x7f92a4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 -- 172.21.15.26:0/3438109094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 msgr2=0x7f92a4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 0x7f92a4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 -- 172.21.15.26:0/3438109094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 msgr2=0x7f92a4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 -- 172.21.15.26:0/3438109094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c009d00 con 0x7f92a4102a50 2023-10-06T21:10:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.135+0000 7f92a9d9c700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d010 tx=0x7f929c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.136+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00e040 con 0x7f92a4102a50 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.136+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a41a0fa0 con 0x7f92a4102a50 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.136+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f929c003e10 con 0x7f92a4102a50 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.136+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c0052a0 con 0x7f92a4102a50 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.136+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a41a1370 con 0x7f92a4102a50 2023-10-06T21:10:07.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.137+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a410c720 con 0x7f92a4102a50 2023-10-06T21:10:07.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.139+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f929c01d480 con 0x7f92a4102a50 2023-10-06T21:10:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.141+0000 7f929a7fc700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 0x7f92900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.141+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f929c0cdde0 con 0x7f92a4102a50 2023-10-06T21:10:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.141+0000 7f92a959b700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 0x7f92900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.142+0000 7f92a959b700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 0x7f92900a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009c60 tx=0x7f92a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.145+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f929c097490 con 0x7f92a4102a50 2023-10-06T21:10:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:07 smithi026 ceph-mon[98490]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:07 smithi026 ceph-mon[98490]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:07 smithi026 ceph-mon[98490]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:10:07.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:07 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4095191047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.354+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92a41a1560 con 0x7f92a4102a50 2023-10-06T21:10:07.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.354+0000 7f929a7fc700 1 -- 172.21.15.26:0/3438109094 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f929c024310 con 0x7f92a4102a50 2023-10-06T21:10:07.357 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:10:07.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.356+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 msgr2=0x7f92900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.356+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 0x7f92900a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009c60 tx=0x7f92a0009260 comp rx=0 tx=0).stop 2023-10-06T21:10:07.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 msgr2=0x7f92a41a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d010 tx=0x7f929c00bc80 comp rx=0 tx=0).stop 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 shutdown_connections 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f92900a1480 0x7f92900a3940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92a4102a50 0x7f92a41a0480 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92a4100f40 0x7f92a4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 --2- 172.21.15.26:0/3438109094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92a4102090 0x7f92a4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 >> 172.21.15.26:0/3438109094 conn(0x7f92a40fc4d0 msgr2=0x7f92a41079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 shutdown_connections 2023-10-06T21:10:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.357+0000 7f92ab7ff700 1 -- 172.21.15.26:0/3438109094 wait complete. 2023-10-06T21:10:07.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:07 smithi078 ceph-mon[134896]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:07 smithi078 ceph-mon[134896]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:07 smithi078 ceph-mon[134896]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:10:07.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:07 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4095191047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:07.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:10:07.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:07 smithi167 ceph-mon[134494]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:07 smithi167 ceph-mon[134494]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-06T21:10:07.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:07 smithi167 ceph-mon[134494]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-06T21:10:07.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:07 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4095191047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:07.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 -- 172.21.15.26:0/1216913109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 msgr2=0x7f5668102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 --2- 172.21.15.26:0/1216913109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668102500 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5658009990 tx=0x7f5658009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 -- 172.21.15.26:0/1216913109 shutdown_connections 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 --2- 172.21.15.26:0/1216913109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 0x7f5668109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 --2- 172.21.15.26:0/1216913109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f56681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 --2- 172.21.15.26:0/1216913109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668102500 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 -- 172.21.15.26:0/1216913109 >> 172.21.15.26:0/1216913109 conn(0x7f56680fc460 msgr2=0x7f56680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 -- 172.21.15.26:0/1216913109 shutdown_connections 2023-10-06T21:10:07.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.585+0000 7f56707ff700 1 -- 172.21.15.26:0/1216913109 wait complete. 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 Processor -- start 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 -- start start 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 0x7f566819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566819e750 con 0x7f5668100ea0 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566819e890 con 0x7f5668102080 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.586+0000 7f56707ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566819ea20 con 0x7f5668102a40 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39060/0 (socket says 172.21.15.26:39060) 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566e59b700 1 -- 172.21.15.26:0/2267700936 learned_addr learned my addr 172.21.15.26:0/2267700936 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566ed9c700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 0x7f566819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 -- 172.21.15.26:0/2267700936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 msgr2=0x7f566819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 0x7f566819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 -- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 msgr2=0x7f5668196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 -- 172.21.15.26:0/2267700936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56580095f0 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566e59b700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f566dd9a700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668197110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5658000c00 tx=0x7f5658005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.587+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565801d070 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.588+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5658022470 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.588+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566819eca0 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.588+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658017650 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.588+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566819f0c0 con 0x7f5668102080 2023-10-06T21:10:07.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.589+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5668107ea0 con 0x7f5668102080 2023-10-06T21:10:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.591+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f56580225e0 con 0x7f5668102080 2023-10-06T21:10:07.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.593+0000 7f565f7fe700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 0x7f56540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.593+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f56580ce850 con 0x7f5668102080 2023-10-06T21:10:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.593+0000 7f566e59b700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 0x7f56540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.594+0000 7f566e59b700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 0x7f56540a3940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5664005950 tx=0x7f5664009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.595+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5658097f00 con 0x7f5668102080 2023-10-06T21:10:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.807+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56680008d0 con 0x7f5668102080 2023-10-06T21:10:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.808+0000 7f565f7fe700 1 -- 172.21.15.26:0/2267700936 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f565809b520 con 0x7f5668102080 2023-10-06T21:10:07.810 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:10:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.810+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 msgr2=0x7f56540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.810+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 0x7f56540a3940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5664005950 tx=0x7f5664009260 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.810+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 msgr2=0x7f5668197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.810+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668197110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5658000c00 tx=0x7f5658005720 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 shutdown_connections 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5668102a40 0x7f566819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f56540a1480 0x7f56540a3940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5668100ea0 0x7f5668196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 --2- 172.21.15.26:0/2267700936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5668102080 0x7f5668197110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 >> 172.21.15.26:0/2267700936 conn(0x7f56680fc460 msgr2=0x7f56680fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 shutdown_connections 2023-10-06T21:10:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:07.811+0000 7f56707ff700 1 -- 172.21.15.26:0/2267700936 wait complete. 2023-10-06T21:10:07.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:10:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/1948743256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8100e90 msgr2=0x7fa3c81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/1948743256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8100e90 0x7fa3c81012b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098f0 tx=0x7fa3bc009c00 comp rx=0 tx=0).stop 2023-10-06T21:10:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/1948743256 shutdown_connections 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/1948743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8102a30 0x7fa3c8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/1948743256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8100e90 0x7fa3c81012b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/1948743256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/1948743256 >> 172.21.15.26:0/1948743256 conn(0x7fa3c80fc430 msgr2=0x7fa3c80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/1948743256 shutdown_connections 2023-10-06T21:10:08.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.037+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/1948743256 wait complete. 2023-10-06T21:10:08.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.038+0000 7fa3cf1ff700 1 Processor -- start 2023-10-06T21:10:08.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.038+0000 7fa3cf1ff700 1 -- start start 2023-10-06T21:10:08.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8100e90 0x7fa3c8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8102a30 0x7fa3c81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a09b0 con 0x7fa3c8102070 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a0af0 con 0x7fa3c8100e90 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cf1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a0c80 con 0x7fa3c8102a30 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39090/0 (socket says 172.21.15.26:39090) 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 -- 172.21.15.26:0/2821938414 learned_addr learned my addr 172.21.15.26:0/2821938414 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3cd79c700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8102a30 0x7fa3c81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3ccf9b700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8100e90 0x7fa3c8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 -- 172.21.15.26:0/2821938414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8102a30 msgr2=0x7fa3c81a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8102a30 0x7fa3c81a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 -- 172.21.15.26:0/2821938414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8100e90 msgr2=0x7fa3c8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8100e90 0x7fa3c8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.039+0000 7fa3c7fff700 1 -- 172.21.15.26:0/2821938414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4009520 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3c7fff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400ea50 tx=0x7fa3b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400cbd0 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc009520 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3b400cd30 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4018680 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.040+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c81a1260 con 0x7fa3c8102070 2023-10-06T21:10:08.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.041+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c81934f0 con 0x7fa3c8102070 2023-10-06T21:10:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.043+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa3b4010430 con 0x7fa3c8102070 2023-10-06T21:10:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.044+0000 7fa3c5ffb700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 0x7fa3b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.045+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3b4014070 con 0x7fa3c8102070 2023-10-06T21:10:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.045+0000 7fa3ccf9b700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 0x7fa3b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.045+0000 7fa3ccf9b700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 0x7fa3b80a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098c0 tx=0x7fa3bc019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:08.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.047+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa3b4096df0 con 0x7fa3c8102070 2023-10-06T21:10:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.260+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3c810c890 con 0x7fa3c8102070 2023-10-06T21:10:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.261+0000 7fa3c5ffb700 1 -- 172.21.15.26:0/2821938414 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3b409a410 con 0x7fa3c8102070 2023-10-06T21:10:08.264 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:10:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.263+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 msgr2=0x7fa3b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.263+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 0x7fa3b80a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098c0 tx=0x7fa3bc019040 comp rx=0 tx=0).stop 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.263+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 msgr2=0x7fa3c8199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.263+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400ea50 tx=0x7fa3b400ed60 comp rx=0 tx=0).stop 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 shutdown_connections 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3c8102a30 0x7fa3c81a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c8100e90 0x7fa3c8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fa3b80a1480 0x7fa3b80a3940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 --2- 172.21.15.26:0/2821938414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3c8102070 0x7fa3c8199370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 >> 172.21.15.26:0/2821938414 conn(0x7fa3c80fc430 msgr2=0x7fa3c8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 shutdown_connections 2023-10-06T21:10:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.264+0000 7fa3cf1ff700 1 -- 172.21.15.26:0/2821938414 wait complete. 2023-10-06T21:10:08.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:10:08.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3438109094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2267700936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:08 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3438109094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:08 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2267700936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 -- 172.21.15.26:0/1916841437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8102a40 msgr2=0x7f27c8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/1916841437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8102a40 0x7f27c8109360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b0a0 tx=0x7f27c000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 -- 172.21.15.26:0/1916841437 shutdown_connections 2023-10-06T21:10:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/1916841437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8102a40 0x7f27c8109360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/1916841437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8100ea0 0x7f27c81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/1916841437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102080 0x7f27c8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 -- 172.21.15.26:0/1916841437 >> 172.21.15.26:0/1916841437 conn(0x7f27c80fc420 msgr2=0x7f27c80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 -- 172.21.15.26:0/1916841437 shutdown_connections 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.488+0000 7f27cd1ff700 1 -- 172.21.15.26:0/1916841437 wait complete. 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.489+0000 7f27cd1ff700 1 Processor -- start 2023-10-06T21:10:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.489+0000 7f27cd1ff700 1 -- start start 2023-10-06T21:10:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8100ea0 0x7f27c8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8102080 0x7f27c8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819e840 con 0x7f27c8102a40 2023-10-06T21:10:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819e9b0 con 0x7f27c8100ea0 2023-10-06T21:10:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27cd1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819eb70 con 0x7f27c8102080 2023-10-06T21:10:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39120/0 (socket says 172.21.15.26:39120) 2023-10-06T21:10:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 -- 172.21.15.26:0/2292532331 learned_addr learned my addr 172.21.15.26:0/2292532331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c6d9d700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8100ea0 0x7f27c8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c659c700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8102080 0x7f27c8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 -- 172.21.15.26:0/2292532331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8102080 msgr2=0x7f27c8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8102080 0x7f27c8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 -- 172.21.15.26:0/2292532331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8100ea0 msgr2=0x7f27c8196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8100ea0 0x7f27c8196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.490+0000 7f27c759e700 1 -- 172.21.15.26:0/2292532331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009d00 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27c759e700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009fd0 tx=0x7f27c000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000e040 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c0003e10 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c819edf0 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c001b6a0 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.491+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c819f1c0 con 0x7f27c8102a40 2023-10-06T21:10:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.492+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8108460 con 0x7f27c8102a40 2023-10-06T21:10:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.494+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f27c001d470 con 0x7f27c8102a40 2023-10-06T21:10:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.495+0000 7f27affff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 0x7f27b00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.496+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f27c00cde60 con 0x7f27c8102a40 2023-10-06T21:10:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.496+0000 7f27c6d9d700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 0x7f27b00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.496+0000 7f27c6d9d700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 0x7f27b00a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27c81981d0 tx=0x7f27b800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.499+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27c0097510 con 0x7f27c8102a40 2023-10-06T21:10:08.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:08 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3438109094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:08 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2267700936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:10:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.767+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27c810c760 con 0x7f27c8102a40 2023-10-06T21:10:08.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.768+0000 7f27affff700 1 -- 172.21.15.26:0/2292532331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f27c009ab30 con 0x7f27c8102a40 2023-10-06T21:10:08.770 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:08.771 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 msgr2=0x7f27b00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 0x7f27b00a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f27c81981d0 tx=0x7f27b800b410 comp rx=0 tx=0).stop 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 msgr2=0x7f27c819e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009fd0 tx=0x7f27c000bce0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 shutdown_connections 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f27b00a14d0 0x7f27b00a3990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c8102a40 0x7f27c819e270 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8100ea0 0x7f27c8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 --2- 172.21.15.26:0/2292532331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c8102080 0x7f27c8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 >> 172.21.15.26:0/2292532331 conn(0x7f27c80fc420 msgr2=0x7f27c81055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 shutdown_connections 2023-10-06T21:10:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.770+0000 7f27cd1ff700 1 -- 172.21.15.26:0/2292532331 wait complete. 2023-10-06T21:10:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:10:08.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:10:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.993+0000 7f546abff700 1 -- 172.21.15.26:0/2285922770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 msgr2=0x7f5464109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.993+0000 7f546abff700 1 --2- 172.21.15.26:0/2285922770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f5464109360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f545c009f90 tx=0x7f545c00b330 comp rx=0 tx=0).stop 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 -- 172.21.15.26:0/2285922770 shutdown_connections 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 --2- 172.21.15.26:0/2285922770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f5464109360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 --2- 172.21.15.26:0/2285922770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464100ea0 0x7f54641012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 --2- 172.21.15.26:0/2285922770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464102080 0x7f5464102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 -- 172.21.15.26:0/2285922770 >> 172.21.15.26:0/2285922770 conn(0x7f54640fc420 msgr2=0x7f54640fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 -- 172.21.15.26:0/2285922770 shutdown_connections 2023-10-06T21:10:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.994+0000 7f546abff700 1 -- 172.21.15.26:0/2285922770 wait complete. 2023-10-06T21:10:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 Processor -- start 2023-10-06T21:10:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 -- start start 2023-10-06T21:10:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 0x7f5464198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464102080 0x7f5464199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54641a0a40 con 0x7f5464102a40 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54641a0b80 con 0x7f5464100ea0 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.995+0000 7f546abff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54641a0d10 con 0x7f5464102080 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39142/0 (socket says 172.21.15.26:39142) 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546899b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 0x7f5464198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 -- 172.21.15.26:0/1203317395 learned_addr learned my addr 172.21.15.26:0/1203317395 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f5463fff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464102080 0x7f5464199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 -- 172.21.15.26:0/1203317395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464102080 msgr2=0x7f5464199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464102080 0x7f5464199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 -- 172.21.15.26:0/1203317395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 msgr2=0x7f5464198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 0x7f5464198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 -- 172.21.15.26:0/1203317395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c009c40 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546899b700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 0x7f5464198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.996+0000 7f546919c700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f545c009f60 tx=0x7f545c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.997+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c00e040 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.997+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f545c00bbf0 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.997+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54641a0f90 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.997+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0052a0 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.997+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54641a13b0 con 0x7f5464102a40 2023-10-06T21:10:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:08.998+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546410c760 con 0x7f5464102a40 2023-10-06T21:10:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.000+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f545c01d470 con 0x7f5464102a40 2023-10-06T21:10:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.001+0000 7f5461ffb700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.001+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f545c0ce180 con 0x7f5464102a40 2023-10-06T21:10:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.001+0000 7f546899b700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.002+0000 7f546899b700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 0x7f54540a3990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5464101bf0 tx=0x7f5458009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.005+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f545c097830 con 0x7f5464102a40 2023-10-06T21:10:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.222+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f546410c970 con 0x7f5464102a40 2023-10-06T21:10:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.223+0000 7f5461ffb700 1 -- 172.21.15.26:0/1203317395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22548 (secure 0 0 0) 0x7f545c09ae50 con 0x7f5464102a40 2023-10-06T21:10:09.226 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:09.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:08:45.432140+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 msgr2=0x7f54540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 0x7f54540a3990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5464101bf0 tx=0x7f5458009260 comp rx=0 tx=0).stop 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 msgr2=0x7f54641a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f545c009f60 tx=0x7f545c004c60 comp rx=0 tx=0).stop 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 shutdown_connections 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f54540a14d0 0x7f54540a3990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5464102a40 0x7f54641a0470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5464100ea0 0x7f5464198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 --2- 172.21.15.26:0/1203317395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5464102080 0x7f5464199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 >> 172.21.15.26:0/1203317395 conn(0x7f54640fc420 msgr2=0x7f5464107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 shutdown_connections 2023-10-06T21:10:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.226+0000 7f546abff700 1 -- 172.21.15.26:0/1203317395 wait complete. 2023-10-06T21:10:09.248 DEBUG:tasks.fwd_scrub:Yielding 2023-10-06T21:10:09.248 INFO:teuthology.run_tasks:Running task exec... 2023-10-06T21:10:09.258 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T21:10:09.258 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:10:09.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-10-06T21:10:09.259 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-06T21:10:09.259 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:10:09.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:10:09.358 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2821938414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:09.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:09 smithi026 ceph-mon[98490]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2023-10-06T21:10:09.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2292532331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:09.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2821938414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:09 smithi078 ceph-mon[134896]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2023-10-06T21:10:09.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2292532331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 -- 172.21.15.26:0/3217229571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 msgr2=0x7f14a8073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 --2- 172.21.15.26:0/3217229571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a8073340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009990 tx=0x7f14a4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 -- 172.21.15.26:0/3217229571 shutdown_connections 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 --2- 172.21.15.26:0/3217229571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 0x7f14a810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 --2- 172.21.15.26:0/3217229571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a8073340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 --2- 172.21.15.26:0/3217229571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a8073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 -- 172.21.15.26:0/3217229571 >> 172.21.15.26:0/3217229571 conn(0x7f14a80fc480 msgr2=0x7f14a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 -- 172.21.15.26:0/3217229571 shutdown_connections 2023-10-06T21:10:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.495+0000 7f14af7ff700 1 -- 172.21.15.26:0/3217229571 wait complete. 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 Processor -- start 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 -- start start 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 0x7f14a819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819ecd0 con 0x7f14a8074ef0 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819ee40 con 0x7f14a810d4c0 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.497+0000 7f14af7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819f000 con 0x7f14a8073910 2023-10-06T21:10:09.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14ad59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39156/0 (socket says 172.21.15.26:39156) 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 -- 172.21.15.26:0/2275544304 learned_addr learned my addr 172.21.15.26:0/2275544304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14add9c700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 0x7f14a819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 -- 172.21.15.26:0/2275544304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 msgr2=0x7f14a819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 -- 172.21.15.26:0/2275544304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 msgr2=0x7f14a819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 0x7f14a819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14acd9a700 1 -- 172.21.15.26:0/2275544304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40095f0 con 0x7f14a8074ef0 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.498+0000 7f14ad59b700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f14acd9a700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f149800d980 tx=0x7f149800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498004020 con 0x7f14a8074ef0 2023-10-06T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a819f2e0 con 0x7f14a8074ef0 2023-10-06T21:10:09.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149800f690 con 0x7f14a8074ef0 2023-10-06T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498003a60 con 0x7f14a8074ef0 2023-10-06T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.499+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a81a5aa0 con 0x7f14a8074ef0 2023-10-06T21:10:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.500+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a8103010 con 0x7f14a8074ef0 2023-10-06T21:10:09.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.503+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f149800fb00 con 0x7f14a8074ef0 2023-10-06T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.504+0000 7f149e7fc700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 0x7f14940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.504+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14980ccc70 con 0x7f14a8074ef0 2023-10-06T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.504+0000 7f14ad59b700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 0x7f14940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.505+0000 7f14ad59b700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 0x7f14940a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b2e0 tx=0x7f14a4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.508+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1498096290 con 0x7f14a8074ef0 2023-10-06T21:10:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.508+0000 7f909759d700 1 -- 172.21.15.26:0/4081253307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 msgr2=0x7f9090103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.508+0000 7f909759d700 1 --2- 172.21.15.26:0/4081253307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090103bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f908c009990 tx=0x7f908c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 -- 172.21.15.26:0/4081253307 shutdown_connections 2023-10-06T21:10:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 --2- 172.21.15.26:0/4081253307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090106a20 0x7f9090108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 --2- 172.21.15.26:0/4081253307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090103bb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 --2- 172.21.15.26:0/4081253307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90901040f0 0x7f90901064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 -- 172.21.15.26:0/4081253307 >> 172.21.15.26:0/4081253307 conn(0x7f90900fb190 msgr2=0x7f90900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 -- 172.21.15.26:0/4081253307 shutdown_connections 2023-10-06T21:10:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.509+0000 7f909759d700 1 -- 172.21.15.26:0/4081253307 wait complete. 2023-10-06T21:10:09.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.510+0000 7f909759d700 1 Processor -- start 2023-10-06T21:10:09.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.510+0000 7f909759d700 1 -- start start 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90901040f0 0x7f90900ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090100450 con 0x7f9090106a20 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901005c0 con 0x7f90901017c0 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909759d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901a09b0 con 0x7f90901040f0 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f9095d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90901040f0 0x7f90900ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39178/0 (socket says 172.21.15.26:39178) 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f9096d9c700 1 -- 172.21.15.26:0/4188532170 learned_addr learned my addr 172.21.15.26:0/4188532170 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.511+0000 7f909659b700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 -- 172.21.15.26:0/4188532170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90901040f0 msgr2=0x7f90900ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90901040f0 0x7f90900ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 -- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 msgr2=0x7f90900ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 -- 172.21.15.26:0/4188532170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c0095f0 con 0x7f90901017c0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f9096d9c700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909659b700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090101130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b680 tx=0x7f908c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f90877fe700 1 -- 172.21.15.26:0/4188532170 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c01d070 con 0x7f90901017c0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f90877fe700 1 -- 172.21.15.26:0/4188532170 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c021410 con 0x7f90901017c0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901a0c30 con 0x7f90901017c0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.512+0000 7f90877fe700 1 -- 172.21.15.26:0/4188532170 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c021ea0 con 0x7f90901017c0 2023-10-06T21:10:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.513+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901a1050 con 0x7f90901017c0 2023-10-06T21:10:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.516+0000 7f90877fe700 1 -- 172.21.15.26:0/4188532170 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f908c0217f0 con 0x7f90901017c0 2023-10-06T21:10:09.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.517+0000 7f90ac0d3b80 1 Processor -- start 2023-10-06T21:10:09.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.517+0000 7f90877fe700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 0x7f907c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.517+0000 7f90877fe700 1 -- 172.21.15.26:0/4188532170 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f908c0cdf30 con 0x7f90901017c0 2023-10-06T21:10:09.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.517+0000 7f90ac0d3b80 1 -- start start 2023-10-06T21:10:09.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.517+0000 7f9096d9c700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 0x7f907c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d292700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d293160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d1f94e0 con 0x55881d292ce0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d1eaca0 con 0x55881d291490 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d1e85c0 con 0x55881d291df0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f9095d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d292700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f909659b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f909659b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52186/0 (socket says 172.21.15.26:52186) 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f9095d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d292700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60174/0 (socket says 172.21.15.26:60174) 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d293160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f909659b700 1 -- 172.21.15.26:0/2231789777 learned_addr learned my addr 172.21.15.26:0/2231789777 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.518+0000 7f909659b700 1 -- 172.21.15.26:0/2231789777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 msgr2=0x55881d292700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f9096d9c700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 0x7f907c0a39e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9090101790 tx=0x7f9088009230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f909659b700 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d292700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f909659b700 1 -- 172.21.15.26:0/2231789777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 msgr2=0x55881d293160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f909659b700 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d293160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f909659b700 1 -- 172.21.15.26:0/2231789777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881d00ee60 con 0x55881d291490 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f909659b700 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f908c09b230 tx=0x7f908c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f8138641b63f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f9084ff9700 1 -- 172.21.15.26:0/2231789777 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a1040 con 0x55881d291490 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f9084ff9700 1 -- 172.21.15.26:0/2231789777 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c09b2a0 con 0x55881d291490 2023-10-06T21:10:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.519+0000 7f9084ff9700 1 -- 172.21.15.26:0/2231789777 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c021610 con 0x55881d291490 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2231789777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 msgr2=0x55881d2918b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f908c09b230 tx=0x7f908c009310 comp rx=0 tx=0).stop 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2231789777 shutdown_connections 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d293160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2918b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2231789777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d292700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2231789777 >> 172.21.15.26:0/2231789777 conn(0x55881d2052e0 msgr2=0x55881d28c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2231789777 shutdown_connections 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.520+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2231789777 wait complete. 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.521+0000 7f90ac0d3b80 1 -- start start 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.521+0000 7f90ac0d3b80 1 Processor -- start 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d29af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d29b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d1e3eb0 con 0x55881d292ce0 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d180790 con 0x55881d291490 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d180430 con 0x55881d291df0 2023-10-06T21:10:09.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f90ac0d3b80 10 client.0 _ll_register_callbacks cb 0x55881cfdd880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f9095d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d29b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39196/0 (socket says 172.21.15.26:39196) 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f9096d9c700 1 -- 172.21.15.26:0/499137704 learned_addr learned my addr 172.21.15.26:0/499137704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.522+0000 7f909659b700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d29af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 -- 172.21.15.26:0/499137704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 msgr2=0x55881d29b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d29b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 -- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 msgr2=0x55881d29af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d29af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 -- 172.21.15.26:0/499137704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901a1050 con 0x55881d292ce0 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f909659b700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d29af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9096d9c700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9088017e90 tx=0x7f9088004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc448ea620d39000 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9079ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088003a00 con 0x55881d292ce0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55881d2841e0 con 0x55881d292ce0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.523+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9088003b60 con 0x55881d292ce0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9088003d20 con 0x55881d292ce0 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 10 client.15264 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55881cff7ce0 con 0x55881d292ce0 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9088032020 con 0x55881d292ce0 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9088023510 con 0x55881d292ce0 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 10 client.15264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 20 client.15264 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2a6dd0 0x55881d2a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 4 client.15264 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.524+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55881d1dce90 con 0x55881d2a6dd0 2023-10-06T21:10:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.525+0000 7f909659b700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2a6dd0 0x55881d2a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.525+0000 7f9079ffb700 10 client.15264 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.525+0000 7f909659b700 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2a6dd0 0x55881d2a91f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5522a67d575ef2f server_cookie=7030a96b54c4b6e6 in_seq=0 out_seq=0 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f9079ffb700 1 -- 172.21.15.26:0/499137704 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55881d1dce90 con 0x55881d2a6dd0 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f9079ffb700 10 client.15264 handle_command_reply: tid=0 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 client.15264 shutdown 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 msgr2=0x55881d2a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9088017e90 tx=0x7f9088004b40 comp rx=0 tx=0).stop 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 shutdown_connections 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2a2480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2a6dd0 0x55881d2a91f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d29af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/499137704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d29b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 >> 172.21.15.26:0/499137704 conn(0x55881d2052e0 msgr2=0x55881d28c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f9079ffb700 0 client.15264 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 shutdown_connections 2023-10-06T21:10:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.526+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/499137704 wait complete. 2023-10-06T21:10:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.527+0000 7f90ac0d3b80 20 client.15264 trim_cache size 0 max 16384 2023-10-06T21:10:09.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2821938414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:10:09.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:09 smithi167 ceph-mon[134494]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2023-10-06T21:10:09.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2292532331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 Processor -- start 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 -- start start 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 0x55881d295c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d0649a0 con 0x55881d291490 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d28d0c0 con 0x55881d292ce0 2023-10-06T21:10:09.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.545+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d28db30 con 0x55881d291df0 2023-10-06T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f909659b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 0x55881d295c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f909659b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 0x55881d295c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39206/0 (socket says 172.21.15.26:39206) 2023-10-06T21:10:09.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52210/0 (socket says 172.21.15.26:52210) 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f909659b700 1 -- 172.21.15.26:0/2350540905 learned_addr learned my addr 172.21.15.26:0/2350540905 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9095d9a700 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 -- 172.21.15.26:0/2350540905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 msgr2=0x55881d2961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 -- 172.21.15.26:0/2350540905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 msgr2=0x55881d295c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 0x55881d295c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.546+0000 7f9096d9c700 1 -- 172.21.15.26:0/2350540905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881d0352d0 con 0x55881d292ce0 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f9096d9c700 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9088017e60 tx=0x7f9088006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc903232cb8929e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f90857fa700 1 -- 172.21.15.26:0/2350540905 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088023d30 con 0x55881d292ce0 2023-10-06T21:10:09.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f90857fa700 1 -- 172.21.15.26:0/2350540905 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9088023e90 con 0x55881d292ce0 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f90857fa700 1 -- 172.21.15.26:0/2350540905 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088004470 con 0x55881d292ce0 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2350540905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 msgr2=0x55881d2a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.547+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9088017e60 tx=0x7f9088006160 comp rx=0 tx=0).stop 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2350540905 shutdown_connections 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d292ce0 0x55881d2a23a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d291490 0x55881d295c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2350540905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2350540905 >> 172.21.15.26:0/2350540905 conn(0x55881d2052e0 msgr2=0x55881d28c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2350540905 shutdown_connections 2023-10-06T21:10:09.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2350540905 wait complete. 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 -- start start 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.548+0000 7f90ac0d3b80 1 Processor -- start 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d2fb410 con 0x55881d292ce0 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d2fb580 con 0x55881d291490 2023-10-06T21:10:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881d2fb740 con 0x55881d291df0 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f90ac0d3b80 10 client.0 _ll_register_callbacks cb 0x7f9088017a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.549+0000 7f909659b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f9096d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52222/0 (socket says 172.21.15.26:52222) 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 -- 172.21.15.26:0/2886044829 learned_addr learned my addr 172.21.15.26:0/2886044829 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f9095d9a700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 -- 172.21.15.26:0/2886044829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 msgr2=0x55881d2a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 -- 172.21.15.26:0/2886044829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 msgr2=0x55881d2faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 -- 172.21.15.26:0/2886044829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55881cff7ce0 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f909659b700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f908c097d10 tx=0x7f908c0993e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1ca745d5e19915 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f9079ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a1040 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55881d28bfc0 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.550+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f908c0a35a0 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908c0b9040 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 10 client.25331 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55881d00d330 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c0a7020 con 0x55881d291490 2023-10-06T21:10:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f908c0a11a0 con 0x55881d291490 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 10 client.25331 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 20 client.25331 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2fcdc0 0x55881d2ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 4 client.25331 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f908c09b2a0 con 0x55881d2fcdc0 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.551+0000 7f9095d9a700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2fcdc0 0x55881d2ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.552+0000 7f9095d9a700 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2fcdc0 0x55881d2ff1e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce33293b48e6c1b3 server_cookie=a3a719525044fcc2 in_seq=0 out_seq=0 2023-10-06T21:10:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.552+0000 7f9079ffb700 10 client.25331 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f9079ffb700 1 -- 172.21.15.26:0/2886044829 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f908c09b2a0 con 0x55881d2fcdc0 2023-10-06T21:10:09.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f9079ffb700 10 client.25331 handle_command_reply: tid=0 2023-10-06T21:10:09.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 client.25331 shutdown 2023-10-06T21:10:09.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 msgr2=0x55881d2a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f908c097d10 tx=0x7f908c0993e0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 shutdown_connections 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55881d292ce0 0x55881d2faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55881d2fcdc0 0x55881d2ff1e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55881d291490 0x55881d2a16a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 --2- 172.21.15.26:0/2886044829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881d291df0 0x55881d2a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f9079ffb700 0 client.25331 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.553+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 >> 172.21.15.26:0/2886044829 conn(0x55881d2052e0 msgr2=0x55881d295c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 shutdown_connections 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f90ac0d3b80 1 -- 172.21.15.26:0/2886044829 wait complete. 2023-10-06T21:10:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f90ac0d3b80 20 client.25331 trim_cache size 0 max 16384 2023-10-06T21:10:09.561 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:10:09.561 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:10:09.561 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "c49c9b10-4278-4554-8aa8-0781544ecfe6", 2023-10-06T21:10:09.561 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:10:09.561 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 msgr2=0x7f907c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 0x7f907c0a39e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9090101790 tx=0x7f9088009230 comp rx=0 tx=0).stop 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 msgr2=0x7f9090101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090101130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b680 tx=0x7f908c005740 comp rx=0 tx=0).stop 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 shutdown_connections 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f907c0a1520 0x7f907c0a39e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090106a20 0x7f90900ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90901017c0 0x7f9090101130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 --2- 172.21.15.26:0/4188532170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90901040f0 0x7f90900ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.554+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 >> 172.21.15.26:0/4188532170 conn(0x7f90900fb190 msgr2=0x7f9090102430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.555+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 shutdown_connections 2023-10-06T21:10:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.555+0000 7f909759d700 1 -- 172.21.15.26:0/4188532170 wait complete. 2023-10-06T21:10:09.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/4137686872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102070 msgr2=0x7f0a781024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/4137686872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102070 0x7f0a781024f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009990 tx=0x7f0a6c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/4137686872 shutdown_connections 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/4137686872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102a30 0x7f0a78109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/4137686872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/4137686872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102070 0x7f0a781024f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/4137686872 >> 172.21.15.26:0/4137686872 conn(0x7f0a780fc430 msgr2=0x7f0a780fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/4137686872 shutdown_connections 2023-10-06T21:10:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.779+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/4137686872 wait complete. 2023-10-06T21:10:09.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 Processor -- start 2023-10-06T21:10:09.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 -- start start 2023-10-06T21:10:09.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102070 0x7f0a781993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 0x7f0a781a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a09e0 con 0x7f0a78102a30 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a0b20 con 0x7f0a78102070 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.780+0000 7f0a7d9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a0cb0 con 0x7f0a78100e90 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 0x7f0a781a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60230/0 (socket says 172.21.15.26:60230) 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a76ffd700 1 -- 172.21.15.26:0/3753931696 learned_addr learned my addr 172.21.15.26:0/3753931696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102070 0x7f0a781993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 -- 172.21.15.26:0/3753931696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 msgr2=0x7f0a78198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 -- 172.21.15.26:0/3753931696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102070 msgr2=0x7f0a781993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102070 0x7f0a781993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 -- 172.21.15.26:0/3753931696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70009c40 con 0x7f0a78102a30 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.781+0000 7f0a777fe700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 0x7f0a781a0410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000efb0 tx=0x7f0a7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.782+0000 7f0a7c9fd700 1 -- 172.21.15.26:0/3753931696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003a00 con 0x7f0a78102a30 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.782+0000 7f0a7c9fd700 1 -- 172.21.15.26:0/3753931696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a7000f040 con 0x7f0a78102a30 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.782+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c0095f0 con 0x7f0a78102a30 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.782+0000 7f0a7c9fd700 1 -- 172.21.15.26:0/3753931696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70004540 con 0x7f0a78102a30 2023-10-06T21:10:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.782+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a1230 con 0x7f0a78102a30 2023-10-06T21:10:09.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.786+0000 7f0a9133cb80 1 Processor -- start 2023-10-06T21:10:09.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a7c9fd700 1 -- 172.21.15.26:0/3753931696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0a70004780 con 0x7f0a78102a30 2023-10-06T21:10:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 -- start start 2023-10-06T21:10:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 0x558443e98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443dff460 con 0x558443e97bc0 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443df0c20 con 0x558443e98a60 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.787+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443dee540 con 0x558443e97260 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a767fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 0x558443e98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39236/0 (socket says 172.21.15.26:39236) 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 -- 172.21.15.26:0/3252801691 learned_addr learned my addr 172.21.15.26:0/3252801691 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 -- 172.21.15.26:0/3252801691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 msgr2=0x558443e97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e97680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 -- 172.21.15.26:0/3252801691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 msgr2=0x558443e98ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 0x558443e98ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 -- 172.21.15.26:0/3252801691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558443c14de0 con 0x558443e97bc0 2023-10-06T21:10:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 0x558443e98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.788+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a68011650 tx=0x7f0a68011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed8cea5fc6e751e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a7c9fd700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 0x7f0a600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a7c9fd700 1 -- 172.21.15.26:0/3753931696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0a700cc910 con 0x7f0a78102a30 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a5dffb700 1 -- 172.21.15.26:0/3252801691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004a30 con 0x558443e97bc0 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 0x7f0a600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a5dffb700 1 -- 172.21.15.26:0/3252801691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a680054e0 con 0x558443e97bc0 2023-10-06T21:10:09.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.789+0000 7f0a5dffb700 1 -- 172.21.15.26:0/3252801691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680197a0 con 0x558443e97bc0 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3252801691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 msgr2=0x558443e98480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a68011650 tx=0x7f0a68011960 comp rx=0 tx=0).stop 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 0x7f0a600a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b2e0 tx=0x7f0a6c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3252801691 shutdown_connections 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e98a60 0x558443e98ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3252801691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443e98480 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3252801691 >> 172.21.15.26:0/3252801691 conn(0x558443e0b260 msgr2=0x558443e92140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3252801691 shutdown_connections 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.790+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3252801691 wait complete. 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 -- start start 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 Processor -- start 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97260 0x558443ea1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 0x558443ea8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443de9e30 con 0x558443e97bc0 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443d86710 con 0x558443e97260 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.791+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443d863b0 con 0x558443e98a60 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a9133cb80 10 client.0 _ll_register_callbacks cb 0x558443be3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39248/0 (socket says 172.21.15.26:39248) 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97260 0x558443ea1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 -- 172.21.15.26:0/3596860654 learned_addr learned my addr 172.21.15.26:0/3596860654 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a777fe700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 0x558443ea8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 -- 172.21.15.26:0/3596860654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 msgr2=0x558443ea8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 0x558443ea8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 -- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97260 msgr2=0x558443ea1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97260 0x558443ea1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a767fc700 1 -- 172.21.15.26:0/3596860654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c020040 con 0x558443e97bc0 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.792+0000 7f0a777fe700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 0x558443ea8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a767fc700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a78100bb0 tx=0x7f0a6c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32993927888a731e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c028860 con 0x558443e97bc0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558443e89f10 con 0x558443e97bc0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0a6c0289c0 con 0x558443e97bc0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c028b80 con 0x558443e97bc0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a9133cb80 10 client.15282 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f0a68009e10 con 0x558443e97bc0 2023-10-06T21:10:09.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.793+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558443e89f10 con 0x558443e97bc0 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a6c03e3e0 con 0x558443e97bc0 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 10 client.15282 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 20 client.15282 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443eacea0 0x558443eaf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 4 client.15282 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558443de2e10 con 0x558443eacea0 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.794+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443eacea0 0x558443eaf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.795+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443eacea0 0x558443eaf2c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b0b895a78c3271 server_cookie=55cc6ac8e7bd17b5 in_seq=0 out_seq=0 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.795+0000 7f0a56ffd700 10 client.15282 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a56ffd700 1 -- 172.21.15.26:0/3596860654 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558443de2e10 con 0x558443eacea0 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a56ffd700 10 client.15282 handle_command_reply: tid=0 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 client.15282 shutdown 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 msgr2=0x558443ea1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a78100bb0 tx=0x7f0a6c009310 comp rx=0 tx=0).stop 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 shutdown_connections 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e98a60 0x558443ea8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443eacea0 0x558443eaf2c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97260 0x558443ea1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a56ffd700 0 client.15282 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/3596860654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e97bc0 0x558443ea1540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 >> 172.21.15.26:0/3596860654 conn(0x558443e0b260 msgr2=0x558443e91f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 shutdown_connections 2023-10-06T21:10:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 1 -- 172.21.15.26:0/3596860654 wait complete. 2023-10-06T21:10:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.796+0000 7f0a9133cb80 20 client.15282 trim_cache size 0 max 16384 2023-10-06T21:10:09.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.806+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f14a804ec90 con 0x7f14a8074ef0 2023-10-06T21:10:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 Processor -- start 2023-10-06T21:10:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 -- start start 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e91f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443e9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443c6a920 con 0x558443e98a60 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443ea04b0 con 0x558443e97bc0 2023-10-06T21:10:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443e93850 con 0x558443e97260 2023-10-06T21:10:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443e9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a767fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e91f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a767fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52274/0 (socket says 172.21.15.26:52274) 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.814+0000 7f0a767fc700 1 -- 172.21.15.26:0/95883833 learned_addr learned my addr 172.21.15.26:0/95883833 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 -- 172.21.15.26:0/95883833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 msgr2=0x558443e91f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e91f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 -- 172.21.15.26:0/95883833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 msgr2=0x558443e9ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443e9ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 -- 172.21.15.26:0/95883833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558443bfdc60 con 0x558443e97bc0 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e91f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a777fe700 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443e9ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a767fc700 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c03e710 tx=0x7f0a6c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12685ecebaafa95f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a5e7fc700 1 -- 172.21.15.26:0/95883833 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c03e3e0 con 0x558443e97bc0 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a5e7fc700 1 -- 172.21.15.26:0/95883833 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6c03e540 con 0x558443e97bc0 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.815+0000 7f0a5e7fc700 1 -- 172.21.15.26:0/95883833 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c03f460 con 0x558443e97bc0 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- 172.21.15.26:0/95883833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 msgr2=0x558443ea8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c03e710 tx=0x7f0a6c005590 comp rx=0 tx=0).stop 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- 172.21.15.26:0/95883833 shutdown_connections 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443e9ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443e91f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/95883833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea8540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- 172.21.15.26:0/95883833 >> 172.21.15.26:0/95883833 conn(0x558443e0b260 msgr2=0x558443e9fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- 172.21.15.26:0/95883833 shutdown_connections 2023-10-06T21:10:09.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- 172.21.15.26:0/95883833 wait complete. 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.816+0000 7f0a9133cb80 1 -- start start 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 Processor -- start 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443ea76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443ea7bf0 con 0x558443e98a60 2023-10-06T21:10:09.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443ea7d60 con 0x558443e97bc0 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558443ea7f20 con 0x558443e97260 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a9133cb80 10 client.0 _ll_register_callbacks cb 0x558443be3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443ea76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60280/0 (socket says 172.21.15.26:60280) 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.817+0000 7f0a76ffd700 1 -- 172.21.15.26:0/53675605 learned_addr learned my addr 172.21.15.26:0/53675605 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a767fc700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 -- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 msgr2=0x558443ea66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 -- 172.21.15.26:0/53675605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 msgr2=0x558443ea76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443ea76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 -- 172.21.15.26:0/53675605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558443c14de0 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a76ffd700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a6801b040 tx=0x7f0a68005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf5b895127ea9b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.818+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68005120 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558443de2b30 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0a68005280 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6802b130 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 10 client.25327 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f0a781a1230 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558443de2b30 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a680046c0 con 0x558443e97260 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 10 client.25327 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 20 client.25327 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443f01ad0 0x558443f03ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 4 client.25327 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.819+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558443c35f70 con 0x558443f01ad0 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a767fc700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443f01ad0 0x558443f03ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a767fc700 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443f01ad0 0x558443f03ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1cb91071054b7 server_cookie=49f85845f6f6a8b5 in_seq=0 out_seq=0 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a56ffd700 10 client.25327 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a56ffd700 1 -- 172.21.15.26:0/53675605 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558443c35f70 con 0x558443f01ad0 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a56ffd700 10 client.25327 handle_command_reply: tid=0 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.820+0000 7f0a9133cb80 1 client.25327 shutdown 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 msgr2=0x558443ea6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a6801b040 tx=0x7f0a68005db0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 shutdown_connections 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558443e98a60 0x558443ea76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558443e97260 0x558443ea6160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558443f01ad0 0x558443f03ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 --2- 172.21.15.26:0/53675605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558443e97bc0 0x558443ea66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 >> 172.21.15.26:0/53675605 conn(0x558443e0b260 msgr2=0x558443e9b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a56ffd700 0 client.25327 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 shutdown_connections 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 1 -- 172.21.15.26:0/53675605 wait complete. 2023-10-06T21:10:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a9133cb80 20 client.25327 trim_cache size 0 max 16384 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "c49c9b10-4278-4554-8aa8-0781544ecfe6": { 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "c49c9b10-4278-4554-8aa8-0781544ecfe6", 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:10:09.829 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:10:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.821+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 msgr2=0x7f0a600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 0x7f0a600a3940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b2e0 tx=0x7f0a6c01a040 comp rx=0 tx=0).stop 2023-10-06T21:10:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 msgr2=0x7f0a781a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 0x7f0a781a0410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000efb0 tx=0x7f0a7000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 shutdown_connections 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f0a600a1480 0x7f0a600a3940 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a78102a30 0x7f0a781a0410 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a78100e90 0x7f0a78198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 --2- 172.21.15.26:0/3753931696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a78102070 0x7f0a781993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 >> 172.21.15.26:0/3753931696 conn(0x7f0a780fc430 msgr2=0x7f0a780fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 shutdown_connections 2023-10-06T21:10:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:09.822+0000 7f0a7d9ff700 1 -- 172.21.15.26:0/3753931696 wait complete. 2023-10-06T21:10:09.842 INFO:tasks.cephfs.filesystem:scrub status for tag:c49c9b10-4278-4554-8aa8-0781544ecfe6 - {'path': '/', 'tag': 'c49c9b10-4278-4554-8aa8-0781544ecfe6', 'options': 'recursive,force'} 2023-10-06T21:10:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.093+0000 7f149e7fc700 1 -- 172.21.15.26:0/2275544304 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f14980998b0 con 0x7f14a8074ef0 2023-10-06T21:10:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 msgr2=0x7f14940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 0x7f14940a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b2e0 tx=0x7f14a4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 msgr2=0x7f14a819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f149800d980 tx=0x7f149800dc90 comp rx=0 tx=0).stop 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 shutdown_connections 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a810d4c0 0x7f14a819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14a8073910 0x7f14a819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f14940a1480 0x7f14940a3940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 --2- 172.21.15.26:0/2275544304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14a8074ef0 0x7f14a819d7f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.096+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 >> 172.21.15.26:0/2275544304 conn(0x7f14a80fc480 msgr2=0x7f14a80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.097+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 shutdown_connections 2023-10-06T21:10:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.099+0000 7f14af7ff700 1 -- 172.21.15.26:0/2275544304 wait complete. 2023-10-06T21:10:10.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-10-06T21:10:10.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 -- 172.21.15.26:0/4077200872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe81042a0 msgr2=0x7fefe8106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 --2- 172.21.15.26:0/4077200872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe81042a0 0x7fefe8106690 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009990 tx=0x7fefdc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 -- 172.21.15.26:0/4077200872 shutdown_connections 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 --2- 172.21.15.26:0/4077200872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8106bd0 0x7fefe8109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 --2- 172.21.15.26:0/4077200872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe8101970 0x7fefe8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 --2- 172.21.15.26:0/4077200872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe81042a0 0x7fefe8106690 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 -- 172.21.15.26:0/4077200872 >> 172.21.15.26:0/4077200872 conn(0x7fefe80fb340 msgr2=0x7fefe80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 -- 172.21.15.26:0/4077200872 shutdown_connections 2023-10-06T21:10:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.332+0000 7fefee1ff700 1 -- 172.21.15.26:0/4077200872 wait complete. 2023-10-06T21:10:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 Processor -- start 2023-10-06T21:10:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 -- start start 2023-10-06T21:10:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8101970 0x7fefe8196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 0x7fefe8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8198740 con 0x7fefe8101970 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe81988b0 con 0x7fefe8106bd0 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.333+0000 7fefee1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8198a70 con 0x7fefe81042a0 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8101970 0x7fefe8196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 0x7fefe8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60308/0 (socket says 172.21.15.26:60308) 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 -- 172.21.15.26:0/3624500295 learned_addr learned my addr 172.21.15.26:0/3624500295 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 msgr2=0x7fefe8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 0x7fefe8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8101970 msgr2=0x7fefe8196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8101970 0x7fefe8196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 -- 172.21.15.26:0/3624500295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0095f0 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe7fff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 0x7fefe8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.334+0000 7fefe6ffd700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefdc004740 tx=0x7fefdc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.335+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc01d070 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.335+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefdc004830 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.335+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0053c0 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.335+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe8198cf0 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.335+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe819f460 con 0x7fefe81042a0 2023-10-06T21:10:10.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.336+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80fcec0 con 0x7fefe81042a0 2023-10-06T21:10:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.338+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7fefdc005560 con 0x7fefe81042a0 2023-10-06T21:10:10.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.339+0000 7fefe4ff9700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 0x7fefd00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.339+0000 7fefe77fe700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 msgr2=0x7fefd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:10.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.339+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fefdc0ce340 con 0x7fefe81042a0 2023-10-06T21:10:10.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.339+0000 7fefe77fe700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 0x7fefd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:10:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.341+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fefdc097a30 con 0x7fefe81042a0 2023-10-06T21:10:10.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1203317395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:10.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: ignoring --setuser ceph since I am not root 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: ignoring --setgroup ceph since I am not root 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-mgr[136399]: -- 172.21.15.167:0/1040961373 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556a47a7b1e0 con 0x556a47a3d000 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-mgr[136399]: -- 172.21.15.167:0/1040961373 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a47a671e0 con 0x556a47a3d000 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:10.275+0000 7f407407e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:10.356+0000 7f407407e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:10.360 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1203317395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: ignoring --setuser ceph since I am not root 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: ignoring --setgroup ceph since I am not root 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-mgr[136380]: -- 172.21.15.78:0/1278239379 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e82f56d1e0 con 0x55e82f542800 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:10.277+0000 7fed58008200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:10:10.361 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:10.356+0000 7fed58008200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:10.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1203317395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:10:10.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-10-06T21:10:10.389 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setuser ceph since I am not root 2023-10-06T21:10:10.389 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: ignoring --setgroup ceph since I am not root 2023-10-06T21:10:10.390 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:10.386+0000 7fc9ef37b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T21:10:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.540+0000 7fefe77fe700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 msgr2=0x7fefd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.540+0000 7fefe77fe700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 0x7fefd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:10:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.557+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fefe804ec90 con 0x7fefe81042a0 2023-10-06T21:10:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.563+0000 7fefe4ff9700 1 -- 172.21.15.26:0/3624500295 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fefdc09b050 con 0x7fefe81042a0 2023-10-06T21:10:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 msgr2=0x7fefd00a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:10:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 0x7fefd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 msgr2=0x7fefe8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefdc004740 tx=0x7fefdc009580 comp rx=0 tx=0).stop 2023-10-06T21:10:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 shutdown_connections 2023-10-06T21:10:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefe8106bd0 0x7fefe8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fefd00a14d0 0x7fefd00a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe8101970 0x7fefe8196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 --2- 172.21.15.26:0/3624500295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefe81042a0 0x7fefe8197290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 >> 172.21.15.26:0/3624500295 conn(0x7fefe80fb340 msgr2=0x7fefe81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 shutdown_connections 2023-10-06T21:10:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.566+0000 7fefee1ff700 1 -- 172.21.15.26:0/3624500295 wait complete. 2023-10-06T21:10:10.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-10-06T21:10:10.620 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:10.447+0000 7fed58008200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:10:10.620 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:10.541+0000 7fed58008200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:10:10.651 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:10.496+0000 7fc9ef37b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:10.651 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:10.621+0000 7fc9ef37b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:10:10.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:10.448+0000 7f407407e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T21:10:10.716 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:10.541+0000 7f407407e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:10:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 -- 172.21.15.26:0/1843947409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508073910 msgr2=0x7f4508073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1843947409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508073910 0x7f4508073d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009990 tx=0x7f44fc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 -- 172.21.15.26:0/1843947409 shutdown_connections 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1843947409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508108dd0 0x7f450810b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1843947409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508074ef0 0x7f4508073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1843947409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508073910 0x7f4508073d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 -- 172.21.15.26:0/1843947409 >> 172.21.15.26:0/1843947409 conn(0x7f45080fc460 msgr2=0x7f45080fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 -- 172.21.15.26:0/1843947409 shutdown_connections 2023-10-06T21:10:10.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.790+0000 7f450d1ff700 1 -- 172.21.15.26:0/1843947409 wait complete. 2023-10-06T21:10:10.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.791+0000 7f450d1ff700 1 Processor -- start 2023-10-06T21:10:10.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 -- start start 2023-10-06T21:10:10.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508073910 0x7f4508198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508108dd0 0x7f450819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819a8f0 con 0x7f4508073910 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819aa60 con 0x7f4508074ef0 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450d1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450819ac20 con 0x7f4508108dd0 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508108dd0 0x7f450819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52310/0 (socket says 172.21.15.26:52310) 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f450659c700 1 -- 172.21.15.26:0/1619335557 learned_addr learned my addr 172.21.15.26:0/1619335557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.792+0000 7f4506d9d700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508073910 0x7f4508198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:10.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508108dd0 msgr2=0x7f450819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508108dd0 0x7f450819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508073910 msgr2=0x7f4508198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508073910 0x7f4508198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 -- 172.21.15.26:0/1619335557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0095f0 con 0x7f4508074ef0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450659c700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00b680 tx=0x7f44fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc01d070 con 0x7f4508074ef0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44fc0054f0 con 0x7f4508074ef0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450819aea0 con 0x7f4508074ef0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004bc0 con 0x7f4508074ef0 2023-10-06T21:10:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.793+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a16c0 con 0x7f4508074ef0 2023-10-06T21:10:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.794+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4508107480 con 0x7f4508074ef0 2023-10-06T21:10:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.796+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7f44fc00fbc0 con 0x7f4508074ef0 2023-10-06T21:10:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.797+0000 7f44effff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 0x7f44f00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.797+0000 7f4506d9d700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 msgr2=0x7f44f00a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.798+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44fc0cdf10 con 0x7f4508074ef0 2023-10-06T21:10:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.798+0000 7f4506d9d700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 0x7f44f00a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:10:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.800+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f44fc097600 con 0x7f4508074ef0 2023-10-06T21:10:10.992 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:10.750+0000 7fc9ef37b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T21:10:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.999+0000 7f4506d9d700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 msgr2=0x7f44f00a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:10.999+0000 7f4506d9d700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 0x7f44f00a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:10:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.013+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f450810e900 con 0x7f4508074ef0 2023-10-06T21:10:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.019+0000 7f44effff700 1 -- 172.21.15.26:0/1619335557 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f44fc09ac20 con 0x7f4508074ef0 2023-10-06T21:10:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 msgr2=0x7f44f00a3a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 0x7f44f00a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 msgr2=0x7f4508199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00b680 tx=0x7f44fc004730 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 shutdown_connections 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4508108dd0 0x7f450819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7f44f00a1550 0x7f44f00a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4508073910 0x7f4508198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 --2- 172.21.15.26:0/1619335557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4508074ef0 0x7f4508199410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 >> 172.21.15.26:0/1619335557 conn(0x7f45080fc460 msgr2=0x7f45080fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 shutdown_connections 2023-10-06T21:10:11.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.022+0000 7f450d1ff700 1 -- 172.21.15.26:0/1619335557 wait complete. 2023-10-06T21:10:11.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:10.713+0000 7f407407e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:10:11.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-10-06T21:10:11.080 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:10.712+0000 7fed58008200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:10:11.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.245+0000 7fade8dff700 1 -- 172.21.15.26:0/3978785527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 msgr2=0x7fade4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.245+0000 7fade8dff700 1 --2- 172.21.15.26:0/3978785527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4109370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b0a0 tx=0x7faddc00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 -- 172.21.15.26:0/3978785527 shutdown_connections 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 --2- 172.21.15.26:0/3978785527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4109370 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 --2- 172.21.15.26:0/3978785527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4100eb0 0x7fade41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 --2- 172.21.15.26:0/3978785527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fade4102090 0x7fade4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 -- 172.21.15.26:0/3978785527 >> 172.21.15.26:0/3978785527 conn(0x7fade40fc430 msgr2=0x7fade40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 -- 172.21.15.26:0/3978785527 shutdown_connections 2023-10-06T21:10:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.246+0000 7fade8dff700 1 -- 172.21.15.26:0/3978785527 wait complete. 2023-10-06T21:10:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.247+0000 7fade8dff700 1 Processor -- start 2023-10-06T21:10:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.247+0000 7fade8dff700 1 -- start start 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4100eb0 0x7fade40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fade4102090 0x7fade40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a0db0 con 0x7fade4100eb0 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a0ef0 con 0x7fade4102a50 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade8dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41a1030 con 0x7fade4102090 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fadda59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4100eb0 0x7fade40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fade4102090 0x7fade40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:32868/0 (socket says 172.21.15.26:32868) 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.248+0000 7fade2d9d700 1 -- 172.21.15.26:0/1111788484 learned_addr learned my addr 172.21.15.26:0/1111788484 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fade4102090 msgr2=0x7fade40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fade4102090 0x7fade40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4100eb0 msgr2=0x7fade40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fade4100eb0 0x7fade40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc009d00 con 0x7fade4102a50 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade2d9d700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4073d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faddc009fd0 tx=0x7faddc00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00e040 con 0x7fade4102a50 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade41a1260 con 0x7fade4102a50 2023-10-06T21:10:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faddc0094a0 con 0x7fade4102a50 2023-10-06T21:10:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc01b6c0 con 0x7fade4102a50 2023-10-06T21:10:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.249+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade41a1680 con 0x7fade4102a50 2023-10-06T21:10:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.250+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade404ec90 con 0x7fade4102a50 2023-10-06T21:10:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.252+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faddc01d470 con 0x7fade4102a50 2023-10-06T21:10:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.253+0000 7faddbfff700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.254+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.254+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faddc0cdd30 con 0x7fade4102a50 2023-10-06T21:10:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.254+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T21:10:11.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.257+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faddc0973e0 con 0x7fade4102a50 2023-10-06T21:10:11.323 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:10.989+0000 7fc9ef37b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:10:11.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.370 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:11.078+0000 7fed58008200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:10:11.371 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:11.214+0000 7fed58008200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:10:11.371 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:11.305+0000 7fed58008200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:10:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:10:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.401 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:11.080+0000 7f407407e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:10:11.401 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:11.214+0000 7f407407e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:11.303+0000 7f407407e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2275544304' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:10:11.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' 2023-10-06T21:10:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.455+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.455+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T21:10:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.500+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fade40612f0 con 0x7fadd00a14d0 2023-10-06T21:10:11.768 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:11.477+0000 7fc9ef37b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T21:10:11.769 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:11.646+0000 7fc9ef37b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T21:10:11.782 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:11.398+0000 7f407407e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:10:11.821 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:11.399+0000 7fed58008200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:10:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.856+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:11.856+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T21:10:12.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:11.765+0000 7fc9ef37b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T21:10:12.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:11.894+0000 7fc9ef37b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T21:10:12.120 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:11.818+0000 7fed58008200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:10:12.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:11.819+0000 7f407407e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:10:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:12.657+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:12.657+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T21:10:12.782 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:12.476+0000 7f407407e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:10:12.783 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:12.558+0000 7f407407e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:10:12.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:12.450+0000 7fc9ef37b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T21:10:12.870 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:12.470+0000 7fed58008200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:10:12.870 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:12.552+0000 7fed58008200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:10:13.589 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:13.322+0000 7fed58008200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:13.589 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:13.413+0000 7fed58008200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:10:13.602 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:13.333+0000 7f407407e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:13.603 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:13.425+0000 7f407407e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:10:13.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:13.333+0000 7fc9ef37b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T21:10:13.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:13.443+0000 7fc9ef37b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T21:10:13.867 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:13.586+0000 7fed58008200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:13.882 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:13.600+0000 7f407407e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:14.120 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:13.865+0000 7fed58008200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:10:14.121 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:14.090+0000 7fed58008200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:10:14.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:14.258+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:14.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:14.258+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T21:10:14.261 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:13.879+0000 7f407407e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:10:14.261 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:14.110+0000 7f407407e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:10:14.504 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:14.238+0000 7fed58008200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:10:14.524 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:14.258+0000 7f407407e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:10:14.782 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:14.522+0000 7f407407e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:10:14.805 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:14.443+0000 7fc9ef37b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:14.805 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:14.568+0000 7fc9ef37b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T21:10:14.870 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:14.501+0000 7fed58008200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:10:15.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:14.802+0000 7fc9ef37b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T21:10:15.466 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:15.179+0000 7fc9ef37b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T21:10:15.529 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:15.201+0000 7fed58008200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:10:15.530 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:15.332+0000 7fed58008200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:10:15.532 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:15.229+0000 7f407407e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:10:15.532 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:15.361+0000 7f407407e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:10:15.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:15.463+0000 7fc9ef37b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T21:10:15.823 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:15.667+0000 7fc9ef37b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T21:10:15.862 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:15.527+0000 7fed58008200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:10:15.862 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:15.641+0000 7fed58008200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:15.890 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:15.556+0000 7f407407e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:10:15.890 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:15.671+0000 7f407407e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:16.121 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:15.859+0000 7fed58008200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:10:16.282 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:15.887+0000 7f407407e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:10:16.325 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:16.028+0000 7fc9ef37b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T21:10:16.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:16.794+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 20) v1 ==== 136020+0+0 (secure 0 0 0) 0x7faddc0977b0 con 0x7fade4102a50 2023-10-06T21:10:17.032 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z[136375]: 2023-10-06T21:10:16.763+0000 7f407407e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: Standby manager daemon y restarted 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: Standby manager daemon y started 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: Standby manager daemon z restarted 2023-10-06T21:10:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: Standby manager daemon z started 2023-10-06T21:10:17.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:10:17.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:10:17.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:16 smithi167 ceph-mon[134494]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:17.006+0000 7fc9ef37b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T21:10:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: Standby manager daemon y restarted 2023-10-06T21:10:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: Standby manager daemon y started 2023-10-06T21:10:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: Standby manager daemon z restarted 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: Standby manager daemon z started 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:10:17.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:16 smithi026 ceph-mon[98490]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.121 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y[136355]: 2023-10-06T21:10:16.719+0000 7fed58008200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: Standby manager daemon y restarted 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: Standby manager daemon y started 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.78:0/645240559' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: Standby manager daemon z restarted 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: Standby manager daemon z started 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-06T21:10:17.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-06T21:10:17.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:16 smithi078 ceph-mon[134896]: from='mgr.? 172.21.15.167:0/3898498813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-06T21:10:17.452 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:17.188+0000 7fc9ef37b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T21:10:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:17.461+0000 7fadda59c700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2830651730 2023-10-06T21:10:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:17.461+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-06T21:10:17.800 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:17.449+0000 7fc9ef37b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T21:10:17.800 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:17.601+0000 7fc9ef37b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T21:10:18.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:17.893+0000 7fc9ef37b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T21:10:18.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:17 smithi026 ceph-mon[98490]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-10-06T21:10:18.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:17 smithi078 ceph-mon[134896]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-10-06T21:10:18.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:17 smithi167 ceph-mon[134494]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-10-06T21:10:19.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:19.027+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 21) v1 ==== 135688+0+0 (secure 0 0 0) 0x7faddc0cb870 con 0x7fade4102a50 2023-10-06T21:10:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:19.027+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 msgr2=0x7fadd00a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:10:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:19.027+0000 7faddbfff700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/2830651730,v1:172.21.15.26:6801/2830651730] conn(0x7fadd00a14d0 0x7fadd00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: Active manager daemon x restarted 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: Activating manager daemon x 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: mgrmap e21: x(active, starting, since 0.0125987s), standbys: z, y 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:10:19.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:10:19.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:10:19.326 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.010+0000 7fc9ef37b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc94f45a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc94f45a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc94f45a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc94f45a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc94f45a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc950c5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc950c5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc950c5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc950c5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.327 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:10:19.237+0000 7fc950c5d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:10:19.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:19 smithi026 ceph-mon[98490]: Manager daemon x is now available 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: Active manager daemon x restarted 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: Activating manager daemon x 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: mgrmap e21: x(active, starting, since 0.0125987s), standbys: z, y 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:10:19.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:10:19.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:10:19.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:10:19.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:19 smithi078 ceph-mon[134896]: Manager daemon x is now available 2023-10-06T21:10:19.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: Active manager daemon x restarted 2023-10-06T21:10:19.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: Activating manager daemon x 2023-10-06T21:10:19.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: osdmap e88: 12 total, 12 up, 12 in 2023-10-06T21:10:19.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: mgrmap e21: x(active, starting, since 0.0125987s), standbys: z, y 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:10:19.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-06T21:10:19.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-06T21:10:19.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:19 smithi167 ceph-mon[134494]: Manager daemon x is now available 2023-10-06T21:10:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.032+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mon.2 v2:172.21.15.78:3300/0 9 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7faddc098e50 con 0x7fade4102a50 2023-10-06T21:10:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.033+0000 7faddbfff700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fadd0026350 0x7fadd0026730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.033+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fade40612f0 con 0x7fadd0026350 2023-10-06T21:10:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.035+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fadd0026350 0x7fadd0026730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.042+0000 7fadda59c700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fadd0026350 0x7fadd0026730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fade4074bb0 tx=0x7fadd4009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.115+0000 7faddbfff700 1 -- 172.21.15.26:0/1111788484 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fade40612f0 con 0x7fadd0026350 2023-10-06T21:10:20.118 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path / 2023-10-06T21:10:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.117+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fadd0026350 msgr2=0x7fadd0026730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.117+0000 7fade8dff700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fadd0026350 0x7fadd0026730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fade4074bb0 tx=0x7fadd4009260 comp rx=0 tx=0).stop 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.117+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 msgr2=0x7fade4073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.117+0000 7fade8dff700 1 --2- 172.21.15.26:0/1111788484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade4102a50 0x7fade4073d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faddc009fd0 tx=0x7faddc00bce0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.118+0000 7fade2d9d700 1 -- 172.21.15.26:0/1111788484 reap_dead start 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.118+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 shutdown_connections 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.118+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 >> 172.21.15.26:0/1111788484 conn(0x7fade40fc430 msgr2=0x7fade40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.118+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 shutdown_connections 2023-10-06T21:10:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.118+0000 7fade8dff700 1 -- 172.21.15.26:0/1111788484 wait complete. 2023-10-06T21:10:20.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-10-06T21:10:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:20 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:20 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:20 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:10:20.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:20 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:10:20.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:20 smithi026 ceph-mon[98490]: mgrmap e22: x(active, since 1.01953s), standbys: z, y 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 -- 172.21.15.26:0/2467010632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 msgr2=0x7faf40101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 --2- 172.21.15.26:0/2467010632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40101160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009990 tx=0x7faf3c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 -- 172.21.15.26:0/2467010632 shutdown_connections 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 --2- 172.21.15.26:0/2467010632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf40109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 --2- 172.21.15.26:0/2467010632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40101160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 --2- 172.21.15.26:0/2467010632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 0x7faf40102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 -- 172.21.15.26:0/2467010632 >> 172.21.15.26:0/2467010632 conn(0x7faf400fc2d0 msgr2=0x7faf400fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 -- 172.21.15.26:0/2467010632 shutdown_connections 2023-10-06T21:10:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.351+0000 7faf4739d700 1 -- 172.21.15.26:0/2467010632 wait complete. 2023-10-06T21:10:20.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.352+0000 7faf4739d700 1 Processor -- start 2023-10-06T21:10:20.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.352+0000 7faf4739d700 1 -- start start 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.352+0000 7faf4739d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.352+0000 7faf4739d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 0x7faf401991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4739d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf401a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4739d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a07a0 con 0x7faf401028d0 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4739d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a0910 con 0x7faf40101f30 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4739d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401a0ad0 con 0x7faf40100d40 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41198/0 (socket says 172.21.15.26:41198) 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 -- 172.21.15.26:0/4165521082 learned_addr learned my addr 172.21.15.26:0/4165521082 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf45b9a700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 0x7faf401991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf46b9c700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf401a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 -- 172.21.15.26:0/4165521082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 msgr2=0x7faf401991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 0x7faf401991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 -- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 msgr2=0x7faf401a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf401a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 -- 172.21.15.26:0/4165521082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0095f0 con 0x7faf40100d40 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf46b9c700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf401a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.353+0000 7faf4639b700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf3c004710 tx=0x7faf3c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.354+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c01d070 con 0x7faf40100d40 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.354+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf3c021410 con 0x7faf40100d40 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.354+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401a0d50 con 0x7faf40100d40 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.354+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c017650 con 0x7faf40100d40 2023-10-06T21:10:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.354+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf401a11d0 con 0x7faf40100d40 2023-10-06T21:10:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.355+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40193390 con 0x7faf40100d40 2023-10-06T21:10:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.357+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7faf3c021580 con 0x7faf40100d40 2023-10-06T21:10:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.358+0000 7faf377fe700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 0x7faf2c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.358+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faf3c0cddf0 con 0x7faf40100d40 2023-10-06T21:10:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.358+0000 7faf45b9a700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 0x7faf2c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.359+0000 7faf45b9a700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 0x7faf2c0a37c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf30009c60 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.362+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faf3c097680 con 0x7faf40100d40 2023-10-06T21:10:20.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:20 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:20 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:20 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:10:20.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:20 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:10:20.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:20 smithi078 ceph-mon[134896]: mgrmap e22: x(active, since 1.01953s), standbys: z, y 2023-10-06T21:10:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:20 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:20 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:20 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-06T21:10:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:20 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-06T21:10:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:20 smithi167 ceph-mon[134494]: mgrmap e22: x(active, since 1.01953s), standbys: z, y 2023-10-06T21:10:20.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.637+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7faf4002d290 con 0x7faf2c0a1300 2023-10-06T21:10:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.698+0000 7faf377fe700 1 -- 172.21.15.26:0/4165521082 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7faf4002d290 con 0x7faf2c0a1300 2023-10-06T21:10:20.701 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path / 2023-10-06T21:10:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 msgr2=0x7faf2c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 0x7faf2c0a37c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf30009c60 tx=0x7faf30009260 comp rx=0 tx=0).stop 2023-10-06T21:10:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 msgr2=0x7faf40198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf3c004710 tx=0x7faf3c009580 comp rx=0 tx=0).stop 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 shutdown_connections 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf2c0a1300 0x7faf2c0a37c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf401028d0 0x7faf401a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40100d40 0x7faf40198c60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 --2- 172.21.15.26:0/4165521082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf40101f30 0x7faf401991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.700+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 >> 172.21.15.26:0/4165521082 conn(0x7faf400fc2d0 msgr2=0x7faf40107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.701+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 shutdown_connections 2023-10-06T21:10:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.701+0000 7faf4739d700 1 -- 172.21.15.26:0/4165521082 wait complete. 2023-10-06T21:10:20.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T21:10:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.926+0000 7f24f79ff700 1 -- 172.21.15.26:0/3157990348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0101a30 msgr2=0x7f24f0103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.926+0000 7f24f79ff700 1 --2- 172.21.15.26:0/3157990348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0101a30 0x7f24f0103e20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009990 tx=0x7f24ec009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 -- 172.21.15.26:0/3157990348 shutdown_connections 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 --2- 172.21.15.26:0/3157990348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0106c90 0x7f24f0109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 --2- 172.21.15.26:0/3157990348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0101a30 0x7f24f0103e20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 --2- 172.21.15.26:0/3157990348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0104360 0x7f24f0106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 -- 172.21.15.26:0/3157990348 >> 172.21.15.26:0/3157990348 conn(0x7f24f00fb340 msgr2=0x7f24f00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 -- 172.21.15.26:0/3157990348 shutdown_connections 2023-10-06T21:10:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.927+0000 7f24f79ff700 1 -- 172.21.15.26:0/3157990348 wait complete. 2023-10-06T21:10:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 Processor -- start 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 -- start start 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0101a30 0x7f24f0196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0106c90 0x7f24f0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f01986d0 con 0x7f24f0106c90 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0198840 con 0x7f24f0104360 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.928+0000 7f24f79ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0198a00 con 0x7f24f0101a30 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:50950/0 (socket says 172.21.15.26:50950) 2023-10-06T21:10:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 -- 172.21.15.26:0/166830620 learned_addr learned my addr 172.21.15.26:0/166830620 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f579b700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0101a30 0x7f24f0196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f5f9c700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0106c90 0x7f24f0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 -- 172.21.15.26:0/166830620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0101a30 msgr2=0x7f24f0196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0101a30 0x7f24f0196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 -- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0106c90 msgr2=0x7f24f0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0106c90 0x7f24f0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 -- 172.21.15.26:0/166830620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0095f0 con 0x7f24f0104360 2023-10-06T21:10:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24f4f9a700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9a0 tx=0x7f24e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.929+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0009790 con 0x7f24f0104360 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.930+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f0198ce0 con 0x7f24f0104360 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.930+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24e0010430 con 0x7f24f0104360 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.930+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0018680 con 0x7f24f0104360 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.930+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f019f450 con 0x7f24f0104360 2023-10-06T21:10:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.931+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f00fcec0 con 0x7f24f0104360 2023-10-06T21:10:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.932+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) v1 ==== 135815+0+0 (secure 0 0 0) 0x7f24e000ca10 con 0x7f24f0104360 2023-10-06T21:10:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.934+0000 7f24e67fc700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 0x7f24dc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.934+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f24e0014070 con 0x7f24f0104360 2023-10-06T21:10:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.934+0000 7f24f579b700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 0x7f24dc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.935+0000 7f24f579b700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 0x7f24dc0a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b2e0 tx=0x7f24ec005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:20.937+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f24e0052100 con 0x7f24f0104360 2023-10-06T21:10:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.083+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 23) v1 ==== 136123+0+0 (secure 0 0 0) 0x7f24e00105a0 con 0x7f24f0104360 2023-10-06T21:10:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.200+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f24f00612f0 con 0x7f24dc0a1280 2023-10-06T21:10:21.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.208+0000 7f24e67fc700 1 -- 172.21.15.26:0/166830620 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f24f00612f0 con 0x7f24dc0a1280 2023-10-06T21:10:21.211 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T21:10:20", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 msgr2=0x7f24dc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 0x7f24dc0a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b2e0 tx=0x7f24ec005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 msgr2=0x7f24f0197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9a0 tx=0x7f24e000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 shutdown_connections 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24dc0a1280 0x7f24dc0a3740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24f0106c90 0x7f24f0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24f0101a30 0x7f24f0196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 --2- 172.21.15.26:0/166830620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24f0104360 0x7f24f0197290 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 >> 172.21.15.26:0/166830620 conn(0x7f24f00fb340 msgr2=0x7f24f01026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 shutdown_connections 2023-10-06T21:10:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.211+0000 7f24f79ff700 1 -- 172.21.15.26:0/166830620 wait complete. 2023-10-06T21:10:21.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T21:10:21.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: [06/Oct/2023:21:10:19] ENGINE Bus STARTING 2023-10-06T21:10:21.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: from='client.25382 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:21.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:21.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: [06/Oct/2023:21:10:20] ENGINE Serving on https://172.21.15.26:7150 2023-10-06T21:10:21.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: [06/Oct/2023:21:10:20] ENGINE Serving on http://172.21.15.26:8765 2023-10-06T21:10:21.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:21 smithi026 ceph-mon[98490]: [06/Oct/2023:21:10:20] ENGINE Bus STARTED 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: [06/Oct/2023:21:10:19] ENGINE Bus STARTING 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: from='client.25382 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: [06/Oct/2023:21:10:20] ENGINE Serving on https://172.21.15.26:7150 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: [06/Oct/2023:21:10:20] ENGINE Serving on http://172.21.15.26:8765 2023-10-06T21:10:21.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:21 smithi078 ceph-mon[134896]: [06/Oct/2023:21:10:20] ENGINE Bus STARTED 2023-10-06T21:10:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 -- 172.21.15.26:0/3703659236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c100eb0 msgr2=0x7fc44c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3703659236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c100eb0 0x7fc44c1012d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009990 tx=0x7fc43c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 -- 172.21.15.26:0/3703659236 shutdown_connections 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3703659236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102a50 0x7fc44c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3703659236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c100eb0 0x7fc44c1012d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3703659236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c102090 0x7fc44c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 -- 172.21.15.26:0/3703659236 >> 172.21.15.26:0/3703659236 conn(0x7fc44c0fc430 msgr2=0x7fc44c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.445+0000 7fc4517ff700 1 -- 172.21.15.26:0/3703659236 shutdown_connections 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.446+0000 7fc4517ff700 1 -- 172.21.15.26:0/3703659236 wait complete. 2023-10-06T21:10:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.447+0000 7fc4517ff700 1 Processor -- start 2023-10-06T21:10:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.447+0000 7fc4517ff700 1 -- start start 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102090 0x7fc44c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c102a50 0x7fc44c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c19a950 con 0x7fc44c100eb0 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c19aac0 con 0x7fc44c102090 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc4517ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c19ac80 con 0x7fc44c102a50 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56860/0 (socket says 172.21.15.26:56860) 2023-10-06T21:10:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 -- 172.21.15.26:0/3880889415 learned_addr learned my addr 172.21.15.26:0/3880889415 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc443fff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102090 0x7fc44c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44b7fe700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c102a50 0x7fc44c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 -- 172.21.15.26:0/3880889415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c102a50 msgr2=0x7fc44c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c102a50 0x7fc44c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 -- 172.21.15.26:0/3880889415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102090 msgr2=0x7fc44c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102090 0x7fc44c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.448+0000 7fc44affd700 1 -- 172.21.15.26:0/3880889415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c0095f0 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc44affd700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004710 tx=0x7fc43c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c01d070 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c19af00 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc43c021410 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c017650 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.449+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1a1720 con 0x7fc44c100eb0 2023-10-06T21:10:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.450+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c04ec90 con 0x7fc44c100eb0 2023-10-06T21:10:21.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.453+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 136123+0+0 (secure 0 0 0) 0x7fc43c021580 con 0x7fc44c100eb0 2023-10-06T21:10:21.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.454+0000 7fc448ff9700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 0x7fc4380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.454+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc43c0ceae0 con 0x7fc44c100eb0 2023-10-06T21:10:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.454+0000 7fc443fff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 0x7fc4380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:21.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.455+0000 7fc443fff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 0x7fc4380a3ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc44c100bd0 tx=0x7fc434009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:21.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.456+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc43c097fb0 con 0x7fc44c100eb0 2023-10-06T21:10:21.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: [06/Oct/2023:21:10:19] ENGINE Bus STARTING 2023-10-06T21:10:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: from='client.25382 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: [06/Oct/2023:21:10:20] ENGINE Serving on https://172.21.15.26:7150 2023-10-06T21:10:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: [06/Oct/2023:21:10:20] ENGINE Serving on http://172.21.15.26:8765 2023-10-06T21:10:21.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:21 smithi167 ceph-mon[134494]: [06/Oct/2023:21:10:20] ENGINE Bus STARTED 2023-10-06T21:10:21.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.710+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc44c108a80 con 0x7fc4380a15f0 2023-10-06T21:10:21.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.717+0000 7fc448ff9700 1 -- 172.21.15.26:0/3880889415 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fc44c108a80 con 0x7fc4380a15f0 2023-10-06T21:10:21.720 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2023-10-06T21:10:21.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 msgr2=0x7fc4380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 0x7fc4380a3ab0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc44c100bd0 tx=0x7fc434009260 comp rx=0 tx=0).stop 2023-10-06T21:10:21.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 msgr2=0x7fc44c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:21.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004710 tx=0x7fc43c009580 comp rx=0 tx=0).stop 2023-10-06T21:10:21.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 shutdown_connections 2023-10-06T21:10:21.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc44c102a50 0x7fc44c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc4380a15f0 0x7fc4380a3ab0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc44c100eb0 0x7fc44c198f30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 --2- 172.21.15.26:0/3880889415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc44c102090 0x7fc44c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 >> 172.21.15.26:0/3880889415 conn(0x7fc44c0fc430 msgr2=0x7fc44c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 shutdown_connections 2023-10-06T21:10:21.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:21.720+0000 7fc4517ff700 1 -- 172.21.15.26:0/3880889415 wait complete. 2023-10-06T21:10:21.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-10-06T21:10:21.820 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-10-06T21:10:21.831 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-06T21:10:21.852 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:10:21.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-06T21:10:21.857 INFO:teuthology.run_tasks:Running task exec... 2023-10-06T21:10:21.867 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T21:10:21.867 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:10:21.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-10-06T21:10:21.977 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:22.028 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem Size Used Avail Use% Mounted on 2023-10-06T21:10:22.028 INFO:teuthology.orchestra.run.smithi026.stdout:0@4dcc50a6-648b-11ee-8db5-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/68c2ada3-e9f1-4f5f-952a-1c21257dff12 340G 0 340G 0% /tmp/cephfs 2023-10-06T21:10:22.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-10-06T21:10:22.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-10-06T21:10:22.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:22 smithi026 ceph-mon[98490]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:22.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:22 smithi026 ceph-mon[98490]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:22.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:22 smithi026 ceph-mon[98490]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:10:22.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:22 smithi026 ceph-mon[98490]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:22.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-10-06T21:10:22.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:22 smithi078 ceph-mon[134896]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:22.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:22 smithi078 ceph-mon[134896]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:22.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:22 smithi078 ceph-mon[134896]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:10:22.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:22 smithi078 ceph-mon[134896]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:22 smithi167 ceph-mon[134494]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:22 smithi167 ceph-mon[134494]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:22 smithi167 ceph-mon[134494]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-10-06T21:10:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:22 smithi167 ceph-mon[134494]: from='client.25430 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:23.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:23 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:23.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:23 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:23.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:23.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:23 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:24.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:24 smithi167 ceph-mon[134494]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-10-06T21:10:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:10:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:10:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:10:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:10:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:24 smithi026 ceph-mon[98490]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:24 smithi078 ceph-mon[134896]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:10:25.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Updating smithi026:/etc/ceph/ceph.conf 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Updating smithi078:/etc/ceph/ceph.conf 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Updating smithi167:/etc/ceph/ceph.conf 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:25.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:25 smithi026 ceph-mon[98490]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:25.833 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-10-06T21:10:25.938 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-10-06T21:10:25.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:10:26.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Updating smithi026:/etc/ceph/ceph.conf 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Updating smithi078:/etc/ceph/ceph.conf 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Updating smithi167:/etc/ceph/ceph.conf 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:26.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:25 smithi167 ceph-mon[134494]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:26.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-06T21:10:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:26.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Updating smithi026:/etc/ceph/ceph.conf 2023-10-06T21:10:26.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Updating smithi078:/etc/ceph/ceph.conf 2023-10-06T21:10:26.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Updating smithi167:/etc/ceph/ceph.conf 2023-10-06T21:10:26.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:26.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:26.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:25 smithi078 ceph-mon[134896]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-06T21:10:26.158 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/: 2023-10-06T21:10:26.158 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:26.158 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:26.158 INFO:teuthology.orchestra.run.smithi026.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:10:26.158 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 data 2023-10-06T21:10:26.159 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:26.159 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data: 2023-10-06T21:10:26.159 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:26.159 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:26.159 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:10:26.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-10-06T21:10:26.295 INFO:teuthology.orchestra.run.smithi026.stdout:[Service] 2023-10-06T21:10:26.295 INFO:teuthology.orchestra.run.smithi026.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-10-06T21:10:26.295 INFO:teuthology.orchestra.run.smithi026.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-10-06T21:10:26.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-10-06T21:10:26.505 INFO:teuthology.orchestra.run.smithi026.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-10-06T21:10:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:26 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:26 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.327 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.328 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-10-06T21:10:27.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:27.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:26 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:28.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:27 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:28.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-10-06T21:10:28.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:28.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:28.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:28.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:27 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: Updating smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:10:28.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:27 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:10:29.490 INFO:teuthology.orchestra.run.smithi026.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-10-06T21:10:29.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/: 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 data 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.611 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data: 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:total 30 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 . 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 base 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 6 21:10 global 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 log 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_commit_ts 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_dynshmem 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 6 21:10 pg_hba.conf 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 6 21:10 pg_ident.conf 2023-10-06T21:10:29.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 pg_logical 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 pg_multixact 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 pg_notify 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_replslot 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_serial 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_snapshots 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_stat 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_stat_tmp 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 pg_subtrans 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_tblspc 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_twophase 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 pg_wal 2023-10-06T21:10:29.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 pg_xact 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 6 21:10 postgresql.auto.conf 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 6 21:10 postgresql.conf 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base: 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 . 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 1 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 12346 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 12347 2023-10-06T21:10:29.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:total 7201 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 . 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 .. 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:10:29.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:10:29.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:10:29.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:10 1259 2023-10-06T21:10:29.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:10:29.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:10:29.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:10:29.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 6 21:10 2608 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:10:29.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:10:29.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:10:29.624 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:10 2619 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2619_fsm 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:10:29.625 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:10 2658 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:10:29.626 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:10:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:10:29.628 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2696 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:10:29.629 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:10:29.630 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:10:29.631 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:10:29.632 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:10:29.633 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:10:29.634 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:10:29.635 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:10:29.636 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:10:29.637 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:10:29.638 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:10:29.638 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:10:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:10:29.641 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:10:29.642 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:total 7201 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 . 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 .. 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:10:29.643 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:10:29.644 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:10:29.645 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:10:29.646 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:10 1259 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:10:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:10:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:10:29.649 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 6 21:10 2608 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:10:29.650 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:10:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:10 2619 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2619_fsm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:10:29.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:10 2658 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:10:29.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:10:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:10:29.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2696 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:10:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:10:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:10:29.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:10:29.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:10:29.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:10:29.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:10:29.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:10:29.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:10:29.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:10:29.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:10:29.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:total 7201 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 . 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 .. 2023-10-06T21:10:29.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:10:29.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:10:29.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:10:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:10 1259 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:10:29.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:10:29.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:10:29.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 6 21:10 2608 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:10:29.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:10:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:10:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:10 2619 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2619_fsm 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:10:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:10 2658 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:10:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:10:29.679 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2696 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:10:29.680 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:10:29.681 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:10:29.682 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:10:29.683 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:10:29.684 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:10:29.685 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:10:29.686 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:10:29.687 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:10:29.688 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:10:29.689 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:10:29.690 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:10:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/global: 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:total 553 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 6 21:10 . 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1136 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1136_fsm 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1136_vm 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1137 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1213 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1213_fsm 2023-10-06T21:10:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1213_vm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1214 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1214_fsm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1214_vm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1232 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1233 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1260 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1260_fsm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1260_vm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1261 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1261_fsm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1261_vm 2023-10-06T21:10:29.693 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1262 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1262_fsm 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1262_vm 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2396 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2396_fsm 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2396_vm 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2397 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2671 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2672 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2676 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2677 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2694 2023-10-06T21:10:29.694 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2695 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2697 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2698 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2846 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2846_vm 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2847 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2964 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2964_vm 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2965 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2966 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2966_vm 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2967 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3592 2023-10-06T21:10:29.695 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3592_vm 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3593 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 4060 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 4060_vm 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 4061 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6000 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6000_vm 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6001 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6002 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6100 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6100_vm 2023-10-06T21:10:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6114 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6115 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 pg_control 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/log: 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.697 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.698 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.698 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.698 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-06T21:10:29.698 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.698 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.699 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.699 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.699 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-06T21:10:29.699 INFO:teuthology.orchestra.run.smithi026.stdout:total 1 2023-10-06T21:10:29.699 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 . 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 mappings 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 6 21:10 replorigin_checkpoint 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 snapshots 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 .. 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-06T21:10:29.700 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 .. 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 . 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 members 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 offsets 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:10:29.701 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 .. 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 .. 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:10:29.702 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.703 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.704 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:total 16384 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 . 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:10 000000010000000000000001 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 archive_status 2023-10-06T21:10:29.706 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 .. 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 6 21:10 .. 2023-10-06T21:10:29.707 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:10:29.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-10-06T21:10:29.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-10-06T21:10:30.046 INFO:teuthology.orchestra.run.smithi026.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-10-06T21:10:30.052 INFO:teuthology.orchestra.run.smithi026.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-10-06T21:10:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-10-06T21:10:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-10-06T21:10:30.086 INFO:teuthology.orchestra.run.smithi026.stderr:creating tables... 2023-10-06T21:10:30.209 INFO:teuthology.orchestra.run.smithi026.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.12 s, remaining 3.78 s) 2023-10-06T21:10:30.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:30 smithi026 ceph-mon[98490]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-10-06T21:10:30.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:30 smithi078 ceph-mon[134896]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-10-06T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.41 s, remaining 6.16 s) 2023-10-06T21:10:30.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:30 smithi167 ceph-mon[134494]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-10-06T21:10:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.72 s, remaining 6.91 s) 2023-10-06T21:10:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.91 s, remaining 6.34 s) 2023-10-06T21:10:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.19 s, remaining 6.41 s) 2023-10-06T21:10:31.648 INFO:teuthology.orchestra.run.smithi026.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.56 s, remaining 6.77 s) 2023-10-06T21:10:31.773 INFO:teuthology.orchestra.run.smithi026.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.69 s, remaining 6.02 s) 2023-10-06T21:10:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.03 s, remaining 6.09 s) 2023-10-06T21:10:32.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:32 smithi078 ceph-mon[134896]: pgmap v9: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 4.7 MiB/s wr, 157 op/s 2023-10-06T21:10:32.431 INFO:teuthology.orchestra.run.smithi026.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.34 s, remaining 5.99 s) 2023-10-06T21:10:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:32 smithi167 ceph-mon[134494]: pgmap v9: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 4.7 MiB/s wr, 157 op/s 2023-10-06T21:10:32.541 INFO:teuthology.orchestra.run.smithi026.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.45 s, remaining 5.40 s) 2023-10-06T21:10:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:32 smithi026 ceph-mon[98490]: pgmap v9: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 4.7 MiB/s wr, 157 op/s 2023-10-06T21:10:32.808 INFO:teuthology.orchestra.run.smithi026.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.72 s, remaining 5.19 s) 2023-10-06T21:10:32.961 INFO:teuthology.orchestra.run.smithi026.stderr:1200000 of 3200000 tuples (37%) done (elapsed 2.87 s, remaining 4.79 s) 2023-10-06T21:10:33.301 INFO:teuthology.orchestra.run.smithi026.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.21 s, remaining 4.69 s) 2023-10-06T21:10:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.53 s, remaining 4.54 s) 2023-10-06T21:10:33.811 INFO:teuthology.orchestra.run.smithi026.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.72 s, remaining 4.22 s) 2023-10-06T21:10:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.00 s, remaining 4.00 s) 2023-10-06T21:10:34.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:33 smithi078 ceph-mon[134896]: pgmap v10: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:33 smithi167 ceph-mon[134494]: pgmap v10: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:33 smithi026 ceph-mon[98490]: pgmap v10: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.34 s, remaining 3.83 s) 2023-10-06T21:10:34.529 INFO:teuthology.orchestra.run.smithi026.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.44 s, remaining 3.45 s) 2023-10-06T21:10:34.944 INFO:teuthology.orchestra.run.smithi026.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.86 s, remaining 3.32 s) 2023-10-06T21:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 -- 172.21.15.26:0/2413042502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 msgr2=0x7faf20068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 --2- 172.21.15.26:0/2413042502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20068750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf14009990 tx=0x7faf14009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 -- 172.21.15.26:0/2413042502 shutdown_connections 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 --2- 172.21.15.26:0/2413042502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20068c90 0x7faf2010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 --2- 172.21.15.26:0/2413042502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20106f60 0x7faf20107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 --2- 172.21.15.26:0/2413042502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20068750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 -- 172.21.15.26:0/2413042502 >> 172.21.15.26:0/2413042502 conn(0x7faf20076b70 msgr2=0x7faf20076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 -- 172.21.15.26:0/2413042502 shutdown_connections 2023-10-06T21:10:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.123+0000 7faf265ff700 1 -- 172.21.15.26:0/2413042502 wait complete. 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.124+0000 7faf265ff700 1 Processor -- start 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.124+0000 7faf265ff700 1 -- start start 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20106f60 0x7faf2019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019e2e0 con 0x7faf20106f60 2023-10-06T21:10:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019e450 con 0x7faf200682d0 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf265ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2019e610 con 0x7faf20068c90 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf1f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf1f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49422/0 (socket says 172.21.15.26:49422) 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf1f7fe700 1 -- 172.21.15.26:0/4138637251 learned_addr learned my addr 172.21.15.26:0/4138637251 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.125+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20106f60 0x7faf2019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 -- 172.21.15.26:0/4138637251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 msgr2=0x7faf20199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 -- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20106f60 msgr2=0x7faf2019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20106f60 0x7faf2019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 -- 172.21.15.26:0/4138637251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140095f0 con 0x7faf20068c90 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf14005910 tx=0x7faf14005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1d7fa700 1 -- 172.21.15.26:0/4138637251 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1401d070 con 0x7faf20068c90 2023-10-06T21:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2019e890 con 0x7faf20068c90 2023-10-06T21:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1d7fa700 1 -- 172.21.15.26:0/4138637251 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf14022470 con 0x7faf20068c90 2023-10-06T21:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.126+0000 7faf1d7fa700 1 -- 172.21.15.26:0/4138637251 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14017650 con 0x7faf20068c90 2023-10-06T21:10:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.127+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201a6130 con 0x7faf20068c90 2023-10-06T21:10:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.130+0000 7faf1d7fa700 1 -- 172.21.15.26:0/4138637251 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7faf140225e0 con 0x7faf20068c90 2023-10-06T21:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.132+0000 7faf39eeab80 1 Processor -- start 2023-10-06T21:10:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.132+0000 7faf39eeab80 1 -- start start 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.132+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.132+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf39eeab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a3905400 con 0x5568a399c520 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf39eeab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a38f6bc0 con 0x5568a399bc00 2023-10-06T21:10:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf39eeab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a38f44e0 con 0x5568a39a73d0 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:32772/0 (socket says 172.21.15.26:32772) 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 -- 172.21.15.26:0/1041184820 learned_addr learned my addr 172.21.15.26:0/1041184820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf1ffff700 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 -- 172.21.15.26:0/1041184820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 msgr2=0x5568a39a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 -- 172.21.15.26:0/1041184820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 msgr2=0x5568a39a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.133+0000 7faf24b9c700 1 -- 172.21.15.26:0/1041184820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a371ad80 con 0x5568a399bc00 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf1d7fa700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 0x7faf080a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf24b9c700 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf18011670 tx=0x7faf18009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bb8ce8615323a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf1d7fa700 1 -- 172.21.15.26:0/4138637251 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faf140ce610 con 0x7faf20068c90 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf067fc700 1 -- 172.21.15.26:0/1041184820 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e040 con 0x5568a399bc00 2023-10-06T21:10:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 0x7faf080a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf067fc700 1 -- 172.21.15.26:0/1041184820 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf1800f040 con 0x5568a399bc00 2023-10-06T21:10:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.134+0000 7faf067fc700 1 -- 172.21.15.26:0/1041184820 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180047b0 con 0x5568a399bc00 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 -- 172.21.15.26:0/1041184820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 msgr2=0x5568a399bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 0x7faf080a3b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf10012640 tx=0x7faf10009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf18011670 tx=0x7faf18009920 comp rx=0 tx=0).stop 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 -- 172.21.15.26:0/1041184820 shutdown_connections 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a399bfe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1041184820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 -- 172.21.15.26:0/1041184820 >> 172.21.15.26:0/1041184820 conn(0x5568a3911200 msgr2=0x5568a3998080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 -- 172.21.15.26:0/1041184820 shutdown_connections 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.135+0000 7faf39eeab80 1 -- 172.21.15.26:0/1041184820 wait complete. 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.136+0000 7faf39eeab80 1 -- start start 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.136+0000 7faf39eeab80 1 Processor -- start 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399bc00 0x5568a39a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a38efdd0 con 0x5568a399bc00 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a388c6b0 con 0x5568a399c520 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a388c350 con 0x5568a39a73d0 2023-10-06T21:10:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf39eeab80 10 client.0 _ll_register_callbacks cb 0x5568a36e97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399bc00 0x5568a39a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:32784/0 (socket says 172.21.15.26:32784) 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 -- 172.21.15.26:0/1722718193 learned_addr learned my addr 172.21.15.26:0/1722718193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 -- 172.21.15.26:0/1722718193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 msgr2=0x5568a39a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 -- 172.21.15.26:0/1722718193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399bc00 msgr2=0x5568a39a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399bc00 0x5568a39a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.137+0000 7faf1f7fe700 1 -- 172.21.15.26:0/1722718193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf18011350 con 0x5568a399c520 2023-10-06T21:10:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf14000c00 tx=0x7faf14097320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74937dfbac1bdd05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faeff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14097920 con 0x5568a399c520 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faf1409bce0 con 0x5568a399c520 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568a398fd50 con 0x5568a399c520 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1409b3b0 con 0x5568a399c520 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faf39eeab80 10 client.25454 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.138+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5568a3703c00 con 0x5568a399c520 2023-10-06T21:10:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568a398fd50 con 0x5568a399c520 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf140a3450 con 0x5568a399c520 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 10 client.25454 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 20 client.25454 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a39b6a00 0x5568a39b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 4 client.25454 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568a38e8db0 con 0x5568a39b6a00 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.139+0000 7faf1ffff700 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a39b6a00 0x5568a39b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.140+0000 7faf1ffff700 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a39b6a00 0x5568a39b8e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348598438b89472f server_cookie=3edfc74d2fad08b2 in_seq=0 out_seq=0 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.140+0000 7faeff7fe700 10 client.25454 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.140+0000 7faeff7fe700 1 -- 172.21.15.26:0/1722718193 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5568a38e8db0 con 0x5568a39b6a00 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.140+0000 7faeff7fe700 10 client.25454 handle_command_reply: tid=0 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.140+0000 7faf39eeab80 1 client.25454 shutdown 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 msgr2=0x5568a39a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf14000c00 tx=0x7faf14097320 comp rx=0 tx=0).stop 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 shutdown_connections 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399bc00 0x5568a39a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a39b6a00 0x5568a39b8e20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 --2- 172.21.15.26:0/1722718193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a4030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 >> 172.21.15.26:0/1722718193 conn(0x5568a3911200 msgr2=0x5568a3997c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faeff7fe700 0 client.25454 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 shutdown_connections 2023-10-06T21:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 1 -- 172.21.15.26:0/1722718193 wait complete. 2023-10-06T21:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.141+0000 7faf39eeab80 20 client.25454 trim_cache size 0 max 16384 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 Processor -- start 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 -- start start 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 0x5568a39a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a37708c0 con 0x5568a39a73d0 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a3998e20 con 0x5568a399c520 2023-10-06T21:10:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.162+0000 7faf39eeab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a39ae4d0 con 0x5568a399bc00 2023-10-06T21:10:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49454/0 (socket says 172.21.15.26:49454) 2023-10-06T21:10:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 -- 172.21.15.26:0/4163821764 learned_addr learned my addr 172.21.15.26:0/4163821764 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf24b9c700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 0x5568a39a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 -- 172.21.15.26:0/4163821764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 msgr2=0x5568a39a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 -- 172.21.15.26:0/4163821764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 msgr2=0x5568a39a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 0x5568a39a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf1ffff700 1 -- 172.21.15.26:0/4163821764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a37411f0 con 0x5568a399bc00 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.163+0000 7faf24b9c700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 0x5568a39a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.164+0000 7faf1ffff700 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf10013010 tx=0x7faf10019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72453d66769b824 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.164+0000 7faf06ffd700 1 -- 172.21.15.26:0/4163821764 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10023030 con 0x5568a399bc00 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.164+0000 7faf06ffd700 1 -- 172.21.15.26:0/4163821764 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf10024040 con 0x5568a399bc00 2023-10-06T21:10:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.164+0000 7faf06ffd700 1 -- 172.21.15.26:0/4163821764 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10017070 con 0x5568a399bc00 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- 172.21.15.26:0/4163821764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 msgr2=0x5568a39adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faf10013010 tx=0x7faf10019000 comp rx=0 tx=0).stop 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- 172.21.15.26:0/4163821764 shutdown_connections 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a39a73d0 0x5568a39a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a399bc00 0x5568a39adf10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 --2- 172.21.15.26:0/4163821764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399c520 0x5568a39a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- 172.21.15.26:0/4163821764 >> 172.21.15.26:0/4163821764 conn(0x5568a3911200 msgr2=0x5568a399f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- 172.21.15.26:0/4163821764 shutdown_connections 2023-10-06T21:10:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- 172.21.15.26:0/4163821764 wait complete. 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 -- start start 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.165+0000 7faf39eeab80 1 Processor -- start 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a39adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a39aa970 con 0x5568a399c520 2023-10-06T21:10:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a39aaae0 con 0x5568a399bc00 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a39aaca0 con 0x5568a39a73d0 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf39eeab80 10 client.0 _ll_register_callbacks cb 0x7faf140d1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf24b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49468/0 (socket says 172.21.15.26:49468) 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf1f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf24b9c700 1 -- 172.21.15.26:0/3280260487 learned_addr learned my addr 172.21.15.26:0/3280260487 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.166+0000 7faf1ffff700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a39adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 -- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 msgr2=0x5568a39adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a39adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 -- 172.21.15.26:0/3280260487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 msgr2=0x5568a39a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 -- 172.21.15.26:0/3280260487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1002a980 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faf24b9c700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf18011670 tx=0x7faf18005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b1a8aaacadc19b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faeff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.167+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800e040 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7faf180093a0 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf1800f040 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf18003c10 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faf39eeab80 10 client.25432 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5568a3719250 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf1800f040 con 0x5568a39a73d0 2023-10-06T21:10:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.168+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf1802e070 con 0x5568a39a73d0 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 10 client.25432 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 20 client.25432 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a3a143f0 0x5568a3a16810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 4 client.25432 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5568a38e8ad0 con 0x5568a3a143f0 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf1ffff700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a3a143f0 0x5568a3a16810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faeff7fe700 10 client.25432 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.169+0000 7faf1ffff700 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a3a143f0 0x5568a3a16810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581796ed890ecbd server_cookie=fe383dd443e973d4 in_seq=0 out_seq=0 2023-10-06T21:10:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.345+0000 7faeff7fe700 1 -- 172.21.15.26:0/3280260487 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5568a38e8ad0 con 0x5568a3a143f0 2023-10-06T21:10:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faeff7fe700 10 client.25432 handle_command_reply: tid=0 2023-10-06T21:10:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 client.25432 shutdown 2023-10-06T21:10:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 msgr2=0x5568a39aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf18011670 tx=0x7faf18005b30 comp rx=0 tx=0).stop 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 shutdown_connections 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568a39a73d0 0x5568a39aa430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5568a3a143f0 0x5568a3a16810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568a399bc00 0x5568a39adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 --2- 172.21.15.26:0/3280260487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568a399c520 0x5568a39a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 >> 172.21.15.26:0/3280260487 conn(0x5568a3911200 msgr2=0x5568a3997250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faeff7fe700 0 client.25432 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 shutdown_connections 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 1 -- 172.21.15.26:0/3280260487 wait complete. 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.346+0000 7faf39eeab80 20 client.25432 trim_cache size 0 max 16384 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:10:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 msgr2=0x7faf080a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 0x7faf080a3b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf10012640 tx=0x7faf10009260 comp rx=0 tx=0).stop 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 msgr2=0x7faf20199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf14005910 tx=0x7faf14005720 comp rx=0 tx=0).stop 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 shutdown_connections 2023-10-06T21:10:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf080a1680 0x7faf080a3b40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf20106f60 0x7faf2019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf200682d0 0x7faf20199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 --2- 172.21.15.26:0/4138637251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf20068c90 0x7faf20199680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 >> 172.21.15.26:0/4138637251 conn(0x7faf20076b70 msgr2=0x7faf2010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 shutdown_connections 2023-10-06T21:10:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:35.347+0000 7faf265ff700 1 -- 172.21.15.26:0/4138637251 wait complete. 2023-10-06T21:10:35.368 INFO:teuthology.orchestra.run.smithi026.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.28 s, remaining 3.17 s) 2023-10-06T21:10:35.474 INFO:teuthology.orchestra.run.smithi026.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.39 s, remaining 2.82 s) 2023-10-06T21:10:35.875 INFO:teuthology.orchestra.run.smithi026.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.79 s, remaining 2.63 s) 2023-10-06T21:10:35.979 INFO:teuthology.orchestra.run.smithi026.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.89 s, remaining 2.31 s) 2023-10-06T21:10:36.303 INFO:teuthology.orchestra.run.smithi026.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.22 s, remaining 2.07 s) 2023-10-06T21:10:36.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:36 smithi078 ceph-mon[134896]: pgmap v11: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:36.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:36 smithi167 ceph-mon[134494]: pgmap v11: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:36.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:36 smithi026 ceph-mon[98490]: pgmap v11: 97 pgs: 97 active+clean; 43 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 4.3 MiB/s wr, 144 op/s 2023-10-06T21:10:36.586 INFO:teuthology.orchestra.run.smithi026.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.50 s, remaining 1.82 s) 2023-10-06T21:10:36.731 INFO:teuthology.orchestra.run.smithi026.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.64 s, remaining 1.53 s) 2023-10-06T21:10:37.051 INFO:teuthology.orchestra.run.smithi026.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.96 s, remaining 1.29 s) 2023-10-06T21:10:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.33 s, remaining 1.05 s) 2023-10-06T21:10:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.43 s, remaining 0.77 s) 2023-10-06T21:10:37.852 INFO:teuthology.orchestra.run.smithi026.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.76 s, remaining 0.52 s) 2023-10-06T21:10:38.058 INFO:teuthology.orchestra.run.smithi026.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.97 s, remaining 0.26 s) 2023-10-06T21:10:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.28 s, remaining 0.00 s) 2023-10-06T21:10:38.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:38 smithi078 ceph-mon[134896]: pgmap v12: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 36 MiB/s wr, 166 op/s 2023-10-06T21:10:38.441 INFO:teuthology.orchestra.run.smithi026.stderr:vacuum... 2023-10-06T21:10:38.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:38 smithi167 ceph-mon[134494]: pgmap v12: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 36 MiB/s wr, 166 op/s 2023-10-06T21:10:38.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:38 smithi026 ceph-mon[98490]: pgmap v12: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 36 MiB/s wr, 166 op/s 2023-10-06T21:10:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:set primary keys... 2023-10-06T21:10:39.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:10:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.073+0000 7fd8a9fff700 1 -- 172.21.15.26:0/607581911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a410d680 msgr2=0x7fd8a410faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.073+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/607581911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a410d680 0x7fd8a410faf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b0a0 tx=0x7fd89c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 -- 172.21.15.26:0/607581911 shutdown_connections 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/607581911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a410d680 0x7fd8a410faf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/607581911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a40736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/607581911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a4073c00 0x7fd8a410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 -- 172.21.15.26:0/607581911 >> 172.21.15.26:0/607581911 conn(0x7fd8a40fc930 msgr2=0x7fd8a40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 -- 172.21.15.26:0/607581911 shutdown_connections 2023-10-06T21:10:40.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.074+0000 7fd8a9fff700 1 -- 172.21.15.26:0/607581911 wait complete. 2023-10-06T21:10:40.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 Processor -- start 2023-10-06T21:10:40.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 -- start start 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a4073c00 0x7fd8a4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 0x7fd8a419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e230 con 0x7fd8a4073c00 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e3a0 con 0x7fd8a410d680 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.075+0000 7fd8a9fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a419e560 con 0x7fd8a40732e0 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a4073c00 0x7fd8a4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 0x7fd8a419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a4199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49494/0 (socket says 172.21.15.26:49494) 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a37fe700 1 -- 172.21.15.26:0/3861518799 learned_addr learned my addr 172.21.15.26:0/3861518799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 -- 172.21.15.26:0/3861518799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 msgr2=0x7fd8a4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 -- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a4073c00 msgr2=0x7fd8a4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a4073c00 0x7fd8a4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 -- 172.21.15.26:0/3861518799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c009d00 con 0x7fd8a410d680 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 0x7fd8a419dbf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b070 tx=0x7fd89c0078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.076+0000 7fd8a0ff9700 1 -- 172.21.15.26:0/3861518799 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00e040 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.077+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a419e7e0 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.077+0000 7fd8a0ff9700 1 -- 172.21.15.26:0/3861518799 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89c01d470 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.077+0000 7fd8a0ff9700 1 -- 172.21.15.26:0/3861518799 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0052a0 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.077+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a41a6080 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.080+0000 7fd8a0ff9700 1 -- 172.21.15.26:0/3861518799 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7fd89c004a50 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.082+0000 7fd8bd84eb80 1 Processor -- start 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.082+0000 7fd8bd84eb80 1 -- start start 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.082+0000 7fd8a0ff9700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 0x7fd88c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a0ff9700 1 -- 172.21.15.26:0/3861518799 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd89c0ce560 con 0x7fd8a410d680 2023-10-06T21:10:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c083350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 0x557d0c085c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bfe1460 con 0x557d0c0780c0 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bfd2c20 con 0x557d0c0789e0 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bfd0540 con 0x557d0c083890 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 0x7fd88c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46100/0 (socket says 172.21.15.26:46100) 2023-10-06T21:10:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a37fe700 1 -- 172.21.15.26:0/3647476707 learned_addr learned my addr 172.21.15.26:0/3647476707 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c083350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.083+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 0x557d0c085c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 -- 172.21.15.26:0/3647476707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 msgr2=0x557d0c085c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 0x557d0c085c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 -- 172.21.15.26:0/3647476707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 msgr2=0x557d0c083350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c083350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 0x7fd88c0a3b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8a419a860 tx=0x7fd898013840 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 -- 172.21.15.26:0/3647476707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d0bdf6de0 con 0x557d0c0780c0 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 0x557d0c085c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:40.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd894009e50 tx=0x7fd894011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2191ad0d4307f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd889ffb700 1 -- 172.21.15.26:0/3647476707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894004a30 con 0x557d0c0780c0 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd889ffb700 1 -- 172.21.15.26:0/3647476707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8940054e0 con 0x557d0c0780c0 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.084+0000 7fd889ffb700 1 -- 172.21.15.26:0/3647476707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940198f0 con 0x557d0c0780c0 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/3647476707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 msgr2=0x557d0c0784a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd894009e50 tx=0x7fd894011720 comp rx=0 tx=0).stop 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/3647476707 shutdown_connections 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c083890 0x557d0c085c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c0780c0 0x557d0c0784a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/3647476707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c083350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/3647476707 >> 172.21.15.26:0/3647476707 conn(0x557d0bfed260 msgr2=0x557d0c074450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/3647476707 shutdown_connections 2023-10-06T21:10:40.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.085+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/3647476707 wait complete. 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.086+0000 7fd8bd84eb80 1 -- start start 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.086+0000 7fd8bd84eb80 1 Processor -- start 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.086+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.086+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0789e0 0x557d0c0804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c080c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bfcbe30 con 0x557d0c083890 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bf68710 con 0x557d0c0780c0 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0bf683b0 con 0x557d0c0789e0 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8bd84eb80 10 client.0 _ll_register_callbacks cb 0x557d0bdc5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:32850/0 (socket says 172.21.15.26:32850) 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 -- 172.21.15.26:0/219851985 learned_addr learned my addr 172.21.15.26:0/219851985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c080c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0789e0 0x557d0c0804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 -- 172.21.15.26:0/219851985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0789e0 msgr2=0x557d0c0804f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0789e0 0x557d0c0804f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 -- 172.21.15.26:0/219851985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 msgr2=0x557d0c080c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c080c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.087+0000 7fd8a37fe700 1 -- 172.21.15.26:0/219851985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a41a6080 con 0x557d0c0780c0 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd89401b040 tx=0x7fd894004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d1d68983711ccd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd882ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:40.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894005d60 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d0c06bf10 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8940052a0 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd894004040 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd8bd84eb80 10 client.25472 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.088+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd894009a80 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d0c06bf10 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd894021040 con 0x557d0c0780c0 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 10 client.25472 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 20 client.25472 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:40.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c093240 0x557d0c095660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 4 client.25472 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d0bfc4e10 con 0x557d0c093240 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.089+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c093240 0x557d0c095660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.090+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c093240 0x557d0c095660 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9f247049a88845 server_cookie=3e27dfe05f175b73 in_seq=0 out_seq=0 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.090+0000 7fd882ffd700 10 client.25472 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd882ffd700 1 -- 172.21.15.26:0/219851985 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d0bfc4e10 con 0x557d0c093240 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd882ffd700 10 client.25472 handle_command_reply: tid=0 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 client.25472 shutdown 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 msgr2=0x557d0c0832e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd89401b040 tx=0x7fd894004b80 comp rx=0 tx=0).stop 2023-10-06T21:10:40.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 shutdown_connections 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c080c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c093240 0x557d0c095660 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0780c0 0x557d0c0832e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd882ffd700 0 client.25472 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/219851985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0789e0 0x557d0c0804f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 >> 172.21.15.26:0/219851985 conn(0x557d0bfed260 msgr2=0x557d0c074040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 shutdown_connections 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/219851985 wait complete. 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.091+0000 7fd8bd84eb80 20 client.25472 trim_cache size 0 max 16384 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.110+0000 7fd8bd84eb80 1 Processor -- start 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.110+0000 7fd8bd84eb80 1 -- start start 2023-10-06T21:10:40.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.110+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.110+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c073f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.110+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0be4c920 con 0x557d0c083890 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0c081250 con 0x557d0c0789e0 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0c079bb0 con 0x557d0c0780c0 2023-10-06T21:10:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49542/0 (socket says 172.21.15.26:49542) 2023-10-06T21:10:40.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 -- 172.21.15.26:0/821773491 learned_addr learned my addr 172.21.15.26:0/821773491 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 -- 172.21.15.26:0/821773491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 msgr2=0x557d0c073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 -- 172.21.15.26:0/821773491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 msgr2=0x557d0c0744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c0744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.111+0000 7fd8a37fe700 1 -- 172.21.15.26:0/821773491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d0c08f940 con 0x557d0c0780c0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.112+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd894005a80 tx=0x7fd894004bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4619715ec85f6f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.112+0000 7fd88a7fc700 1 -- 172.21.15.26:0/821773491 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894009d50 con 0x557d0c0780c0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.112+0000 7fd88a7fc700 1 -- 172.21.15.26:0/821773491 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd894019760 con 0x557d0c0780c0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.112+0000 7fd88a7fc700 1 -- 172.21.15.26:0/821773491 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894004430 con 0x557d0c0780c0 2023-10-06T21:10:40.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/821773491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 msgr2=0x557d0c082e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd894005a80 tx=0x7fd894004bf0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/821773491 shutdown_connections 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c0744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c082e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/821773491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/821773491 >> 172.21.15.26:0/821773491 conn(0x557d0bfed260 msgr2=0x557d0c07b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/821773491 shutdown_connections 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/821773491 wait complete. 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 -- start start 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.113+0000 7fd8bd84eb80 1 Processor -- start 2023-10-06T21:10:40.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c07c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c07cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0c07d2d0 con 0x557d0c083890 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0c07d440 con 0x557d0c0789e0 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d0c07d600 con 0x557d0c0780c0 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8bd84eb80 10 client.0 _ll_register_callbacks cb 0x7fd898009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c07c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8a37fe700 1 -- 172.21.15.26:0/1466273746 learned_addr learned my addr 172.21.15.26:0/1466273746 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.114+0000 7fd8a3fff700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c07cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 -- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 msgr2=0x557d0c07c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c07c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 -- 172.21.15.26:0/1466273746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 msgr2=0x557d0c07cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c07cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 -- 172.21.15.26:0/1466273746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd894009a80 con 0x557d0c0780c0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8a37fe700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd894004cc0 tx=0x7fd894004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340912322eef9120 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd882ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89403d070 con 0x557d0c0780c0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8940054e0 con 0x557d0c0780c0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd894000e40 con 0x557d0c0780c0 2023-10-06T21:10:40.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.115+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd894009d50 con 0x557d0c0780c0 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 10 client.25450 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557d0c070140 con 0x557d0c0780c0 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89404e080 con 0x557d0c0780c0 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd894011790 con 0x557d0c0780c0 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 10 client.25450 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 20 client.25450 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c0e8190 0x557d0c0ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 4 client.25450 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.116+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d0bfc4b30 con 0x557d0c0e8190 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c0e8190 0x557d0c0ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8a2ffd700 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c0e8190 0x557d0c0ea5b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eef7144b82ed765 server_cookie=4361e0fd906e0d7e in_seq=0 out_seq=0 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd882ffd700 10 client.25450 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd882ffd700 1 -- 172.21.15.26:0/1466273746 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557d0bfc4b30 con 0x557d0c0e8190 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd882ffd700 10 client.25450 handle_command_reply: tid=0 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 client.25450 shutdown 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 msgr2=0x557d0c08a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd894004cc0 tx=0x7fd894004a60 comp rx=0 tx=0).stop 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 shutdown_connections 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d0c083890 0x557d0c07cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d0c0780c0 0x557d0c08a3e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d0c0e8190 0x557d0c0ea5b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 --2- 172.21.15.26:0/1466273746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d0c0789e0 0x557d0c07c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd882ffd700 0 client.25450 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.117+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 >> 172.21.15.26:0/1466273746 conn(0x557d0bfed260 msgr2=0x557d0c073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 shutdown_connections 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8bd84eb80 1 -- 172.21.15.26:0/1466273746 wait complete. 2023-10-06T21:10:40.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8bd84eb80 20 client.25450 trim_cache size 0 max 16384 2023-10-06T21:10:40.207 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:10:40.207 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:10:40.207 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:10:40.207 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:10:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 msgr2=0x7fd88c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 0x7fd88c0a3b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8a419a860 tx=0x7fd898013840 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 msgr2=0x7fd8a419dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.118+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 0x7fd8a419dbf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00b070 tx=0x7fd89c0078e0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 shutdown_connections 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8a410d680 0x7fd8a419dbf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8a40732e0 0x7fd8a4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd88c0a1680 0x7fd88c0a3b40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 --2- 172.21.15.26:0/3861518799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8a4073c00 0x7fd8a4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 >> 172.21.15.26:0/3861518799 conn(0x7fd8a40fc930 msgr2=0x7fd8a410f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 shutdown_connections 2023-10-06T21:10:40.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.119+0000 7fd8a9fff700 1 -- 172.21.15.26:0/3861518799 wait complete. 2023-10-06T21:10:40.210 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:10:40.210 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:10:40.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:40 smithi026 ceph-mon[98490]: pgmap v13: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 36 MiB/s wr, 126 op/s 2023-10-06T21:10:40.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:40 smithi078 ceph-mon[134896]: pgmap v13: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 36 MiB/s wr, 126 op/s 2023-10-06T21:10:40.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 -- 172.21.15.26:0/2182858677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241010d0 msgr2=0x7f87241014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 --2- 172.21.15.26:0/2182858677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241010d0 0x7f87241014b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8720009990 tx=0x7f8720009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 -- 172.21.15.26:0/2182858677 shutdown_connections 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 --2- 172.21.15.26:0/2182858677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87241061f0 0x7f872410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 --2- 172.21.15.26:0/2182858677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241010d0 0x7f87241014b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 --2- 172.21.15.26:0/2182858677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724101a80 0x7f8724105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 -- 172.21.15.26:0/2182858677 >> 172.21.15.26:0/2182858677 conn(0x7f87240fc990 msgr2=0x7f87240fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 -- 172.21.15.26:0/2182858677 shutdown_connections 2023-10-06T21:10:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.419+0000 7f872c1ff700 1 -- 172.21.15.26:0/2182858677 wait complete. 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 Processor -- start 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 -- start start 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87241010d0 0x7f8724199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 0x7f8724199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419e650 con 0x7f87241061f0 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419e7c0 con 0x7f872419dbd0 2023-10-06T21:10:40.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.420+0000 7f872c1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872419e980 con 0x7f87241010d0 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872979a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 0x7f8724199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872979a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 0x7f8724199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46142/0 (socket says 172.21.15.26:46142) 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 -- 172.21.15.26:0/778107332 learned_addr learned my addr 172.21.15.26:0/778107332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f8729f9b700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87241010d0 0x7f8724199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 -- 172.21.15.26:0/778107332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87241010d0 msgr2=0x7f8724199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87241010d0 0x7f8724199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 -- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 msgr2=0x7f8724199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 0x7f8724199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.421+0000 7f872a79c700 1 -- 172.21.15.26:0/778107332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200095f0 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f872a79c700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f871c00ef30 tx=0x7f871c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c018070 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871c00f040 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87241a5e80 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00cd00 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.422+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87241a6310 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.423+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8724067cd0 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.425+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7f871c003a00 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.427+0000 7f871affd700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 0x7f87100a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.427+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f871c0cce90 con 0x7f872419dbd0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.427+0000 7f8729f9b700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 0x7f87100a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:40.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.427+0000 7f8729f9b700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 0x7f87100a3b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f872000b2e0 tx=0x7f8720005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:40.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.429+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f871c0960e0 con 0x7f872419dbd0 2023-10-06T21:10:40.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:40 smithi167 ceph-mon[134494]: pgmap v13: 97 pgs: 97 active+clean; 235 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 36 MiB/s wr, 126 op/s 2023-10-06T21:10:40.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.760+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8724101c90 con 0x7f872419dbd0 2023-10-06T21:10:40.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.760+0000 7f871affd700 1 -- 172.21.15.26:0/778107332 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f871c099700 con 0x7f872419dbd0 2023-10-06T21:10:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:10:40.763 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:08:21.849944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:10:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.764+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 msgr2=0x7f87100a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.764+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 0x7f87100a3b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f872000b2e0 tx=0x7f8720005fb0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.764+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 msgr2=0x7f872419e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.764+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f871c00ef30 tx=0x7f871c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 shutdown_connections 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872419dbd0 0x7f872419e050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87241010d0 0x7f8724199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87100a16c0 0x7f87100a3b80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 --2- 172.21.15.26:0/778107332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87241061f0 0x7f8724199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 >> 172.21.15.26:0/778107332 conn(0x7f87240fc990 msgr2=0x7f87240fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 shutdown_connections 2023-10-06T21:10:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:40.765+0000 7f872c1ff700 1 -- 172.21.15.26:0/778107332 wait complete. 2023-10-06T21:10:40.770 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 23 2023-10-06T21:10:40.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:10:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.027+0000 7f20121ff700 1 -- 172.21.15.26:0/2109381041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073ba0 msgr2=0x7f200c10d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.027+0000 7f20121ff700 1 --2- 172.21.15.26:0/2109381041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073ba0 0x7f200c10d1d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2000009990 tx=0x7f2000009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 -- 172.21.15.26:0/2109381041 shutdown_connections 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 --2- 172.21.15.26:0/2109381041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 --2- 172.21.15.26:0/2109381041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073280 0x7f200c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 --2- 172.21.15.26:0/2109381041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073ba0 0x7f200c10d1d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 -- 172.21.15.26:0/2109381041 >> 172.21.15.26:0/2109381041 conn(0x7f200c0fc960 msgr2=0x7f200c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 -- 172.21.15.26:0/2109381041 shutdown_connections 2023-10-06T21:10:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.028+0000 7f20121ff700 1 -- 172.21.15.26:0/2109381041 wait complete. 2023-10-06T21:10:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.029+0000 7f20121ff700 1 Processor -- start 2023-10-06T21:10:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.029+0000 7f20121ff700 1 -- start start 2023-10-06T21:10:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073280 0x7f200c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 0x7f200c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c19e190 con 0x7f200c073ba0 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c19e300 con 0x7f200c073280 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f20121ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c19e4c0 con 0x7f200c10d710 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45314/0 (socket says 172.21.15.26:45314) 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 -- 172.21.15.26:0/3351892071 learned_addr learned my addr 172.21.15.26:0/3351892071 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200affd700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 0x7f200c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073280 0x7f200c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 -- 172.21.15.26:0/3351892071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073280 msgr2=0x7f200c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073280 0x7f200c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 -- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 msgr2=0x7f200c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.030+0000 7f200bfff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 0x7f200c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f200bfff700 1 -- 172.21.15.26:0/3351892071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000095f0 con 0x7f200c10d710 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f200affd700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 0x7f200c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f200bfff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2004007ec0 tx=0x7f200400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f2008ff9700 1 -- 172.21.15.26:0/3351892071 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004017070 con 0x7f200c10d710 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f2008ff9700 1 -- 172.21.15.26:0/3351892071 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f200400f040 con 0x7f200c10d710 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f2008ff9700 1 -- 172.21.15.26:0/3351892071 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004004280 con 0x7f200c10d710 2023-10-06T21:10:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.031+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c19e7a0 con 0x7f200c10d710 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.032+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c1a5fe0 con 0x7f200c10d710 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.039+0000 7f2008ff9700 1 -- 172.21.15.26:0/3351892071 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7f200400dc60 con 0x7f200c10d710 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.042+0000 7f2008ff9700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 0x7f1ff40a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.042+0000 7f2008ff9700 1 -- 172.21.15.26:0/3351892071 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f20040cc7d0 con 0x7f200c10d710 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.042+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 0x7f1ff40a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.043+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 0x7f1ff40a3b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009bf0 tx=0x7f1ffc009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.044+0000 7f2025bbcb80 1 Processor -- start 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.044+0000 7f2025bbcb80 1 -- start start 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f2a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e7d460 con 0x559a63f140c0 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e6ec20 con 0x559a63f149e0 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e6c540 con 0x559a63f280a0 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39976/0 (socket says 172.21.15.26:39976) 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f200b7fe700 1 -- 172.21.15.26:0/1513064660 learned_addr learned my addr 172.21.15.26:0/1513064660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.045+0000 7f200affd700 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200bfff700 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f2a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 -- 172.21.15.26:0/1513064660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 msgr2=0x559a63f2a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f2a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 -- 172.21.15.26:0/1513064660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 msgr2=0x559a63f27b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f27b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 -- 172.21.15.26:0/1513064660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a63c92de0 con 0x559a63f140c0 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.046+0000 7f200b7fe700 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc011e40 tx=0x7f1ffc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44da1b1502e8e04b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.047+0000 7f1ff1ffb700 1 -- 172.21.15.26:0/1513064660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc012dc0 con 0x559a63f140c0 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.047+0000 7f1ff1ffb700 1 -- 172.21.15.26:0/1513064660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ffc016070 con 0x559a63f140c0 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.047+0000 7f1ff1ffb700 1 -- 172.21.15.26:0/1513064660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc003dd0 con 0x559a63f140c0 2023-10-06T21:10:41.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.047+0000 7f2025bbcb80 1 -- 172.21.15.26:0/1513064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 msgr2=0x559a63f144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.047+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc011e40 tx=0x7f1ffc014000 comp rx=0 tx=0).stop 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 -- 172.21.15.26:0/1513064660 shutdown_connections 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f2a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f144a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/1513064660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 -- 172.21.15.26:0/1513064660 >> 172.21.15.26:0/1513064660 conn(0x559a63e89260 msgr2=0x559a63f10450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 -- 172.21.15.26:0/1513064660 shutdown_connections 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 -- 172.21.15.26:0/1513064660 wait complete. 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.048+0000 7f2025bbcb80 1 -- start start 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 Processor -- start 2023-10-06T21:10:41.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f27b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f25480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e67e30 con 0x559a63f140c0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e04710 con 0x559a63f149e0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63e043b0 con 0x559a63f280a0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f2025bbcb80 10 client.0 _ll_register_callbacks cb 0x559a63c61800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f200affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.049+0000 7f200affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53466/0 (socket says 172.21.15.26:53466) 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 -- 172.21.15.26:0/784186263 learned_addr learned my addr 172.21.15.26:0/784186263 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200b7fe700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f27b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200bfff700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f25480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 -- 172.21.15.26:0/784186263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 msgr2=0x559a63f25480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f25480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 -- 172.21.15.26:0/784186263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 msgr2=0x559a63f27b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f27b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 -- 172.21.15.26:0/784186263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc011b20 con 0x559a63f149e0 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200b7fe700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f27b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f200affd700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2000015000 tx=0x7f2000005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5e8b9b52c53ede server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.050+0000 7f1feaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200001d070 con 0x559a63f149e0 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a63f07f10 con 0x559a63f149e0 2023-10-06T21:10:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2000004910 con 0x559a63f149e0 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2000004ad0 con 0x559a63f149e0 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f2025bbcb80 10 client.25496 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559a63c7bc60 con 0x559a63f149e0 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.051+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2000033020 con 0x559a63f149e0 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2000024450 con 0x559a63f149e0 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 10 client.25496 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 10 client.25496 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 20 client.25496 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f37b40 0x559a63f39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 4 client.25496 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a63e60e10 con 0x559a63f37b40 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f200b7fe700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f37b40 0x559a63f39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f1feaffd700 10 client.25496 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.052+0000 7f200b7fe700 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f37b40 0x559a63f39f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=988bcb98219a4b9a server_cookie=ecc3f1b7355848e5 in_seq=0 out_seq=0 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.053+0000 7f1feaffd700 1 -- 172.21.15.26:0/784186263 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559a63e60e10 con 0x559a63f37b40 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.053+0000 7f1feaffd700 10 client.25496 handle_command_reply: tid=0 2023-10-06T21:10:41.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.053+0000 7f2025bbcb80 1 client.25496 shutdown 2023-10-06T21:10:41.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.053+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 msgr2=0x559a63f24d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.053+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2000015000 tx=0x7f2000005c90 comp rx=0 tx=0).stop 2023-10-06T21:10:41.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 shutdown_connections 2023-10-06T21:10:41.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f37b40 0x559a63f39f60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f25480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f140c0 0x559a63f27b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f1feaffd700 0 client.25496 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/784186263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f149e0 0x559a63f24d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 >> 172.21.15.26:0/784186263 conn(0x559a63e89260 msgr2=0x559a63f0ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 shutdown_connections 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 1 -- 172.21.15.26:0/784186263 wait complete. 2023-10-06T21:10:41.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.054+0000 7f2025bbcb80 20 client.25496 trim_cache size 0 max 16384 2023-10-06T21:10:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.070+0000 7f2025bbcb80 1 Processor -- start 2023-10-06T21:10:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 -- start start 2023-10-06T21:10:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f33370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f280a0 0x559a63f200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63ce8920 con 0x559a63f149e0 2023-10-06T21:10:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63f20cb0 con 0x559a63f280a0 2023-10-06T21:10:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63f15070 con 0x559a63f140c0 2023-10-06T21:10:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f200affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f33370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45342/0 (socket says 172.21.15.26:45342) 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.071+0000 7f200b7fe700 1 -- 172.21.15.26:0/484713078 learned_addr learned my addr 172.21.15.26:0/484713078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200bfff700 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f280a0 0x559a63f200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 -- 172.21.15.26:0/484713078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f280a0 msgr2=0x559a63f200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f280a0 0x559a63f200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 -- 172.21.15.26:0/484713078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 msgr2=0x559a63f33370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f33370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 -- 172.21.15.26:0/484713078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a63cb9250 con 0x559a63f140c0 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f200b7fe700 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc011770 tx=0x7f1ffc003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa8a9af15985af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f1ff27fc700 1 -- 172.21.15.26:0/484713078 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc012bc0 con 0x559a63f140c0 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f1ff27fc700 1 -- 172.21.15.26:0/484713078 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ffc02e040 con 0x559a63f140c0 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.072+0000 7f1ff27fc700 1 -- 172.21.15.26:0/484713078 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0053d0 con 0x559a63f140c0 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 -- 172.21.15.26:0/484713078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 msgr2=0x559a63f25460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc011770 tx=0x7f1ffc003800 comp rx=0 tx=0).stop 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 -- 172.21.15.26:0/484713078 shutdown_connections 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f280a0 0x559a63f200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f140c0 0x559a63f25460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/484713078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f33370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 -- 172.21.15.26:0/484713078 >> 172.21.15.26:0/484713078 conn(0x559a63e89260 msgr2=0x559a63f33ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 -- 172.21.15.26:0/484713078 shutdown_connections 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.073+0000 7f2025bbcb80 1 -- 172.21.15.26:0/484713078 wait complete. 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.074+0000 7f2025bbcb80 1 -- start start 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.074+0000 7f2025bbcb80 1 Processor -- start 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f8a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63f2ec20 con 0x559a63f149e0 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63f8aeb0 con 0x559a63f140c0 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a63f8aff0 con 0x559a63f280a0 2023-10-06T21:10:41.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f2025bbcb80 10 client.0 _ll_register_callbacks cb 0x559a63c61800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f200bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f8a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f200b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53492/0 (socket says 172.21.15.26:53492) 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f200b7fe700 1 -- 172.21.15.26:0/3387945820 learned_addr learned my addr 172.21.15.26:0/3387945820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.075+0000 7f200affd700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 -- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 msgr2=0x559a63f2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 -- 172.21.15.26:0/3387945820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 msgr2=0x559a63f2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 -- 172.21.15.26:0/3387945820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a63c92de0 con 0x559a63f280a0 2023-10-06T21:10:41.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f200bfff700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f8a970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f20040c9e70 tx=0x7f2004009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8624787d703561eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f1feaffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004099b40 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ffc016070 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2004099de0 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20040996e0 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f2025bbcb80 10 client.25474 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.076+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559a63c912b0 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ffc016070 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f20040a1550 con 0x559a63f280a0 2023-10-06T21:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 10 client.25474 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 10 client.25474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:10:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 20 client.25474 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f8c720 0x559a63f8eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 4 client.25474 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:10:41.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a63e60b30 con 0x559a63f8c720 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.077+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f8c720 0x559a63f8eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f200b7fe700 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f8c720 0x559a63f8eb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69cdf670faa6c470 server_cookie=928c3f3020e484a in_seq=0 out_seq=0 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f1feaffd700 10 client.25474 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f1feaffd700 1 -- 172.21.15.26:0/3387945820 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a63e60b30 con 0x559a63f8c720 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f1feaffd700 10 client.25474 handle_command_reply: tid=0 2023-10-06T21:10:41.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 client.25474 shutdown 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 msgr2=0x559a63f8a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f8a970 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f20040c9e70 tx=0x7f2004009f60 comp rx=0 tx=0).stop 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 shutdown_connections 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559a63f8c720 0x559a63f8eb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a63f280a0 0x559a63f8a970 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a63f140c0 0x559a63f2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 --2- 172.21.15.26:0/3387945820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a63f149e0 0x559a63f2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f1feaffd700 0 client.25474 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:10:41.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 >> 172.21.15.26:0/3387945820 conn(0x559a63e89260 msgr2=0x559a63f33370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 shutdown_connections 2023-10-06T21:10:41.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 1 -- 172.21.15.26:0/3387945820 wait complete. 2023-10-06T21:10:41.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.078+0000 7f2025bbcb80 20 client.25474 trim_cache size 0 max 16384 2023-10-06T21:10:41.139 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 msgr2=0x7f1ff40a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 0x7f1ff40a3b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009bf0 tx=0x7f1ffc009260 comp rx=0 tx=0).stop 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 msgr2=0x7f200c19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2004007ec0 tx=0x7f200400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 shutdown_connections 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f200c10d710 0x7f200c19db50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f200c073280 0x7f200c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1ff40a1680 0x7f1ff40a3b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 --2- 172.21.15.26:0/3351892071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200c073ba0 0x7f200c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 >> 172.21.15.26:0/3351892071 conn(0x7f200c0fc960 msgr2=0x7f200c10f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 shutdown_connections 2023-10-06T21:10:41.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.079+0000 7f20121ff700 1 -- 172.21.15.26:0/3351892071 wait complete. 2023-10-06T21:10:41.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:10:41.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:41 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/778107332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 -- 172.21.15.26:0/3964919250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b0101a60 msgr2=0x7fc7b0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/3964919250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b0101a60 0x7fc7b0105aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009990 tx=0x7fc79c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 -- 172.21.15.26:0/3964919250 shutdown_connections 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/3964919250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b01061d0 0x7fc7b010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/3964919250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 0x7fc7b0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/3964919250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b0101a60 0x7fc7b0105aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 -- 172.21.15.26:0/3964919250 >> 172.21.15.26:0/3964919250 conn(0x7fc7b00fc930 msgr2=0x7fc7b00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 -- 172.21.15.26:0/3964919250 shutdown_connections 2023-10-06T21:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.373+0000 7fc7b71ff700 1 -- 172.21.15.26:0/3964919250 wait complete. 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.374+0000 7fc7b71ff700 1 Processor -- start 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.374+0000 7fc7b71ff700 1 -- start start 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 0x7fc7b0197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 0x7fc7b019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019c410 con 0x7fc7b01010b0 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019c580 con 0x7fc7b01061d0 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b71ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019c740 con 0x7fc7b019b990 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b579c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 0x7fc7b019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53508/0 (socket says 172.21.15.26:53508) 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b579c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 0x7fc7b019be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45372/0 (socket says 172.21.15.26:45372) 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 -- 172.21.15.26:0/2328521347 learned_addr learned my addr 172.21.15.26:0/2328521347 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 0x7fc7b0197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 -- 172.21.15.26:0/2328521347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 msgr2=0x7fc7b019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 0x7fc7b019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 -- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 msgr2=0x7fc7b0197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 0x7fc7b0197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.375+0000 7fc7affff700 1 -- 172.21.15.26:0/2328521347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0095f0 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7affff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc79c000c00 tx=0x7fc79c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7adffb700 1 -- 172.21.15.26:0/2328521347 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c01d070 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7adffb700 1 -- 172.21.15.26:0/2328521347 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc79c021410 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b01a3c40 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7adffb700 1 -- 172.21.15.26:0/2328521347 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c017650 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.376+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01a4110 con 0x7fc7b01061d0 2023-10-06T21:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.379+0000 7fc7adffb700 1 -- 172.21.15.26:0/2328521347 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7fc79c021580 con 0x7fc7b01061d0 2023-10-06T21:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.380+0000 7fc7cabb5b80 1 Processor -- start 2023-10-06T21:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7adffb700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 0x7fc7a00a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 -- start start 2023-10-06T21:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7adffb700 1 -- 172.21.15.26:0/2328521347 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc79c0ce450 con 0x7fc7b01061d0 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7b579c700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 0x7fc7a00a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572eae460 con 0x563572f54ca0 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572e9fc20 con 0x563572f450c0 2023-10-06T21:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572e9d540 con 0x563572f459e0 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53512/0 (socket says 172.21.15.26:53512) 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.381+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/2243843784 learned_addr learned my addr 172.21.15.26:0/2243843784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7affff700 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/2243843784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 msgr2=0x563572f54760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/2243843784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 msgr2=0x563572f57090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/2243843784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563572cc3de0 con 0x563572f450c0 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d80 tx=0x7fc7a4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b973b5dbd184f103 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc7b579c700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 0x7fc7a00a3bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800f590 tx=0x7fc7a8009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc79affd700 1 -- 172.21.15.26:0/2243843784 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4004230 con 0x563572f450c0 2023-10-06T21:10:41.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc79affd700 1 -- 172.21.15.26:0/2243843784 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a4004390 con 0x563572f450c0 2023-10-06T21:10:41.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.382+0000 7fc79affd700 1 -- 172.21.15.26:0/2243843784 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40198c0 con 0x563572f450c0 2023-10-06T21:10:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/2243843784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 msgr2=0x563572f454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009d80 tx=0x7fc7a4011720 comp rx=0 tx=0).stop 2023-10-06T21:10:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/2243843784 shutdown_connections 2023-10-06T21:10:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f454a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/2243843784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/2243843784 >> 172.21.15.26:0/2243843784 conn(0x563572eba260 msgr2=0x563572f41450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/2243843784 shutdown_connections 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.383+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/2243843784 wait complete. 2023-10-06T21:10:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 -- start start 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 Processor -- start 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f459e0 0x563572f519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f54ca0 0x563572f52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572e98e30 con 0x563572f459e0 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.384+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572e35710 con 0x563572f450c0 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572e353b0 con 0x563572f54ca0 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7cabb5b80 10 client.0 _ll_register_callbacks cb 0x563572c92800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7affff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f459e0 0x563572f519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53522/0 (socket says 172.21.15.26:53522) 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/3843018044 learned_addr learned my addr 172.21.15.26:0/3843018044 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b579c700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f54ca0 0x563572f52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/3843018044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f54ca0 msgr2=0x563572f52110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f54ca0 0x563572f52110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f459e0 msgr2=0x563572f519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f459e0 0x563572f519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.385+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/3843018044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a40099e0 con 0x563572f450c0 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4000c00 tx=0x7fc7a40038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5722152e20b20945 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc78ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4005450 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563572f38f10 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc7a4005ce0 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a4009ed0 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc7cabb5b80 10 client.25514 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563572cacc60 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.386+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563572f38f10 con 0x563572f450c0 2023-10-06T21:10:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc7a4021040 con 0x563572f450c0 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 10 client.25514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 20 client.25514 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572f64630 0x563572f66a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 4 client.25514 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563572e91e10 con 0x563572f64630 2023-10-06T21:10:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.387+0000 7fc7affff700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572f64630 0x563572f66a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc78ffff700 10 client.25514 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7affff700 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572f64630 0x563572f66a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f255ceb9f5a01d2 server_cookie=de4206318844d4ed in_seq=0 out_seq=0 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc78ffff700 1 -- 172.21.15.26:0/3843018044 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563572e91e10 con 0x563572f64630 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc78ffff700 10 client.25514 handle_command_reply: tid=0 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7cabb5b80 1 client.25514 shutdown 2023-10-06T21:10:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 msgr2=0x563572f54750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4000c00 tx=0x7fc7a40038b0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 shutdown_connections 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.388+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572f64630 0x563572f66a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f54ca0 0x563572f52110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f54750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3843018044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f459e0 0x563572f519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 >> 172.21.15.26:0/3843018044 conn(0x563572eba260 msgr2=0x563572f41040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc78ffff700 0 client.25514 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 shutdown_connections 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3843018044 wait complete. 2023-10-06T21:10:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.389+0000 7fc7cabb5b80 20 client.25514 trim_cache size 0 max 16384 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.402+0000 7fc7cabb5b80 1 Processor -- start 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 -- start start 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572d19920 con 0x563572f54ca0 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572f46020 con 0x563572f450c0 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572f47f60 con 0x563572f459e0 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53526/0 (socket says 172.21.15.26:53526) 2023-10-06T21:10:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/104602057 learned_addr learned my addr 172.21.15.26:0/104602057 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7b579c700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.403+0000 7fc7affff700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/104602057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 msgr2=0x563572f54280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/104602057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 msgr2=0x563572f4cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f4cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 -- 172.21.15.26:0/104602057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563572cea250 con 0x563572f450c0 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7affff700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b579c700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f4cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005a00 tx=0x7fc7a4005a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6adf8ca6c999917f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc79b7fe700 1 -- 172.21.15.26:0/104602057 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4009ed0 con 0x563572f450c0 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc79b7fe700 1 -- 172.21.15.26:0/104602057 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a4003d10 con 0x563572f450c0 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.404+0000 7fc79b7fe700 1 -- 172.21.15.26:0/104602057 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4005580 con 0x563572f450c0 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/104602057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 msgr2=0x563572f57090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005a00 tx=0x7fc7a4005a30 comp rx=0 tx=0).stop 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/104602057 shutdown_connections 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f4cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f450c0 0x563572f57090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/104602057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f459e0 0x563572f54280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/104602057 >> 172.21.15.26:0/104602057 conn(0x563572eba260 msgr2=0x563572f60a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/104602057 shutdown_connections 2023-10-06T21:10:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/104602057 wait complete. 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 -- start start 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.405+0000 7fc7cabb5b80 1 Processor -- start 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f450c0 0x563572f577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572fb7470 con 0x563572f54ca0 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572fb75b0 con 0x563572f459e0 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563572fb7720 con 0x563572f450c0 2023-10-06T21:10:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7cabb5b80 10 client.0 _ll_register_callbacks cb 0x7fc7a4009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7b579c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7b4f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f450c0 0x563572f577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53532/0 (socket says 172.21.15.26:53532) 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.406+0000 7fc7affff700 1 -- 172.21.15.26:0/3061656721 learned_addr learned my addr 172.21.15.26:0/3061656721 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 -- 172.21.15.26:0/3061656721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f450c0 msgr2=0x563572f577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f450c0 0x563572f577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 -- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 msgr2=0x563572f58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f58270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 -- 172.21.15.26:0/3061656721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563572cacc60 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7affff700 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0cc1b0 tx=0x7fc79c097070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8aed50cd9930dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc78ffff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c09ba60 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc79c0b2070 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7a4004390 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c09b360 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7cabb5b80 10 client.25520 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.407+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563572cc22b0 con 0x563572f459e0 2023-10-06T21:10:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a4004390 con 0x563572f459e0 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc79c0cbda0 con 0x563572f459e0 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 10 client.25520 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 10 client.25520 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 20 client.25520 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572fb8b70 0x563572fbaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 4 client.25520 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563572e91b30 con 0x563572fb8b70 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.408+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572fb8b70 0x563572fbaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7b4f9b700 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572fb8b70 0x563572fbaf90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c779ef730eca6d58 server_cookie=fb65abc8324f1577 in_seq=0 out_seq=0 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc78ffff700 10 client.25520 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc78ffff700 1 -- 172.21.15.26:0/3061656721 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563572e91b30 con 0x563572fb8b70 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc78ffff700 10 client.25520 handle_command_reply: tid=0 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 client.25520 shutdown 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 msgr2=0x563572f57d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0cc1b0 tx=0x7fc79c097070 comp rx=0 tx=0).stop 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 shutdown_connections 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563572f54ca0 0x563572f58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563572fb8b70 0x563572fbaf90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563572f450c0 0x563572f577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 --2- 172.21.15.26:0/3061656721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563572f459e0 0x563572f57d30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc78ffff700 0 client.25520 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 >> 172.21.15.26:0/3061656721 conn(0x563572eba260 msgr2=0x563572f4cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 shutdown_connections 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 1 -- 172.21.15.26:0/3061656721 wait complete. 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.409+0000 7fc7cabb5b80 20 client.25520 trim_cache size 0 max 16384 2023-10-06T21:10:41.435 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:10:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 msgr2=0x7fc7a00a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 0x7fc7a00a3bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800f590 tx=0x7fc7a8009920 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 msgr2=0x7fc7b01975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc79c000c00 tx=0x7fc79c005740 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 shutdown_connections 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7b019b990 0x7fc7b019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc7a00a1700 0x7fc7a00a3bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b01010b0 0x7fc7b0197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 --2- 172.21.15.26:0/2328521347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7b01061d0 0x7fc7b01975d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 >> 172.21.15.26:0/2328521347 conn(0x7fc7b00fc930 msgr2=0x7fc7b00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 shutdown_connections 2023-10-06T21:10:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.410+0000 7fc7b71ff700 1 -- 172.21.15.26:0/2328521347 wait complete. 2023-10-06T21:10:41.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:41 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/778107332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:41.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:10:41.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:41 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/778107332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:10:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 -- 172.21.15.26:0/631740066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c101a60 msgr2=0x7fb42c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 --2- 172.21.15.26:0/631740066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c101a60 0x7fb42c105aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb420009990 tx=0x7fb420009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 -- 172.21.15.26:0/631740066 shutdown_connections 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 --2- 172.21.15.26:0/631740066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1061d0 0x7fb42c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 --2- 172.21.15.26:0/631740066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1010b0 0x7fb42c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 --2- 172.21.15.26:0/631740066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c101a60 0x7fb42c105aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 -- 172.21.15.26:0/631740066 >> 172.21.15.26:0/631740066 conn(0x7fb42c0fc930 msgr2=0x7fb42c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.682+0000 7fb4329ff700 1 -- 172.21.15.26:0/631740066 shutdown_connections 2023-10-06T21:10:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.683+0000 7fb4329ff700 1 -- 172.21.15.26:0/631740066 wait complete. 2023-10-06T21:10:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.683+0000 7fb4329ff700 1 Processor -- start 2023-10-06T21:10:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 -- start start 2023-10-06T21:10:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1010b0 0x7fb42c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c19e5f0 con 0x7fb42c1061d0 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c19e760 con 0x7fb42c1010b0 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb4329ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42c19e920 con 0x7fb42c19db70 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb430f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1010b0 0x7fb42c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40066/0 (socket says 172.21.15.26:40066) 2023-10-06T21:10:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb430f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45420/0 (socket says 172.21.15.26:45420) 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.684+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2915931109 learned_addr learned my addr 172.21.15.26:0/2915931109 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2915931109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 msgr2=0x7fb42c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2915931109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1010b0 msgr2=0x7fb42c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1010b0 0x7fb42c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2915931109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424009c40 con 0x7fb42c1061d0 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb430f9c700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb420000c00 tx=0x7fb420005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.685+0000 7fb4297fa700 1 -- 172.21.15.26:0/2915931109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42001d070 con 0x7fb42c1061d0 2023-10-06T21:10:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.686+0000 7fb4297fa700 1 -- 172.21.15.26:0/2915931109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb420021410 con 0x7fb42c1061d0 2023-10-06T21:10:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.686+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4200095f0 con 0x7fb42c1061d0 2023-10-06T21:10:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.686+0000 7fb4297fa700 1 -- 172.21.15.26:0/2915931109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420017650 con 0x7fb42c1061d0 2023-10-06T21:10:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.686+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c1a60b0 con 0x7fb42c1061d0 2023-10-06T21:10:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.689+0000 7fb4297fa700 1 -- 172.21.15.26:0/2915931109 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7fb420021880 con 0x7fb42c1061d0 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.691+0000 7fb4462fbb80 1 Processor -- start 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4297fa700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 0x7fb4140a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 -- start start 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4297fa700 1 -- 172.21.15.26:0/2915931109 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb4200ce4e0 con 0x7fb42c1061d0 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb430f9c700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 0x7fb4140a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76914b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 0x55f769156350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f769156890 0x55f769158c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7690b4460 con 0x55f76914b0c0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7690a5c20 con 0x55f76914b9e0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7690a3540 con 0x55f769156890 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.692+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76914b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76914b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40072/0 (socket says 172.21.15.26:40072) 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 0x55f769156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42bfff700 1 -- 172.21.15.26:0/3403214028 learned_addr learned my addr 172.21.15.26:0/3403214028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb430f9c700 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f769156890 0x55f769158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3403214028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f769156890 msgr2=0x55f769158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f769156890 0x55f769158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb430f9c700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 0x7fb4140a3b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb424009fd0 tx=0x7fb42400b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3403214028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 msgr2=0x55f76914b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76914b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3403214028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f768ec9de0 con 0x55f76914b9e0 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 0x55f769156350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cbb20 tx=0x7fb420097750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3541660a5c1f9876 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb4127fc700 1 -- 172.21.15.26:0/3403214028 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42009acf0 con 0x55f76914b9e0 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb4127fc700 1 -- 172.21.15.26:0/3403214028 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42009ba90 con 0x55f76914b9e0 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.693+0000 7fb4127fc700 1 -- 172.21.15.26:0/3403214028 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200cbdd0 con 0x55f76914b9e0 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3403214028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 msgr2=0x55f769156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 0x55f769156350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cbb20 tx=0x7fb420097750 comp rx=0 tx=0).stop 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3403214028 shutdown_connections 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f769156890 0x55f769158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76914b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3403214028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b9e0 0x55f769156350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.694+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3403214028 >> 172.21.15.26:0/3403214028 conn(0x55f7690c0260 msgr2=0x55f769147450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.695+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3403214028 shutdown_connections 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.695+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3403214028 wait complete. 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.695+0000 7fb4462fbb80 1 -- start start 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.696+0000 7fb4462fbb80 1 Processor -- start 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.696+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.696+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f7691571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f7691576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76909ee30 con 0x55f76914b0c0 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76903b710 con 0x55f769156890 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76903b3b0 con 0x55f76914b9e0 2023-10-06T21:10:41.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb4462fbb80 10 client.0 _ll_register_callbacks cb 0x55f768e98800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f7691571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40078/0 (socket says 172.21.15.26:40078) 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42bfff700 1 -- 172.21.15.26:0/3847253459 learned_addr learned my addr 172.21.15.26:0/3847253459 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb430f9c700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f7691576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 msgr2=0x55f7691576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f7691576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3847253459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 msgr2=0x55f769156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42b7fe700 1 -- 172.21.15.26:0/3847253459 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42c1a60b0 con 0x55f76914b9e0 2023-10-06T21:10:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.697+0000 7fb42bfff700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f7691571b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b2e0 tx=0x7fb420021c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e8e9e5b1060da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200a3690 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f76913ef10 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb42009f4e0 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb42009acf0 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb4462fbb80 10 client.25504 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f768eb2c60 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f76913ef10 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.698+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4200a3200 con 0x55f76914b9e0 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 10 client.25504 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 20 client.25504 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f769165ae0 0x55f769167f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 4 client.25504 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f769097e10 con 0x55f769165ae0 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb42bfff700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f769165ae0 0x55f769167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb42bfff700 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f769165ae0 0x55f769167f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803b0d8e4903e41f server_cookie=fab1fc398b882868 in_seq=0 out_seq=0 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.699+0000 7fb40b7fe700 10 client.25504 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb40b7fe700 1 -- 172.21.15.26:0/3847253459 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f769097e10 con 0x55f769165ae0 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb40b7fe700 10 client.25504 handle_command_reply: tid=0 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 client.25504 shutdown 2023-10-06T21:10:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 msgr2=0x55f7691571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f7691571b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb42000b2e0 tx=0x7fb420021c80 comp rx=0 tx=0).stop 2023-10-06T21:10:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 shutdown_connections 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f769165ae0 0x55f769167f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f7691576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.700+0000 7fb40b7fe700 0 client.25504 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f769156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/3847253459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f7691571b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 >> 172.21.15.26:0/3847253459 conn(0x55f7690c0260 msgr2=0x55f769146fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 shutdown_connections 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 1 -- 172.21.15.26:0/3847253459 wait complete. 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.701+0000 7fb4462fbb80 20 client.25504 trim_cache size 0 max 16384 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.715+0000 7fb4462fbb80 1 Processor -- start 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.715+0000 7fb4462fbb80 1 -- start start 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f769152c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f769156890 0x55f769153140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f768f1f920 con 0x55f769156890 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f769151840 con 0x55f76914b0c0 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76914c020 con 0x55f76914b9e0 2023-10-06T21:10:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53578/0 (socket says 172.21.15.26:53578) 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb42bfff700 1 -- 172.21.15.26:0/2548708948 learned_addr learned my addr 172.21.15.26:0/2548708948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f769152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.716+0000 7fb430f9c700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f769156890 0x55f769153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 -- 172.21.15.26:0/2548708948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 msgr2=0x55f769152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f769152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 -- 172.21.15.26:0/2548708948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f769156890 msgr2=0x55f769153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f769156890 0x55f769153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 -- 172.21.15.26:0/2548708948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7691562c0 con 0x55f76914b0c0 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f769152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb42bfff700 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c01fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2d49930fbed54e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb412ffd700 1 -- 172.21.15.26:0/2548708948 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c010a90 con 0x55f76914b0c0 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.717+0000 7fb412ffd700 1 -- 172.21.15.26:0/2548708948 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb41c023070 con 0x55f76914b0c0 2023-10-06T21:10:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb412ffd700 1 -- 172.21.15.26:0/2548708948 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004cf0 con 0x55f76914b0c0 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2548708948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 msgr2=0x55f7691613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c01fcd0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2548708948 shutdown_connections 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f769156890 0x55f769153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f76914b0c0 0x55f7691613e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2548708948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f769152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2548708948 >> 172.21.15.26:0/2548708948 conn(0x55f7690c0260 msgr2=0x55f769150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2548708948 shutdown_connections 2023-10-06T21:10:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.718+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2548708948 wait complete. 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.719+0000 7fb4462fbb80 1 -- start start 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.719+0000 7fb4462fbb80 1 Processor -- start 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76915b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f76915bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f769159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f769159ff0 con 0x55f76914b0c0 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76915a160 con 0x55f769156890 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76915a320 con 0x55f76914b9e0 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb4462fbb80 10 client.0 _ll_register_callbacks cb 0x55f768e98800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76915b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb42bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76915b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40090/0 (socket says 172.21.15.26:40090) 2023-10-06T21:10:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb42bfff700 1 -- 172.21.15.26:0/2842610240 learned_addr learned my addr 172.21.15.26:0/2842610240 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f76915bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.720+0000 7fb430f9c700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f769159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 msgr2=0x55f769159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f769159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2842610240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 msgr2=0x55f76915b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76915b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 -- 172.21.15.26:0/2842610240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f768eb2c60 con 0x55f76914b9e0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb42b7fe700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f76915bba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4200996e0 tx=0x7fb4200970d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a551b7809a9ab88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb40b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200a3690 con 0x55f76914b9e0 2023-10-06T21:10:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb42009f4e0 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb42009ba90 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.721+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb420009290 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 10 client.25510 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fb41c005ce0 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42009ba90 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4200a1290 con 0x55f76914b9e0 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 20 client.25510 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f7691ba610 0x55f7691bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 4 client.25510 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f769097b30 con 0x55f7691ba610 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.722+0000 7fb42bfff700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f7691ba610 0x55f7691bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb42bfff700 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f7691ba610 0x55f7691bca00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2de83eeb1d886d1 server_cookie=4cf5f9601dba9f2 in_seq=0 out_seq=0 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb40b7fe700 10 client.25510 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb40b7fe700 1 -- 172.21.15.26:0/2842610240 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f769097b30 con 0x55f7691ba610 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb40b7fe700 10 client.25510 handle_command_reply: tid=0 2023-10-06T21:10:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 client.25510 shutdown 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 msgr2=0x55f76915bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f76915bba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4200996e0 tx=0x7fb4200970d0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 shutdown_connections 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f7691ba610 0x55f7691bca00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f769156890 0x55f769159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f76914b0c0 0x55f76915b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 --2- 172.21.15.26:0/2842610240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f76914b9e0 0x55f76915bba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 >> 172.21.15.26:0/2842610240 conn(0x55f7690c0260 msgr2=0x55f769154120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb40b7fe700 0 client.25510 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.723+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 shutdown_connections 2023-10-06T21:10:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4462fbb80 1 -- 172.21.15.26:0/2842610240 wait complete. 2023-10-06T21:10:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4462fbb80 20 client.25510 trim_cache size 0 max 16384 2023-10-06T21:10:41.753 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:10:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 msgr2=0x7fb4140a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 0x7fb4140a3b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb424009fd0 tx=0x7fb42400b040 comp rx=0 tx=0).stop 2023-10-06T21:10:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 msgr2=0x7fb42c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb420000c00 tx=0x7fb420005740 comp rx=0 tx=0).stop 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 shutdown_connections 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42c19db70 0x7fb42c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb42c1010b0 0x7fb42c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4140a1680 0x7fb4140a3b40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 --2- 172.21.15.26:0/2915931109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42c1061d0 0x7fb42c199710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.724+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 >> 172.21.15.26:0/2915931109 conn(0x7fb42c0fc930 msgr2=0x7fb42c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.725+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 shutdown_connections 2023-10-06T21:10:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:41.725+0000 7fb4329ff700 1 -- 172.21.15.26:0/2915931109 wait complete. 2023-10-06T21:10:41.985 INFO:teuthology.orchestra.run.smithi026.stderr:done. 2023-10-06T21:10:41.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-10-06T21:10:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:starting vacuum...end. 2023-10-06T21:10:42.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:42 smithi026 ceph-mon[98490]: pgmap v14: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 60 MiB/s wr, 326 op/s 2023-10-06T21:10:42.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:42 smithi078 ceph-mon[134896]: pgmap v14: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 60 MiB/s wr, 326 op/s 2023-10-06T21:10:42.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:42 smithi167 ceph-mon[134494]: pgmap v14: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 60 MiB/s wr, 326 op/s 2023-10-06T21:10:42.756 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:10:42.757 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:10:43.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.096+0000 7f79e25ff700 1 -- 172.21.15.26:0/2942877898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc0732e0 msgr2=0x7f79dc0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.096+0000 7f79e25ff700 1 --2- 172.21.15.26:0/2942877898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc0732e0 0x7f79dc0736c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009990 tx=0x7f79cc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.096+0000 7f79e25ff700 1 -- 172.21.15.26:0/2942877898 shutdown_connections 2023-10-06T21:10:43.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 --2- 172.21.15.26:0/2942877898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc10d680 0x7f79dc10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 --2- 172.21.15.26:0/2942877898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc0732e0 0x7f79dc0736c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 --2- 172.21.15.26:0/2942877898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 -- 172.21.15.26:0/2942877898 >> 172.21.15.26:0/2942877898 conn(0x7f79dc0fc950 msgr2=0x7f79dc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 -- 172.21.15.26:0/2942877898 shutdown_connections 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.097+0000 7f79e25ff700 1 -- 172.21.15.26:0/2942877898 wait complete. 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.098+0000 7f79e25ff700 1 Processor -- start 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 -- start start 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc10d680 0x7f79dc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19e1a0 con 0x7f79dc0732e0 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19e310 con 0x7f79dc073c00 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79e25ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19e4d0 con 0x7f79dc10d680 2023-10-06T21:10:43.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.099+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53594/0 (socket says 172.21.15.26:53594) 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 -- 172.21.15.26:0/3790953308 learned_addr learned my addr 172.21.15.26:0/3790953308 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40104/0 (socket says 172.21.15.26:40104) 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79e0b9c700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc10d680 0x7f79dc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 -- 172.21.15.26:0/3790953308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc10d680 msgr2=0x7f79dc19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc10d680 0x7f79dc19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 -- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 msgr2=0x7f79dc1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 -- 172.21.15.26:0/3790953308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0095f0 con 0x7f79dc073c00 2023-10-06T21:10:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79dbfff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79d000e9a0 tx=0x7f79d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.100+0000 7f79d97fa700 1 -- 172.21.15.26:0/3790953308 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0009790 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.101+0000 7f79d97fa700 1 -- 172.21.15.26:0/3790953308 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d0010430 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.101+0000 7f79d97fa700 1 -- 172.21.15.26:0/3790953308 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0018680 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.101+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc19e7b0 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.101+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc1a5ff0 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.106+0000 7f79d97fa700 1 -- 172.21.15.26:0/3790953308 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7f79d00105a0 con 0x7f79dc073c00 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.107+0000 7f79f5f7bb80 1 Processor -- start 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.107+0000 7f79f5f7bb80 1 -- start start 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27370520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e2737b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e2737b910 0x562e2737dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e272d94e0 con 0x562e27370140 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e272caca0 con 0x562e27370a60 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e272c85c0 con 0x562e2737b910 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79e0b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27370520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79e0b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27370520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40116/0 (socket says 172.21.15.26:40116) 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79e0b9c700 1 -- 172.21.15.26:0/1386325015 learned_addr learned my addr 172.21.15.26:0/1386325015 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79dbfff700 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e2737b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79db7fe700 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e2737b910 0x562e2737dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.108+0000 7f79d97fa700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 0x7f79c40a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 -- 172.21.15.26:0/1386325015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e2737b910 msgr2=0x562e2737dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e2737b910 0x562e2737dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 -- 172.21.15.26:0/1386325015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 msgr2=0x562e27370520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27370520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 -- 172.21.15.26:0/1386325015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e270eee60 con 0x562e27370a60 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79d97fa700 1 -- 172.21.15.26:0/3790953308 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f79d0014070 con 0x7f79dc073c00 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79e0b9c700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 0x7f79c40a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79dbfff700 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e2737b3d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c00 tx=0x7f79cc005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c6d718c18e5c61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79c27fc700 1 -- 172.21.15.26:0/1386325015 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01d070 con 0x562e27370a60 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79c27fc700 1 -- 172.21.15.26:0/1386325015 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc01e040 con 0x562e27370a60 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.109+0000 7f79c27fc700 1 -- 172.21.15.26:0/1386325015 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc004da0 con 0x562e27370a60 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.110+0000 7f79e0b9c700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 0x7f79c40a3bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f79d40075c0 tx=0x7f79d4009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.110+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/1386325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 msgr2=0x562e2737b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.110+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e2737b3d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79cc000c00 tx=0x7f79cc005ee0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/1386325015 shutdown_connections 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e2737b910 0x562e2737dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27370520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/1386325015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e2737b3d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/1386325015 >> 172.21.15.26:0/1386325015 conn(0x562e272e52e0 msgr2=0x562e2736c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/1386325015 shutdown_connections 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/1386325015 wait complete. 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.111+0000 7f79f5f7bb80 1 -- start start 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.112+0000 7f79f5f7bb80 1 Processor -- start 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.112+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.112+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e27378b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.112+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e2737b910 0x562e273790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.112+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e272c3eb0 con 0x562e2737b910 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e27260790 con 0x562e27370a60 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e27260430 con 0x562e27370140 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79f5f7bb80 10 client.0 _ll_register_callbacks cb 0x562e270bd880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e27378b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45478/0 (socket says 172.21.15.26:45478) 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 -- 172.21.15.26:0/3920577948 learned_addr learned my addr 172.21.15.26:0/3920577948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79e0b9c700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e2737b910 0x562e273790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 -- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 msgr2=0x562e27378b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e27378b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 -- 172.21.15.26:0/3920577948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e2737b910 msgr2=0x562e273790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e2737b910 0x562e273790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79dbfff700 1 -- 172.21.15.26:0/3920577948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc1107b0 con 0x562e27370140 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.113+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e27378b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79dbfff700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00f690 tx=0x7f79cc00f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3fca54dcfa1785c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01d040 con 0x562e27370140 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e27363f90 con 0x562e27370140 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f79cc034460 con 0x562e27370140 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79cc00fe60 con 0x562e27370140 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79f5f7bb80 10 client.25522 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:43.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562e270d7ce0 con 0x562e27370140 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.114+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e27363f90 con 0x562e27370140 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f79cc0225c0 con 0x562e27370140 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 10 client.25522 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 10 client.25522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 20 client.25522 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e2738aea0 0x562e2738d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 4 client.25522 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e272bce90 con 0x562e2738aea0 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e2738aea0 0x562e2738d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79bb7fe700 10 client.25522 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.115+0000 7f79db7fe700 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e2738aea0 0x562e2738d2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b20e97a0f31ca5 server_cookie=70391339a557b67a in_seq=0 out_seq=0 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79bb7fe700 1 -- 172.21.15.26:0/3920577948 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e272bce90 con 0x562e2738aea0 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79bb7fe700 10 client.25522 handle_command_reply: tid=0 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 client.25522 shutdown 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 msgr2=0x562e27378650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00f690 tx=0x7f79cc00f6c0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 shutdown_connections 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e2737b910 0x562e273790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370140 0x562e27378650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e2738aea0 0x562e2738d2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/3920577948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e27370a60 0x562e27378b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79bb7fe700 0 client.25522 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.116+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 >> 172.21.15.26:0/3920577948 conn(0x562e272e52e0 msgr2=0x562e2736c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.117+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 shutdown_connections 2023-10-06T21:10:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.117+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/3920577948 wait complete. 2023-10-06T21:10:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.117+0000 7f79f5f7bb80 20 client.25522 trim_cache size 0 max 16384 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.150+0000 7f79f5f7bb80 1 Processor -- start 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.150+0000 7f79f5f7bb80 1 -- start start 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27378f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e27375e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e271449a0 con 0x562e27370140 2023-10-06T21:10:43.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e273744a0 con 0x562e2737b910 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2736ae90 con 0x562e27370a60 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79e0b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e27375e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45482/0 (socket says 172.21.15.26:45482) 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 -- 172.21.15.26:0/2791825320 learned_addr learned my addr 172.21.15.26:0/2791825320 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79e0b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e27375e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53620/0 (socket says 172.21.15.26:53620) 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 -- 172.21.15.26:0/2791825320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 msgr2=0x562e27375e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e27375e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79db7fe700 1 -- 172.21.15.26:0/2791825320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 msgr2=0x562e27378f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.151+0000 7f79dbfff700 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27378f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79db7fe700 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27378f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79db7fe700 1 -- 172.21.15.26:0/2791825320 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e27391eb0 con 0x562e27370a60 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79dbfff700 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27378f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79db7fe700 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c2d0 tx=0x7f79d000c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506b3fcfa0a8f991 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79c2ffd700 1 -- 172.21.15.26:0/2791825320 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d009a0a0 con 0x562e27370a60 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79c2ffd700 1 -- 172.21.15.26:0/2791825320 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d0099e10 con 0x562e27370a60 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.152+0000 7f79c2ffd700 1 -- 172.21.15.26:0/2791825320 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d009e130 con 0x562e27370a60 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/2791825320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 msgr2=0x562e273758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c2d0 tx=0x7f79d000c4d0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/2791825320 shutdown_connections 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e27375e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27378f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/2791825320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e273758c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/2791825320 >> 172.21.15.26:0/2791825320 conn(0x562e272e52e0 msgr2=0x562e27377440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/2791825320 shutdown_connections 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.153+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/2791825320 wait complete. 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 -- start start 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 Processor -- start 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27381520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e273e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e273e0a20 con 0x562e27370140 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e273e0b90 con 0x562e2737b910 2023-10-06T21:10:43.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.154+0000 7f79f5f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e273e0d50 con 0x562e27370a60 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79f5f7bb80 10 client.0 _ll_register_callbacks cb 0x7f79d4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45490/0 (socket says 172.21.15.26:45490) 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 -- 172.21.15.26:0/4031131498 learned_addr learned my addr 172.21.15.26:0/4031131498 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79e0b9c700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e273e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79dbfff700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27381520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 -- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 msgr2=0x562e273e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e273e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 -- 172.21.15.26:0/4031131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 msgr2=0x562e27381520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27381520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 -- 172.21.15.26:0/4031131498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc1a5ff0 con 0x562e27370a60 2023-10-06T21:10:43.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79db7fe700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79d009d040 tx=0x7f79d00caa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed22396f0f4c2c76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.155+0000 7f79dbfff700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27381520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.156+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d009a9d0 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.156+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f79d009a2d0 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.156+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79cc01e040 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d00a2660 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 10 client.25528 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562e271152d0 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc01e040 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f79d00a21d0 con 0x562e27370a60 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 10 client.25528 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 20 client.25528 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e273e9430 0x562e273eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 4 client.25528 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562e272bcbb0 con 0x562e273e9430 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.157+0000 7f79dbfff700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e273e9430 0x562e273eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.158+0000 7f79dbfff700 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e273e9430 0x562e273eb850 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d8fcbf3a101a74 server_cookie=a6470ef8ed3676ba in_seq=0 out_seq=0 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.158+0000 7f79bb7fe700 10 client.25528 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.159+0000 7f79bb7fe700 1 -- 172.21.15.26:0/4031131498 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562e272bcbb0 con 0x562e273e9430 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.159+0000 7f79bb7fe700 10 client.25528 handle_command_reply: tid=0 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.159+0000 7f79f5f7bb80 1 client.25528 shutdown 2023-10-06T21:10:43.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 msgr2=0x562e27381a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f79d009d040 tx=0x7f79d00caa90 comp rx=0 tx=0).stop 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 shutdown_connections 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562e273e9430 0x562e273eb850 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e2737b910 0x562e273e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562e27370140 0x562e27381520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 --2- 172.21.15.26:0/4031131498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e27370a60 0x562e27381a60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79bb7fe700 0 client.25528 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 >> 172.21.15.26:0/4031131498 conn(0x562e272e52e0 msgr2=0x562e2736b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 shutdown_connections 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.160+0000 7f79f5f7bb80 1 -- 172.21.15.26:0/4031131498 wait complete. 2023-10-06T21:10:43.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79f5f7bb80 20 client.25528 trim_cache size 0 max 16384 2023-10-06T21:10:43.172 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:10:43.172 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:10:43.172 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "3b1d5b86-9c30-4bb8-a408-7612a48acc29", 2023-10-06T21:10:43.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:10:43.172 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 msgr2=0x7f79c40a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 0x7f79c40a3bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f79d40075c0 tx=0x7f79d4009040 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 msgr2=0x7f79dc199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79d000e9a0 tx=0x7f79d000ecb0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 shutdown_connections 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79dc10d680 0x7f79dc19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f79c40a1700 0x7f79c40a3bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79dc0732e0 0x7f79dc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 --2- 172.21.15.26:0/3790953308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79dc073c00 0x7f79dc199700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.161+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 >> 172.21.15.26:0/3790953308 conn(0x7f79dc0fc950 msgr2=0x7f79dc10f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.162+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 shutdown_connections 2023-10-06T21:10:43.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.162+0000 7f79e25ff700 1 -- 172.21.15.26:0/3790953308 wait complete. 2023-10-06T21:10:43.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 -- 172.21.15.26:0/3501674938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7410d570 msgr2=0x7f6e7410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3501674938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7410d570 0x7f6e7410d950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009920 tx=0x7f6e64009c30 comp rx=0 tx=0).stop 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 -- 172.21.15.26:0/3501674938 shutdown_connections 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3501674938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e74107f90 0x7f6e74114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3501674938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7410d570 0x7f6e7410d950 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3501674938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e741075d0 0x7f6e74107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.523+0000 7f6e791ff700 1 -- 172.21.15.26:0/3501674938 >> 172.21.15.26:0/3501674938 conn(0x7f6e74076d50 msgr2=0x7f6e74105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.525+0000 7f6e791ff700 1 -- 172.21.15.26:0/3501674938 shutdown_connections 2023-10-06T21:10:43.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.525+0000 7f6e791ff700 1 -- 172.21.15.26:0/3501674938 wait complete. 2023-10-06T21:10:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.526+0000 7f6e791ff700 1 Processor -- start 2023-10-06T21:10:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 -- start start 2023-10-06T21:10:43.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 0x7f6e7419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e74107f90 0x7f6e7419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 0x7f6e741a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e741a25b0 con 0x7f6e74107f90 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e741a2720 con 0x7f6e7410d570 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e791ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e741a28e0 con 0x7f6e741075d0 2023-10-06T21:10:43.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 0x7f6e7419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 0x7f6e7419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45504/0 (socket says 172.21.15.26:45504) 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.527+0000 7f6e72d9d700 1 -- 172.21.15.26:0/3564990039 learned_addr learned my addr 172.21.15.26:0/3564990039 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 0x7f6e741a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 -- 172.21.15.26:0/3564990039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 msgr2=0x7f6e7419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 0x7f6e7419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 -- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e74107f90 msgr2=0x7f6e7419db10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e74107f90 0x7f6e7419db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.528+0000 7f6e7359e700 1 -- 172.21.15.26:0/3564990039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009c40 con 0x7f6e7410d570 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.529+0000 7f6e7359e700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 0x7f6e741a1f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00efc0 tx=0x7f6e6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.529+0000 7f6e5bfff700 1 -- 172.21.15.26:0/3564990039 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c018070 con 0x7f6e7410d570 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.529+0000 7f6e5bfff700 1 -- 172.21.15.26:0/3564990039 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e6c00f040 con 0x7f6e7410d570 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.529+0000 7f6e5bfff700 1 -- 172.21.15.26:0/3564990039 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c004280 con 0x7f6e7410d570 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.529+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e64009520 con 0x7f6e7410d570 2023-10-06T21:10:43.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.530+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e741aa220 con 0x7f6e7410d570 2023-10-06T21:10:43.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.538+0000 7f6e5bfff700 1 -- 172.21.15.26:0/3564990039 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7f6e6c0043e0 con 0x7f6e7410d570 2023-10-06T21:10:43.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 Processor -- start 2023-10-06T21:10:43.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 -- start start 2023-10-06T21:10:43.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e840350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e842c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e79e460 con 0x55df9e840890 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e78fc20 con 0x55df9e8359e0 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e78d540 con 0x55df9e8350c0 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.540+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45512/0 (socket says 172.21.15.26:45512) 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 -- 172.21.15.26:0/1551322466 learned_addr learned my addr 172.21.15.26:0/1551322466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e842c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e840350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 -- 172.21.15.26:0/1551322466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 msgr2=0x55df9e840350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e840350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 -- 172.21.15.26:0/1551322466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 msgr2=0x55df9e842c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e842c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.541+0000 7f6e7259c700 1 -- 172.21.15.26:0/1551322466 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9e5b3de0 con 0x55df9e8350c0 2023-10-06T21:10:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.542+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e68013840 tx=0x7f6e680096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd5c72344b37d7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.542+0000 7f6e5bfff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 0x7f6e5c0a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.542+0000 7f6e5bfff700 1 -- 172.21.15.26:0/3564990039 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e6c0cda20 con 0x7f6e7410d570 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.544+0000 7f6e597fa700 1 -- 172.21.15.26:0/1551322466 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680132d0 con 0x55df9e8350c0 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.544+0000 7f6e597fa700 1 -- 172.21.15.26:0/1551322466 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e68019030 con 0x55df9e8350c0 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.544+0000 7f6e597fa700 1 -- 172.21.15.26:0/1551322466 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003d50 con 0x55df9e8350c0 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.544+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 0x7f6e5c0a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.545+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1551322466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 msgr2=0x55df9e8354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.545+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e68013840 tx=0x7f6e680096e0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.546+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 0x7f6e5c0a3d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c010 tx=0x7f6e64019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1551322466 shutdown_connections 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e842c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e8354a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1551322466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e840350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1551322466 >> 172.21.15.26:0/1551322466 conn(0x55df9e7aa260 msgr2=0x55df9e831450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1551322466 shutdown_connections 2023-10-06T21:10:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.547+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1551322466 wait complete. 2023-10-06T21:10:43.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.548+0000 7f6e8cbc1b80 1 -- start start 2023-10-06T21:10:43.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 Processor -- start 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e840330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e83d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e83dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e788e30 con 0x55df9e840890 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e725710 con 0x55df9e8359e0 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.549+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e7253b0 con 0x55df9e8350c0 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e8cbc1b80 10 client.0 _ll_register_callbacks cb 0x55df9e582800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e83d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e840330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e7259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e83d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53650/0 (socket says 172.21.15.26:53650) 2023-10-06T21:10:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e7259c700 1 -- 172.21.15.26:0/417192202 learned_addr learned my addr 172.21.15.26:0/417192202 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e7359e700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e83dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 -- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 msgr2=0x55df9e83d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e83d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 -- 172.21.15.26:0/417192202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 msgr2=0x55df9e83dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e83dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.550+0000 7f6e72d9d700 1 -- 172.21.15.26:0/417192202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e68013520 con 0x55df9e8350c0 2023-10-06T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.551+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e840330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098f0 tx=0x7f6e64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c84ca0051c1569c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.551+0000 7f6e56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.551+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64005620 con 0x55df9e8350c0 2023-10-06T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.551+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e64003680 con 0x55df9e8350c0 2023-10-06T21:10:43.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.552+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df9e828f10 con 0x55df9e8350c0 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e64003840 con 0x55df9e8350c0 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 10 client.25540 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55df9e59cc60 con 0x55df9e8350c0 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df9e828f10 con 0x55df9e8350c0 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e64037070 con 0x55df9e8350c0 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 10 client.25540 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 20 client.25540 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8611a0 0x55df9e8635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 4 client.25540 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.553+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df9e781e10 con 0x55df9e8611a0 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.554+0000 7f6e7259c700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8611a0 0x55df9e8635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.554+0000 7f6e7259c700 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8611a0 0x55df9e8635c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0fdbcc340f401e0 server_cookie=44cc95d213cf11df in_seq=0 out_seq=0 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.554+0000 7f6e56ff5700 10 client.25540 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e56ff5700 1 -- 172.21.15.26:0/417192202 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df9e781e10 con 0x55df9e8611a0 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e56ff5700 10 client.25540 handle_command_reply: tid=0 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 client.25540 shutdown 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 msgr2=0x55df9e840330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e840330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098f0 tx=0x7f6e64009c30 comp rx=0 tx=0).stop 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 shutdown_connections 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e840890 0x55df9e83dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.555+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e8350c0 0x55df9e840330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8611a0 0x55df9e8635c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/417192202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8359e0 0x55df9e83d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 >> 172.21.15.26:0/417192202 conn(0x55df9e7aa260 msgr2=0x55df9e830fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e56ff5700 0 client.25540 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 shutdown_connections 2023-10-06T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.556+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/417192202 wait complete. 2023-10-06T21:10:43.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.557+0000 7f6e8cbc1b80 20 client.25540 trim_cache size 0 max 16384 2023-10-06T21:10:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.591+0000 7f6e8cbc1b80 1 Processor -- start 2023-10-06T21:10:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 -- start start 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e8473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e85d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e84bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e609920 con 0x55df9e8359e0 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e836b80 con 0x55df9e8350c0 2023-10-06T21:10:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.592+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e847920 con 0x55df9e840890 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e8473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e8473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53658/0 (socket says 172.21.15.26:53658) 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e72d9d700 1 -- 172.21.15.26:0/1499490652 learned_addr learned my addr 172.21.15.26:0/1499490652 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e84bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 -- 172.21.15.26:0/1499490652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 msgr2=0x55df9e8473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e8473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 -- 172.21.15.26:0/1499490652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 msgr2=0x55df9e85d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e85d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.593+0000 7f6e7359e700 1 -- 172.21.15.26:0/1499490652 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9e5da250 con 0x55df9e840890 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.594+0000 7f6e7359e700 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e84bb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cb060 tx=0x7f6e6c09a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9304a95a6204a602 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.594+0000 7f6e59ffb700 1 -- 172.21.15.26:0/1499490652 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c09a8d0 con 0x55df9e840890 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.594+0000 7f6e59ffb700 1 -- 172.21.15.26:0/1499490652 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e6c09ea20 con 0x55df9e840890 2023-10-06T21:10:43.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.594+0000 7f6e59ffb700 1 -- 172.21.15.26:0/1499490652 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0048f0 con 0x55df9e840890 2023-10-06T21:10:43.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1499490652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 msgr2=0x55df9e84bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e84bb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0cb060 tx=0x7f6e6c09a4e0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1499490652 shutdown_connections 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e84bb30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e8473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1499490652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e85d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1499490652 >> 172.21.15.26:0/1499490652 conn(0x55df9e7aa260 msgr2=0x55df9e82fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.595+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1499490652 shutdown_connections 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.596+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1499490652 wait complete. 2023-10-06T21:10:43.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.596+0000 7f6e8cbc1b80 1 -- start start 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.596+0000 7f6e8cbc1b80 1 Processor -- start 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e8470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e8453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e845a10 con 0x55df9e8359e0 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e845b80 con 0x55df9e8350c0 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9e845d40 con 0x55df9e840890 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e8cbc1b80 10 client.0 _ll_register_callbacks cb 0x7f6e6400b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.597+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53662/0 (socket says 172.21.15.26:53662) 2023-10-06T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 -- 172.21.15.26:0/1573399206 learned_addr learned my addr 172.21.15.26:0/1573399206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:43.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e7359e700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e8453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 -- 172.21.15.26:0/1573399206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 msgr2=0x55df9e8453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e8453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 -- 172.21.15.26:0/1573399206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 msgr2=0x55df9e8470a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e8470a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 -- 172.21.15.26:0/1573399206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e741aa220 con 0x55df9e8350c0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e72d9d700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098f0 tx=0x7f6e64023720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac6478593ec56fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.598+0000 7f6e56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.599+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003a90 con 0x55df9e8350c0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.599+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e68019030 con 0x55df9e8350c0 2023-10-06T21:10:43.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.599+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e640056e0 con 0x55df9e8350c0 2023-10-06T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.599+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e64023e10 con 0x55df9e8350c0 2023-10-06T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.601+0000 7f6e8cbc1b80 10 client.25574 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:43.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.601+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55df9e5b22b0 con 0x55df9e8350c0 2023-10-06T21:10:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.601+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e6404f020 con 0x55df9e8350c0 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.602+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6e64037030 con 0x55df9e8350c0 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 20 client.25574 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8c6be0 0x55df9e8c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 4 client.25574 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:10:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.603+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df9e781b30 con 0x55df9e8c6be0 2023-10-06T21:10:43.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.604+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8c6be0 0x55df9e8c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.605+0000 7f6e56ff5700 10 client.25574 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.605+0000 7f6e7259c700 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8c6be0 0x55df9e8c8fd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3296e805d892c25 server_cookie=d49d9a729b9aa72d in_seq=0 out_seq=0 2023-10-06T21:10:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e56ff5700 1 -- 172.21.15.26:0/1573399206 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55df9e781b30 con 0x55df9e8c6be0 2023-10-06T21:10:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e56ff5700 10 client.25574 handle_command_reply: tid=0 2023-10-06T21:10:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 client.25574 shutdown 2023-10-06T21:10:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 msgr2=0x55df9e846b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e640098f0 tx=0x7f6e64023720 comp rx=0 tx=0).stop 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 shutdown_connections 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df9e840890 0x55df9e8453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df9e8c6be0 0x55df9e8c8fd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df9e8350c0 0x55df9e846b60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 --2- 172.21.15.26:0/1573399206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df9e8359e0 0x55df9e8470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 >> 172.21.15.26:0/1573399206 conn(0x55df9e7aa260 msgr2=0x55df9e8631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e56ff5700 0 client.25574 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 shutdown_connections 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.606+0000 7f6e8cbc1b80 1 -- 172.21.15.26:0/1573399206 wait complete. 2023-10-06T21:10:43.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e8cbc1b80 20 client.25574 trim_cache size 0 max 16384 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "3b1d5b86-9c30-4bb8-a408-7612a48acc29": { 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b1d5b86-9c30-4bb8-a408-7612a48acc29", 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:10:43.611 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 msgr2=0x7f6e5c0a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 0x7f6e5c0a3d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c010 tx=0x7f6e64019040 comp rx=0 tx=0).stop 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 msgr2=0x7f6e741a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 0x7f6e741a1f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00efc0 tx=0x7f6e6c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.607+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 shutdown_connections 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.608+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e7410d570 0x7f6e741a1f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.608+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e741075d0 0x7f6e7419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.608+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e5c0a1850 0x7f6e5c0a3d10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.608+0000 7f6e791ff700 1 --2- 172.21.15.26:0/3564990039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e74107f90 0x7f6e7419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.608+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 >> 172.21.15.26:0/3564990039 conn(0x7f6e74076d50 msgr2=0x7f6e74077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.610+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 shutdown_connections 2023-10-06T21:10:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:10:43.610+0000 7f6e791ff700 1 -- 172.21.15.26:0/3564990039 wait complete. 2023-10-06T21:10:43.643 INFO:tasks.cephfs.filesystem:scrub status for tag:3b1d5b86-9c30-4bb8-a408-7612a48acc29 - {'path': '/', 'tag': '3b1d5b86-9c30-4bb8-a408-7612a48acc29', 'options': 'recursive,force'} 2023-10-06T21:10:44.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:43 smithi026 ceph-mon[98490]: pgmap v15: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 MiB/s wr, 222 op/s 2023-10-06T21:10:44.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:43 smithi167 ceph-mon[134494]: pgmap v15: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 MiB/s wr, 222 op/s 2023-10-06T21:10:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:43 smithi078 ceph-mon[134896]: pgmap v15: 97 pgs: 97 active+clean; 376 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 MiB/s wr, 222 op/s 2023-10-06T21:10:45.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:44 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:10:45.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:44 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:45.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:44 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:10:45.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:44 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:45.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:44 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:10:45.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:44 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:45.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:44 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:10:45.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:44 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:45.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:44 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:10:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:44 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:44 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:10:45.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:44 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:10:46.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:45 smithi167 ceph-mon[134494]: pgmap v16: 97 pgs: 97 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 60 MiB/s wr, 238 op/s 2023-10-06T21:10:46.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:45 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:10:46.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:45 smithi026 ceph-mon[98490]: pgmap v16: 97 pgs: 97 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 60 MiB/s wr, 238 op/s 2023-10-06T21:10:46.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:45 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:10:46.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:45 smithi078 ceph-mon[134896]: pgmap v16: 97 pgs: 97 active+clean; 426 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 60 MiB/s wr, 238 op/s 2023-10-06T21:10:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:45 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:10:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 5.0 s, 2208.0 tps, lat 43.519 ms stddev 57.511 2023-10-06T21:10:48.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:48 smithi078 ceph-mon[134896]: pgmap v17: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 84 MiB/s wr, 457 op/s 2023-10-06T21:10:48.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:48 smithi167 ceph-mon[134494]: pgmap v17: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 84 MiB/s wr, 457 op/s 2023-10-06T21:10:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 -- 172.21.15.26:0/235249244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 msgr2=0x7f5834102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 --2- 172.21.15.26:0/235249244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834102440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58240098f0 tx=0x7f5824009c00 comp rx=0 tx=0).stop 2023-10-06T21:10:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 -- 172.21.15.26:0/235249244 shutdown_connections 2023-10-06T21:10:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 --2- 172.21.15.26:0/235249244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583410b480 0x7f583410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 --2- 172.21.15.26:0/235249244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834102440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 --2- 172.21.15.26:0/235249244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5834102980 0x7f583410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 -- 172.21.15.26:0/235249244 >> 172.21.15.26:0/235249244 conn(0x7f58340fb860 msgr2=0x7f58340fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.556+0000 7f583bdff700 1 -- 172.21.15.26:0/235249244 shutdown_connections 2023-10-06T21:10:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.557+0000 7f583bdff700 1 -- 172.21.15.26:0/235249244 wait complete. 2023-10-06T21:10:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.558+0000 7f583bdff700 1 Processor -- start 2023-10-06T21:10:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.558+0000 7f583bdff700 1 -- start start 2023-10-06T21:10:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5834102980 0x7f583410a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 0x7f58341a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a2870 con 0x7f5834102980 2023-10-06T21:10:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a29e0 con 0x7f5834102060 2023-10-06T21:10:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583bdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341a2ba0 con 0x7f583410b480 2023-10-06T21:10:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53684/0 (socket says 172.21.15.26:53684) 2023-10-06T21:10:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f5839b9b700 1 -- 172.21.15.26:0/2642013657 learned_addr learned my addr 172.21.15.26:0/2642013657 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.559+0000 7f583a39c700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 0x7f58341a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 -- 172.21.15.26:0/2642013657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 msgr2=0x7f58341a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 0x7f58341a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 -- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5834102980 msgr2=0x7f583410a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5834102980 0x7f583410a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 -- 172.21.15.26:0/2642013657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830009d00 con 0x7f5834102060 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f583a39c700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 0x7f58341a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:10:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.560+0000 7f5839b9b700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5824009810 tx=0x7f582400b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.564+0000 7f582affd700 1 -- 172.21.15.26:0/2642013657 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582401c070 con 0x7f5834102060 2023-10-06T21:10:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.565+0000 7f582affd700 1 -- 172.21.15.26:0/2642013657 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5824005040 con 0x7f5834102060 2023-10-06T21:10:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.565+0000 7f582affd700 1 -- 172.21.15.26:0/2642013657 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824004a40 con 0x7f5834102060 2023-10-06T21:10:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.565+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5824009520 con 0x7f5834102060 2023-10-06T21:10:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.565+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58341aa4c0 con 0x7f5834102060 2023-10-06T21:10:48.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:48 smithi026 ceph-mon[98490]: pgmap v17: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 84 MiB/s wr, 457 op/s 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.571+0000 7f584f81ab80 1 Processor -- start 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 -- start start 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a88dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a898e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a89b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a7f7400 con 0x55f64a88e520 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a7e8bc0 con 0x55f64a8993d0 2023-10-06T21:10:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.572+0000 7f584f81ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a7e64e0 con 0x55f64a88dc00 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583a39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a88dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583a39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a88dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45554/0 (socket says 172.21.15.26:45554) 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583a39c700 1 -- 172.21.15.26:0/2306214239 learned_addr learned my addr 172.21.15.26:0/2306214239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a89b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 -- 172.21.15.26:0/2306214239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 msgr2=0x55f64a88dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a88dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 -- 172.21.15.26:0/2306214239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 msgr2=0x55f64a898e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a898e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.573+0000 7f583939a700 1 -- 172.21.15.26:0/2306214239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f64a60cd80 con 0x55f64a8993d0 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.574+0000 7f583939a700 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a89b7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f583410b370 tx=0x7f583000e820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=591f424041f18a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.574+0000 7f581ffff700 1 -- 172.21.15.26:0/2306214239 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000e050 con 0x55f64a8993d0 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.574+0000 7f581ffff700 1 -- 172.21.15.26:0/2306214239 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583000f040 con 0x55f64a8993d0 2023-10-06T21:10:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.574+0000 7f581ffff700 1 -- 172.21.15.26:0/2306214239 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830013780 con 0x55f64a8993d0 2023-10-06T21:10:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.575+0000 7f582affd700 1 -- 172.21.15.26:0/2642013657 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) v1 ==== 136179+0+0 (secure 0 0 0) 0x7f5824020070 con 0x7f5834102060 2023-10-06T21:10:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.575+0000 7f584f81ab80 1 -- 172.21.15.26:0/2306214239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 msgr2=0x55f64a89b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.575+0000 7f584f81ab80 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a89b7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f583410b370 tx=0x7f583000e820 comp rx=0 tx=0).stop 2023-10-06T21:10:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.577+0000 7f582affd700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 0x7f58200a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.578+0000 7f582affd700 1 -- 172.21.15.26:0/2642013657 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58240ce0c0 con 0x7f5834102060 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.579+0000 7f584f81ab80 1 -- 172.21.15.26:0/2306214239 shutdown_connections 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.579+0000 7f584f81ab80 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a89b7c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.579+0000 7f584f81ab80 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a88dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.579+0000 7f584f81ab80 1 --2- 172.21.15.26:0/2306214239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a898e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.579+0000 7f584f81ab80 1 -- 172.21.15.26:0/2306214239 >> 172.21.15.26:0/2306214239 conn(0x55f64a803200 msgr2=0x55f64a88a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.581+0000 7f584f81ab80 1 -- 172.21.15.26:0/2306214239 shutdown_connections 2023-10-06T21:10:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.581+0000 7f583a39c700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 0x7f58200a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.581+0000 7f584f81ab80 1 -- 172.21.15.26:0/2306214239 wait complete. 2023-10-06T21:10:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.582+0000 7f584f81ab80 1 -- start start 2023-10-06T21:10:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.582+0000 7f583a39c700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 0x7f58200a3d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f582c009440 tx=0x7f582c009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 Processor -- start 2023-10-06T21:10:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88e520 0x55f64a896690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a896bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a7e1dd0 con 0x55f64a88dc00 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a77e6b0 con 0x55f64a8993d0 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.583+0000 7f584f81ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a77e350 con 0x55f64a88e520 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f584f81ab80 10 client.0 _ll_register_callbacks cb 0x55f64a5db7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40208/0 (socket says 172.21.15.26:40208) 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 -- 172.21.15.26:0/1811514981 learned_addr learned my addr 172.21.15.26:0/1811514981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f583a39c700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a896bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f583939a700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88e520 0x55f64a896690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 -- 172.21.15.26:0/1811514981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88e520 msgr2=0x55f64a896690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88e520 0x55f64a896690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 -- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 msgr2=0x55f64a896bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a896bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.584+0000 7f5839b9b700 1 -- 172.21.15.26:0/1811514981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583000e220 con 0x55f64a88dc00 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.585+0000 7f5839b9b700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f582400bf60 tx=0x7f582409a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d153d43867d7a90a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.585+0000 7f581d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.585+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582409f0c0 con 0x55f64a88dc00 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.585+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58240a3270 con 0x55f64a88dc00 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.585+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f64a881d50 con 0x55f64a88dc00 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.586+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58240051b0 con 0x55f64a88dc00 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.586+0000 7f584f81ab80 10 client.15492 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.586+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f64a5f5c00 con 0x55f64a88dc00 2023-10-06T21:10:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.586+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58240b7030 con 0x55f64a88dc00 2023-10-06T21:10:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f582409a460 con 0x55f64a88dc00 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 10 client.15492 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 20 client.15492 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a8a8b90 0x55f64a8aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 4 client.15492 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f64a7dadb0 con 0x55f64a8a8b90 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.587+0000 7f583939a700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a8a8b90 0x55f64a8aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.588+0000 7f583939a700 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a8a8b90 0x55f64a8aaf80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5eb013937913cdb server_cookie=79740591fa651707 in_seq=0 out_seq=0 2023-10-06T21:10:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.588+0000 7f581d7fa700 10 client.15492 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.588+0000 7f581d7fa700 1 -- 172.21.15.26:0/1811514981 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f64a7dadb0 con 0x55f64a8a8b90 2023-10-06T21:10:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.588+0000 7f581d7fa700 10 client.15492 handle_command_reply: tid=0 2023-10-06T21:10:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.588+0000 7f584f81ab80 1 client.15492 shutdown 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 msgr2=0x55f64a896150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f582400bf60 tx=0x7f582409a9c0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 shutdown_connections 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a8a8b90 0x55f64a8aaf80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a8993d0 0x55f64a896bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88dc00 0x55f64a896150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 --2- 172.21.15.26:0/1811514981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88e520 0x55f64a896690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 >> 172.21.15.26:0/1811514981 conn(0x55f64a803200 msgr2=0x55f64a889c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f581d7fa700 0 client.15492 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 shutdown_connections 2023-10-06T21:10:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.589+0000 7f584f81ab80 1 -- 172.21.15.26:0/1811514981 wait complete. 2023-10-06T21:10:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.590+0000 7f584f81ab80 20 client.15492 trim_cache size 0 max 16384 2023-10-06T21:10:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.614+0000 7f584f81ab80 1 Processor -- start 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.614+0000 7f584f81ab80 1 -- start start 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a896a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a8993d0 0x55f64a889b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a6628c0 con 0x55f64a88e520 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a8818c0 con 0x55f64a88dc00 2023-10-06T21:10:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.615+0000 7f584f81ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a88e230 con 0x55f64a8993d0 2023-10-06T21:10:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.616+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.616+0000 7f5839b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53714/0 (socket says 172.21.15.26:53714) 2023-10-06T21:10:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.616+0000 7f5839b9b700 1 -- 172.21.15.26:0/3463501418 learned_addr learned my addr 172.21.15.26:0/3463501418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.616+0000 7f583939a700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a896a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.616+0000 7f583a39c700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a8993d0 0x55f64a889b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 -- 172.21.15.26:0/3463501418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a8993d0 msgr2=0x55f64a889b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a8993d0 0x55f64a889b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 -- 172.21.15.26:0/3463501418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 msgr2=0x55f64a896a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a896a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 -- 172.21.15.26:0/3463501418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f64a6331f0 con 0x55f64a88dc00 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f583939a700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a896a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:10:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5824097990 tx=0x7f582409b0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d80bb2d47007eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.617+0000 7f5828bff700 1 -- 172.21.15.26:0/3463501418 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240972b0 con 0x55f64a88dc00 2023-10-06T21:10:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.620+0000 7f5828bff700 1 -- 172.21.15.26:0/3463501418 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58240b7070 con 0x55f64a88dc00 2023-10-06T21:10:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.620+0000 7f5828bff700 1 -- 172.21.15.26:0/3463501418 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582409f0c0 con 0x55f64a88dc00 2023-10-06T21:10:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.621+0000 7f584f81ab80 1 -- 172.21.15.26:0/3463501418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 msgr2=0x55f64a890aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.621+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5824097990 tx=0x7f582409b0f0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.622+0000 7f584f81ab80 1 -- 172.21.15.26:0/3463501418 shutdown_connections 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.622+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a8993d0 0x55f64a889b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.622+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88dc00 0x55f64a890aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.622+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3463501418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a88e520 0x55f64a896a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.622+0000 7f584f81ab80 1 -- 172.21.15.26:0/3463501418 >> 172.21.15.26:0/3463501418 conn(0x55f64a803200 msgr2=0x55f64a8914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.623+0000 7f584f81ab80 1 -- 172.21.15.26:0/3463501418 shutdown_connections 2023-10-06T21:10:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.623+0000 7f584f81ab80 1 -- 172.21.15.26:0/3463501418 wait complete. 2023-10-06T21:10:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.624+0000 7f584f81ab80 1 -- start start 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.625+0000 7f584f81ab80 1 Processor -- start 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.625+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a8f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88e520 0x55f64a8f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 0x55f64a89f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a8f8720 con 0x55f64a8993d0 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a89f620 con 0x55f64a88e520 2023-10-06T21:10:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64a89f7b0 con 0x55f64a88dc00 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f584f81ab80 10 client.0 _ll_register_callbacks cb 0x7f582409aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f583a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 0x55f64a89f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f583a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 0x55f64a89f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40224/0 (socket says 172.21.15.26:40224) 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f583a39c700 1 -- 172.21.15.26:0/3444624959 learned_addr learned my addr 172.21.15.26:0/3444624959 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.626+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a8f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 -- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88e520 msgr2=0x55f64a8f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88e520 0x55f64a8f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 -- 172.21.15.26:0/3444624959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 msgr2=0x55f64a89f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 0x55f64a89f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 -- 172.21.15.26:0/3444624959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f64a60cd80 con 0x55f64a88dc00 2023-10-06T21:10:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.627+0000 7f5839b9b700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a8f78d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5824097990 tx=0x7f58240053e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c09fba636cf441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:10:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.628+0000 7f581d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:10:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.628+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240a3270 con 0x55f64a88dc00 2023-10-06T21:10:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.628+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f583000f040 con 0x55f64a88dc00 2023-10-06T21:10:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.628+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f58240a2e30 con 0x55f64a88dc00 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.628+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58240cb640 con 0x55f64a88dc00 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.630+0000 7f584f81ab80 10 client.25561 fetch_fsmap learned FSMap version 23 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.630+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f64a5f5c00 con 0x55f64a88dc00 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.631+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583000f040 con 0x55f64a88dc00 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.631+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f58240a9070 con 0x55f64a88dc00 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 10 client.25561 fetch_fsmap finished waiting for FSMap version 23 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 20 client.25561 populate_metadata read hostname 'smithi026' 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a90e180 0x55f64a910570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:10:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 4 client.25561 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:10:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.632+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f64a7daad0 con 0x55f64a90e180 2023-10-06T21:10:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.635+0000 7f583939a700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a90e180 0x55f64a910570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:10:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.640+0000 7f583939a700 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a90e180 0x55f64a910570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16693488504b6199 server_cookie=b0054ced17e84540 in_seq=0 out_seq=0 2023-10-06T21:10:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.640+0000 7f581d7fa700 10 client.25561 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.670+0000 7f581d7fa700 1 -- 172.21.15.26:0/3444624959 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f64a7daad0 con 0x55f64a90e180 2023-10-06T21:10:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.670+0000 7f581d7fa700 10 client.25561 handle_command_reply: tid=0 2023-10-06T21:10:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.670+0000 7f584f81ab80 1 client.25561 shutdown 2023-10-06T21:10:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.670+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 msgr2=0x55f64a8f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.670+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a8f78d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5824097990 tx=0x7f58240053e0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 shutdown_connections 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f64a8993d0 0x55f64a89f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f64a88dc00 0x55f64a8f78d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f64a90e180 0x55f64a910570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 --2- 172.21.15.26:0/3444624959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f64a88e520 0x55f64a8f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 >> 172.21.15.26:0/3444624959 conn(0x55f64a803200 msgr2=0x55f64a889ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f581d7fa700 0 client.25561 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:10:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 shutdown_connections 2023-10-06T21:10:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.671+0000 7f584f81ab80 1 -- 172.21.15.26:0/3444624959 wait complete. 2023-10-06T21:10:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f584f81ab80 20 client.25561 trim_cache size 0 max 16384 2023-10-06T21:10:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:10:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:10:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:10:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:10:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 msgr2=0x7f58200a3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 0x7f58200a3d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f582c009440 tx=0x7f582c009260 comp rx=0 tx=0).stop 2023-10-06T21:10:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 msgr2=0x7f5834109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:10:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5824009810 tx=0x7f582400b560 comp rx=0 tx=0).stop 2023-10-06T21:10:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.673+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 shutdown_connections 2023-10-06T21:10:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f583410b480 0x7f58341a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5834102060 0x7f5834109cf0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5824009810 tx=0x7f582400b560 comp rx=0 tx=0).stop 2023-10-06T21:10:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f58200a1880 0x7f58200a3d40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 --2- 172.21.15.26:0/2642013657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5834102980 0x7f583410a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:10:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 >> 172.21.15.26:0/2642013657 conn(0x7f58340fb860 msgr2=0x7f583410d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:10:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 shutdown_connections 2023-10-06T21:10:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:10:48.674+0000 7f583bdff700 1 -- 172.21.15.26:0/2642013657 wait complete. 2023-10-06T21:10:50.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:50 smithi078 ceph-mon[134896]: pgmap v18: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 52 MiB/s wr, 435 op/s 2023-10-06T21:10:50.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:50 smithi167 ceph-mon[134494]: pgmap v18: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 52 MiB/s wr, 435 op/s 2023-10-06T21:10:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:50 smithi026 ceph-mon[98490]: pgmap v18: 97 pgs: 97 active+clean; 937 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 52 MiB/s wr, 435 op/s 2023-10-06T21:10:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 10.0 s, 2292.0 tps, lat 42.256 ms stddev 56.496 2023-10-06T21:10:52.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:52 smithi167 ceph-mon[134494]: pgmap v19: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 738 op/s 2023-10-06T21:10:52.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:52 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:52.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:52 smithi026 ceph-mon[98490]: pgmap v19: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 738 op/s 2023-10-06T21:10:52.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:52 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:52 smithi078 ceph-mon[134896]: pgmap v19: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 738 op/s 2023-10-06T21:10:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:52 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:54.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:53 smithi167 ceph-mon[134494]: pgmap v20: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 49 MiB/s wr, 538 op/s 2023-10-06T21:10:54.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:53 smithi026 ceph-mon[98490]: pgmap v20: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 49 MiB/s wr, 538 op/s 2023-10-06T21:10:54.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:53 smithi078 ceph-mon[134896]: pgmap v20: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 49 MiB/s wr, 538 op/s 2023-10-06T21:10:56.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:56 smithi078 ceph-mon[134896]: pgmap v21: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 55 MiB/s wr, 615 op/s 2023-10-06T21:10:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:56 smithi167 ceph-mon[134494]: pgmap v21: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 55 MiB/s wr, 615 op/s 2023-10-06T21:10:56.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:56 smithi026 ceph-mon[98490]: pgmap v21: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 55 MiB/s wr, 615 op/s 2023-10-06T21:10:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 15.0 s, 2573.6 tps, lat 40.169 ms stddev 69.716 2023-10-06T21:10:57.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:57 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:57.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:57 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:57.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:57 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:10:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:10:58 smithi167 ceph-mon[134494]: pgmap v22: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 91 MiB/s wr, 1.13k op/s 2023-10-06T21:10:58.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:10:58 smithi026 ceph-mon[98490]: pgmap v22: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 91 MiB/s wr, 1.13k op/s 2023-10-06T21:10:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:10:58 smithi078 ceph-mon[134896]: pgmap v22: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 91 MiB/s wr, 1.13k op/s 2023-10-06T21:11:00.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:00 smithi167 ceph-mon[134494]: pgmap v23: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 67 MiB/s wr, 911 op/s 2023-10-06T21:11:00.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-mon[98490]: pgmap v23: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 67 MiB/s wr, 911 op/s 2023-10-06T21:11:00.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:00 smithi078 ceph-mon[134896]: pgmap v23: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 67 MiB/s wr, 911 op/s 2023-10-06T21:11:01.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:11:00.720+0000 7fc93005c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:11:01.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:11:00.720+0000 7fc93005c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:11:01.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:11:00.720+0000 7fc93005c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:11:01.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:11:00.720+0000 7fc93005c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:11:01.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:11:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:11:00.720+0000 7fc93005c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T21:11:01.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:01 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:01.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:01 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:01.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:01 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 -- 172.21.15.26:0/1678391423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 msgr2=0x7f992411ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 --2- 172.21.15.26:0/1678391423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f992411ca80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f991c009f90 tx=0x7f991c00b330 comp rx=0 tx=0).stop 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 -- 172.21.15.26:0/1678391423 shutdown_connections 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 --2- 172.21.15.26:0/1678391423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f992411ca80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 --2- 172.21.15.26:0/1678391423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410f620 0x7f9924107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 --2- 172.21.15.26:0/1678391423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992410fa90 0x7f99241083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.009+0000 7f99295ff700 1 -- 172.21.15.26:0/1678391423 >> 172.21.15.26:0/1678391423 conn(0x7f992406cfa0 msgr2=0x7f992406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.013+0000 7f99295ff700 1 -- 172.21.15.26:0/1678391423 shutdown_connections 2023-10-06T21:11:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.016+0000 7f99295ff700 1 -- 172.21.15.26:0/1678391423 wait complete. 2023-10-06T21:11:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.018+0000 7f99295ff700 1 Processor -- start 2023-10-06T21:11:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.019+0000 7f99295ff700 1 -- start start 2023-10-06T21:11:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992410f620 0x7f99241a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f99241aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99241ab170 con 0x7f992410f620 2023-10-06T21:11:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99241ab2e0 con 0x7f992411a690 2023-10-06T21:11:02.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.020+0000 7f99295ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99241ab4a0 con 0x7f992410fa90 2023-10-06T21:11:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.023+0000 7f99227fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.023+0000 7f99237fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f99241aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.023+0000 7f99227fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:40286/0 (socket says 172.21.15.26:40286) 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.023+0000 7f99227fc700 1 -- 172.21.15.26:0/835891015 learned_addr learned my addr 172.21.15.26:0/835891015 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 -- 172.21.15.26:0/835891015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 msgr2=0x7f99241aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f99241aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 -- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992410f620 msgr2=0x7f99241a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992410f620 0x7f99241a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 -- 172.21.15.26:0/835891015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c009c40 con 0x7f992410fa90 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.024+0000 7f99227fc700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f991800b530 tx=0x7f991800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.025+0000 7f990bfff700 1 -- 172.21.15.26:0/835891015 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918011830 con 0x7f992410fa90 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.025+0000 7f990bfff700 1 -- 172.21.15.26:0/835891015 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9918011e50 con 0x7f992410fa90 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.025+0000 7f990bfff700 1 -- 172.21.15.26:0/835891015 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800f2e0 con 0x7f992410fa90 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.025+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99241ab780 con 0x7f992410fa90 2023-10-06T21:11:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.025+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99241b2fc0 con 0x7f992410fa90 2023-10-06T21:11:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.033+0000 7f993cf00b80 1 Processor -- start 2023-10-06T21:11:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 -- start start 2023-10-06T21:11:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631646520 0x558631650e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586316513d0 0x5586316537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586315af400 con 0x558631646520 2023-10-06T21:11:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586315a0bc0 con 0x5586316513d0 2023-10-06T21:11:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.034+0000 7f993cf00b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863159e4e0 con 0x558631645c00 2023-10-06T21:11:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.037+0000 7f990bfff700 1 -- 172.21.15.26:0/835891015 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f9918011990 con 0x7f992410fa90 2023-10-06T21:11:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.040+0000 7f99237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.040+0000 7f99237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:40290/0 (socket says 172.21.15.26:40290) 2023-10-06T21:11:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.040+0000 7f99237fe700 1 -- 172.21.15.26:0/2643294556 learned_addr learned my addr 172.21.15.26:0/2643294556 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.040+0000 7f990bfff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 0x7f990c0a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.040+0000 7f990bfff700 1 -- 172.21.15.26:0/835891015 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99180cc0a0 con 0x7f992410fa90 2023-10-06T21:11:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99227fc700 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586316513d0 0x5586316537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99237fe700 1 -- 172.21.15.26:0/2643294556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586316513d0 msgr2=0x5586316537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99237fe700 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586316513d0 0x5586316537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99237fe700 1 -- 172.21.15.26:0/2643294556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631646520 msgr2=0x558631650e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99237fe700 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631646520 0x558631650e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.041+0000 7f99237fe700 1 -- 172.21.15.26:0/2643294556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586313c4d80 con 0x558631645c00 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.042+0000 7f99237fe700 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f991c009360 tx=0x7f991c009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2deec33a75155a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.042+0000 7f99237fe700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 0x7f990c0a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.043+0000 7f99097fa700 1 -- 172.21.15.26:0/2643294556 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00e040 con 0x558631645c00 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.043+0000 7f99097fa700 1 -- 172.21.15.26:0/2643294556 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f991c007d90 con 0x558631645c00 2023-10-06T21:11:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.043+0000 7f99097fa700 1 -- 172.21.15.26:0/2643294556 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c01d470 con 0x558631645c00 2023-10-06T21:11:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 -- 172.21.15.26:0/2643294556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 msgr2=0x558631645fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f991c009360 tx=0x7f991c009390 comp rx=0 tx=0).stop 2023-10-06T21:11:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 -- 172.21.15.26:0/2643294556 shutdown_connections 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586316513d0 0x5586316537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x558631645fe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2643294556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631646520 0x558631650e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.044+0000 7f993cf00b80 1 -- 172.21.15.26:0/2643294556 >> 172.21.15.26:0/2643294556 conn(0x5586315bb200 msgr2=0x558631642080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.045+0000 7f993cf00b80 1 -- 172.21.15.26:0/2643294556 shutdown_connections 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.045+0000 7f993cf00b80 1 -- 172.21.15.26:0/2643294556 wait complete. 2023-10-06T21:11:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.045+0000 7f99237fe700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 0x7f990c0a3de0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f991c004fd0 tx=0x7f991c00bf10 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.046+0000 7f993cf00b80 1 -- start start 2023-10-06T21:11:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.046+0000 7f993cf00b80 1 Processor -- start 2023-10-06T21:11:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.047+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631645c00 0x558631650d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558631599dd0 con 0x5586316513d0 2023-10-06T21:11:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586315366b0 con 0x558631645c00 2023-10-06T21:11:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558631536350 con 0x558631646520 2023-10-06T21:11:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.048+0000 7f993cf00b80 10 client.0 _ll_register_callbacks cb 0x5586313937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:40304/0 (socket says 172.21.15.26:40304) 2023-10-06T21:11:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 -- 172.21.15.26:0/2285529888 learned_addr learned my addr 172.21.15.26:0/2285529888 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f9922ffd700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631645c00 0x558631650d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 -- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631645c00 msgr2=0x558631650d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631645c00 0x558631650d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 -- 172.21.15.26:0/2285529888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 msgr2=0x55863164e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.049+0000 7f99227fc700 1 -- 172.21.15.26:0/2285529888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992411d3b0 con 0x558631646520 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.050+0000 7f99227fc700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f991800bcf0 tx=0x7f9918099720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ee2bf9595abe49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.050+0000 7f9906ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180985e0 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99180a1280 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558631639d50 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99180a0df0 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f993cf00b80 10 client.25579 fetch_fsmap learned FSMap version 26 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5586313adc00 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.051+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558631639d50 con 0x558631646520 2023-10-06T21:11:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9918094ce0 con 0x558631646520 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 10 client.25579 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 20 client.25579 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558631671fc0 0x5586316743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 4 client.25579 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558631592db0 con 0x558631671fc0 2023-10-06T21:11:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.052+0000 7f9922ffd700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558631671fc0 0x5586316743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.053+0000 7f9922ffd700 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558631671fc0 0x5586316743e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28e49e3acf42eba server_cookie=7a2156d15d121294 in_seq=0 out_seq=0 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.053+0000 7f9906ff5700 10 client.25579 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.054+0000 7f9906ff5700 1 -- 172.21.15.26:0/2285529888 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558631592db0 con 0x558631671fc0 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.054+0000 7f9906ff5700 10 client.25579 handle_command_reply: tid=0 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.054+0000 7f993cf00b80 1 client.25579 shutdown 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.054+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 msgr2=0x55863164e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.054+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f991800bcf0 tx=0x7f9918099720 comp rx=0 tx=0).stop 2023-10-06T21:11:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 shutdown_connections 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558631671fc0 0x5586316743e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631645c00 0x558631650d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2285529888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631646520 0x55863164e060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f9906ff5700 0 client.25579 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.060+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 >> 172.21.15.26:0/2285529888 conn(0x5586315bb200 msgr2=0x558631641c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.061+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 shutdown_connections 2023-10-06T21:11:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.061+0000 7f993cf00b80 1 -- 172.21.15.26:0/2285529888 wait complete. 2023-10-06T21:11:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.061+0000 7f993cf00b80 20 client.25579 trim_cache size 0 max 16384 2023-10-06T21:11:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.092+0000 7f993cf00b80 1 Processor -- start 2023-10-06T21:11:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.092+0000 7f993cf00b80 1 -- start start 2023-10-06T21:11:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631645c00 0x558631657f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x558631650910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 0x55863164d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863141a8c0 con 0x558631645c00 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558631640e30 con 0x558631646520 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f993cf00b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586316473f0 con 0x5586316513d0 2023-10-06T21:11:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f99227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x558631650910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f99227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x558631650910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56660/0 (socket says 172.21.15.26:56660) 2023-10-06T21:11:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.093+0000 7f99227fc700 1 -- 172.21.15.26:0/304517995 learned_addr learned my addr 172.21.15.26:0/304517995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.094+0000 7f99237fe700 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 0x55863164d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.094+0000 7f9922ffd700 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631645c00 0x558631657f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.095+0000 7f99237fe700 1 -- 172.21.15.26:0/304517995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 msgr2=0x558631650910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.095+0000 7f99237fe700 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x558631650910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.095+0000 7f99237fe700 1 -- 172.21.15.26:0/304517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631645c00 msgr2=0x558631657f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.095+0000 7f99237fe700 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631645c00 0x558631657f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.095+0000 7f99237fe700 1 -- 172.21.15.26:0/304517995 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586313eb1f0 con 0x5586316513d0 2023-10-06T21:11:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.096+0000 7f99237fe700 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 0x55863164d820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f991c009360 tx=0x7f991c02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92464ad8c0a10f50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.097+0000 7f9909ffb700 1 -- 172.21.15.26:0/304517995 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00e070 con 0x5586316513d0 2023-10-06T21:11:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.097+0000 7f9909ffb700 1 -- 172.21.15.26:0/304517995 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f991c02c070 con 0x5586316513d0 2023-10-06T21:11:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.097+0000 7f9909ffb700 1 -- 172.21.15.26:0/304517995 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0073e0 con 0x5586316513d0 2023-10-06T21:11:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.099+0000 7f993cf00b80 1 -- 172.21.15.26:0/304517995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 msgr2=0x55863164d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.099+0000 7f993cf00b80 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 0x55863164d820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f991c009360 tx=0x7f991c02a000 comp rx=0 tx=0).stop 2023-10-06T21:11:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.101+0000 7f993cf00b80 1 -- 172.21.15.26:0/304517995 shutdown_connections 2023-10-06T21:11:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.101+0000 7f993cf00b80 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586316513d0 0x55863164d820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.101+0000 7f993cf00b80 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558631645c00 0x558631657f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.101+0000 7f993cf00b80 1 --2- 172.21.15.26:0/304517995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x558631650910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.101+0000 7f993cf00b80 1 -- 172.21.15.26:0/304517995 >> 172.21.15.26:0/304517995 conn(0x5586315bb200 msgr2=0x55863166e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.102+0000 7f993cf00b80 1 -- 172.21.15.26:0/304517995 shutdown_connections 2023-10-06T21:11:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.102+0000 7f993cf00b80 1 -- 172.21.15.26:0/304517995 wait complete. 2023-10-06T21:11:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.102+0000 7f993cf00b80 1 -- start start 2023-10-06T21:11:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.103+0000 7f993cf00b80 1 Processor -- start 2023-10-06T21:11:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x55863164b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863164bf20 con 0x5586316513d0 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863164c090 con 0x558631646520 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863164a820 con 0x558631645c00 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f993cf00b80 10 client.0 _ll_register_callbacks cb 0x7f9918098a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f99227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f99227fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56662/0 (socket says 172.21.15.26:56662) 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f99227fc700 1 -- 172.21.15.26:0/2036102060 learned_addr learned my addr 172.21.15.26:0/2036102060 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f9922ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x55863164b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f99227fc700 1 -- 172.21.15.26:0/2036102060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 msgr2=0x55863164b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.104+0000 7f99227fc700 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x55863164b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f99227fc700 1 -- 172.21.15.26:0/2036102060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 msgr2=0x55863164a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f99227fc700 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f99227fc700 1 -- 172.21.15.26:0/2036102060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586313c4d80 con 0x558631646520 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f99227fc700 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9918098e80 tx=0x7f991800bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727241f40cc8aa2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f9906ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180981d0 con 0x558631646520 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99180a1280 con 0x558631646520 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.105+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991c007d90 con 0x558631646520 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.106+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99180983e0 con 0x558631646520 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.106+0000 7f993cf00b80 10 client.25607 fetch_fsmap learned FSMap version 26 2023-10-06T21:11:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.106+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5586313c3250 con 0x558631646520 2023-10-06T21:11:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 20.0 s, 2441.0 tps, lat 36.606 ms stddev 45.086 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.123+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f991c007d90 con 0x558631646520 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.123+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9918005b60 con 0x558631646520 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 10 client.25607 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 20 client.25607 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55863165d240 0x55863165f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 4 client.25607 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:11:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.124+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558631592ad0 con 0x55863165d240 2023-10-06T21:11:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.135+0000 7f9922ffd700 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55863165d240 0x55863165f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.157+0000 7f9922ffd700 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55863165d240 0x55863165f660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89137402942c4a8 server_cookie=da6d4b63112ee428 in_seq=0 out_seq=0 2023-10-06T21:11:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.157+0000 7f9906ff5700 10 client.25607 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.205+0000 7f9906ff5700 1 -- 172.21.15.26:0/2036102060 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558631592ad0 con 0x55863165d240 2023-10-06T21:11:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.205+0000 7f9906ff5700 10 client.25607 handle_command_reply: tid=0 2023-10-06T21:11:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.206+0000 7f993cf00b80 1 client.25607 shutdown 2023-10-06T21:11:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.208+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 msgr2=0x55863164b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.208+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9918098e80 tx=0x7f991800bf50 comp rx=0 tx=0).stop 2023-10-06T21:11:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 shutdown_connections 2023-10-06T21:11:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5586316513d0 0x55863164a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558631645c00 0x55863164b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55863165d240 0x55863165f660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 --2- 172.21.15.26:0/2036102060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558631646520 0x55863164b910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.210+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 >> 172.21.15.26:0/2036102060 conn(0x5586315bb200 msgr2=0x558631641250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.211+0000 7f9906ff5700 0 client.25607 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.211+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 shutdown_connections 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.211+0000 7f993cf00b80 1 -- 172.21.15.26:0/2036102060 wait complete. 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.212+0000 7f993cf00b80 20 client.25607 trim_cache size 0 max 16384 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:11:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:11:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.219+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 msgr2=0x7f990c0a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.219+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 0x7f990c0a3de0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f991c004fd0 tx=0x7f991c00bf10 comp rx=0 tx=0).stop 2023-10-06T21:11:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.219+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 msgr2=0x7f99241a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.219+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f991800b530 tx=0x7f991800b8f0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 shutdown_connections 2023-10-06T21:11:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f992411a690 0x7f99241aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f990c0a1920 0x7f990c0a3de0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992410f620 0x7f99241a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 --2- 172.21.15.26:0/835891015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f992410fa90 0x7f99241a64c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 >> 172.21.15.26:0/835891015 conn(0x7f992406cfa0 msgr2=0x7f9924118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 shutdown_connections 2023-10-06T21:11:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:02.220+0000 7f99295ff700 1 -- 172.21.15.26:0/835891015 wait complete. 2023-10-06T21:11:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:02 smithi167 ceph-mon[134494]: pgmap v24: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 MiB/s wr, 1.23k op/s 2023-10-06T21:11:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:02 smithi167 ceph-mon[134494]: mgrmap e25: x(active, since 42s), standbys: z, y 2023-10-06T21:11:02.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:02 smithi026 ceph-mon[98490]: pgmap v24: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 MiB/s wr, 1.23k op/s 2023-10-06T21:11:02.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:02 smithi026 ceph-mon[98490]: mgrmap e25: x(active, since 42s), standbys: z, y 2023-10-06T21:11:02.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:02 smithi078 ceph-mon[134896]: pgmap v24: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 MiB/s wr, 1.23k op/s 2023-10-06T21:11:02.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:02 smithi078 ceph-mon[134896]: mgrmap e25: x(active, since 42s), standbys: z, y 2023-10-06T21:11:04.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:04 smithi167 ceph-mon[134494]: pgmap v25: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 MiB/s wr, 925 op/s 2023-10-06T21:11:04.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:04 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:04.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:04 smithi026 ceph-mon[98490]: pgmap v25: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 MiB/s wr, 925 op/s 2023-10-06T21:11:04.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:04 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:04 smithi078 ceph-mon[134896]: pgmap v25: 97 pgs: 97 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 MiB/s wr, 925 op/s 2023-10-06T21:11:04.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:04 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:06 smithi167 ceph-mon[134494]: pgmap v26: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 MiB/s wr, 1.43k op/s 2023-10-06T21:11:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:06 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:06 smithi026 ceph-mon[98490]: pgmap v26: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 MiB/s wr, 1.43k op/s 2023-10-06T21:11:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:06 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:06 smithi078 ceph-mon[134896]: pgmap v26: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 MiB/s wr, 1.43k op/s 2023-10-06T21:11:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:06 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 25.0 s, 2753.5 tps, lat 40.174 ms stddev 70.362 2023-10-06T21:11:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:08 smithi167 ceph-mon[134494]: pgmap v27: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 MiB/s wr, 2.45k op/s 2023-10-06T21:11:08.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:08 smithi026 ceph-mon[98490]: pgmap v27: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 MiB/s wr, 2.45k op/s 2023-10-06T21:11:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:08 smithi078 ceph-mon[134896]: pgmap v27: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 MiB/s wr, 2.45k op/s 2023-10-06T21:11:10.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:10 smithi167 ceph-mon[134494]: pgmap v28: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 36 MiB/s wr, 1.92k op/s 2023-10-06T21:11:10.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:10 smithi026 ceph-mon[98490]: pgmap v28: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 36 MiB/s wr, 1.92k op/s 2023-10-06T21:11:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:10 smithi078 ceph-mon[134896]: pgmap v28: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 36 MiB/s wr, 1.92k op/s 2023-10-06T21:11:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 30.0 s, 2813.5 tps, lat 35.545 ms stddev 44.451 2023-10-06T21:11:12.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:12 smithi167 ceph-mon[134494]: pgmap v29: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 46 MiB/s wr, 2.23k op/s 2023-10-06T21:11:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:12 smithi026 ceph-mon[98490]: pgmap v29: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 46 MiB/s wr, 2.23k op/s 2023-10-06T21:11:12.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:12 smithi078 ceph-mon[134896]: pgmap v29: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 46 MiB/s wr, 2.23k op/s 2023-10-06T21:11:13.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:11:14.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 -- 172.21.15.26:0/3754543525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e140ff530 msgr2=0x7f6e140ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/3754543525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e140ff530 0x7f6e140ff910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009f90 tx=0x7f6e0c00b330 comp rx=0 tx=0).stop 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 -- 172.21.15.26:0/3754543525 shutdown_connections 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/3754543525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e1410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/3754543525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e140ff530 0x7f6e140ff910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/3754543525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ffee0 0x7f6e14105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:13.999+0000 7f6e1bfff700 1 -- 172.21.15.26:0/3754543525 >> 172.21.15.26:0/3754543525 conn(0x7f6e14077330 msgr2=0x7f6e140fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.001+0000 7f6e1bfff700 1 -- 172.21.15.26:0/3754543525 shutdown_connections 2023-10-06T21:11:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.001+0000 7f6e1bfff700 1 -- 172.21.15.26:0/3754543525 wait complete. 2023-10-06T21:11:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.002+0000 7f6e1bfff700 1 Processor -- start 2023-10-06T21:11:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.003+0000 7f6e1bfff700 1 -- start start 2023-10-06T21:11:14.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 0x7f6e141a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e1410cd00 0x7f6e1410d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410d780 con 0x7f6e1410cd00 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410d8c0 con 0x7f6e140ff530 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1bfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410da80 con 0x7f6e14106230 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e1959a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.004+0000 7f6e19d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 0x7f6e141a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e1959a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54592/0 (socket says 172.21.15.26:54592) 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e1959a700 1 -- 172.21.15.26:0/792998843 learned_addr learned my addr 172.21.15.26:0/792998843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e1410cd00 0x7f6e1410d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e19d9b700 1 -- 172.21.15.26:0/792998843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 msgr2=0x7f6e141a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e19d9b700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e19d9b700 1 -- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e1410cd00 msgr2=0x7f6e1410d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e19d9b700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e1410cd00 0x7f6e1410d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e19d9b700 1 -- 172.21.15.26:0/792998843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009c40 con 0x7f6e140ff530 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.005+0000 7f6e1959a700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e19d9b700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 0x7f6e141a36d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000f80 tx=0x7f6e0c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e0affd700 1 -- 172.21.15.26:0/792998843 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00e040 con 0x7f6e140ff530 2023-10-06T21:11:14.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e0affd700 1 -- 172.21.15.26:0/792998843 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0c008e80 con 0x7f6e140ff530 2023-10-06T21:11:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e0affd700 1 -- 172.21.15.26:0/792998843 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c005040 con 0x7f6e140ff530 2023-10-06T21:11:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e141aa4e0 con 0x7f6e140ff530 2023-10-06T21:11:14.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.006+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e141aa970 con 0x7f6e140ff530 2023-10-06T21:11:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.014+0000 7f6e2f8f3b80 1 Processor -- start 2023-10-06T21:11:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 -- start start 2023-10-06T21:11:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da8199b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da8199dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da818f9460 con 0x55da819909e0 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da818eac20 con 0x55da8199b890 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.015+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da818e8540 con 0x55da819900c0 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54604/0 (socket says 172.21.15.26:54604) 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 -- 172.21.15.26:0/2470955047 learned_addr learned my addr 172.21.15.26:0/2470955047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 -- 172.21.15.26:0/2470955047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 msgr2=0x55da8199dc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da8199dc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 -- 172.21.15.26:0/2470955047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 msgr2=0x55da8199b350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da8199b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.016+0000 7f6e1a59c700 1 -- 172.21.15.26:0/2470955047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da8170ede0 con 0x55da819900c0 2023-10-06T21:11:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.017+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009c70 tx=0x7f6e04009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea5a21ddddc1b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.017+0000 7f6e0affd700 1 -- 172.21.15.26:0/792998843 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f6e0c01d470 con 0x7f6e140ff530 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.018+0000 7f6dfffff700 1 -- 172.21.15.26:0/2470955047 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04011dc0 con 0x55da819900c0 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.018+0000 7f6dfffff700 1 -- 172.21.15.26:0/2470955047 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e04004510 con 0x55da819900c0 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.018+0000 7f6dfffff700 1 -- 172.21.15.26:0/2470955047 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04005280 con 0x55da819900c0 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.019+0000 7f6e0affd700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 0x7f6e000a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.019+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/2470955047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 msgr2=0x55da819904a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.019+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009c70 tx=0x7f6e04009f80 comp rx=0 tx=0).stop 2023-10-06T21:11:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.019+0000 7f6e0affd700 1 -- 172.21.15.26:0/792998843 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e0c0ce740 con 0x7f6e140ff530 2023-10-06T21:11:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.020+0000 7f6e19d9b700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 0x7f6e000a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.021+0000 7f6e19d9b700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 0x7f6e000a3d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0cbd20 tx=0x7f6e0c009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.027+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/2470955047 shutdown_connections 2023-10-06T21:11:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.027+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da8199dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.027+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819904a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.027+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/2470955047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da8199b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.027+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/2470955047 >> 172.21.15.26:0/2470955047 conn(0x55da81905260 msgr2=0x55da8198c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.028+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/2470955047 shutdown_connections 2023-10-06T21:11:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.028+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/2470955047 wait complete. 2023-10-06T21:11:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.029+0000 7f6e2f8f3b80 1 -- start start 2023-10-06T21:11:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.029+0000 7f6e2f8f3b80 1 Processor -- start 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da81998530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da818e3e30 con 0x55da819909e0 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da81880710 con 0x55da8199b890 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da818803b0 con 0x55da819900c0 2023-10-06T21:11:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.030+0000 7f6e2f8f3b80 10 client.0 _ll_register_callbacks cb 0x55da816dd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54614/0 (socket says 172.21.15.26:54614) 2023-10-06T21:11:14.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 -- 172.21.15.26:0/1802606293 learned_addr learned my addr 172.21.15.26:0/1802606293 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 -- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 msgr2=0x55da81998bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.031+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.032+0000 7f6e1a59c700 1 -- 172.21.15.26:0/1802606293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 msgr2=0x55da81998530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.032+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da81998530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.032+0000 7f6e1a59c700 1 -- 172.21.15.26:0/1802606293 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e04009920 con 0x55da819900c0 2023-10-06T21:11:14.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.043+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e04012f90 tx=0x7f6e04004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52b98461585f8423 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.043+0000 7f6dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:14.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.044+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04011bc0 con 0x55da819900c0 2023-10-06T21:11:14.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.044+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e04011d20 con 0x55da819900c0 2023-10-06T21:11:14.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.044+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da81983f10 con 0x55da819900c0 2023-10-06T21:11:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.046+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e040051d0 con 0x55da819900c0 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.048+0000 7f6e2f8f3b80 10 client.25597 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.048+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55da816f7c60 con 0x55da819900c0 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.050+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da81983f10 con 0x55da819900c0 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.050+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6e04003a70 con 0x55da819900c0 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 20 client.25597 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ab1e0 0x55da819ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 4 client.25597 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.052+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da818dce10 con 0x55da819ab1e0 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.053+0000 7f6e1959a700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ab1e0 0x55da819ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.053+0000 7f6e1959a700 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ab1e0 0x55da819ad620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cddc1dd924f65e8 server_cookie=21b232ed6d92a55e in_seq=0 out_seq=0 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.054+0000 7f6dfd7fa700 10 client.25597 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.055+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/1802606293 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da818dce10 con 0x55da819ab1e0 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.055+0000 7f6dfd7fa700 10 client.25597 handle_command_reply: tid=0 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.055+0000 7f6e2f8f3b80 1 client.25597 shutdown 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 msgr2=0x55da8199b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e04012f90 tx=0x7f6e04004ea0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 shutdown_connections 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ab1e0 0x55da819ad620 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da8199b2b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6dfd7fa700 0 client.25597 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/1802606293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da81998530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 >> 172.21.15.26:0/1802606293 conn(0x55da81905260 msgr2=0x55da8198bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 shutdown_connections 2023-10-06T21:11:14.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.056+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/1802606293 wait complete. 2023-10-06T21:11:14.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.057+0000 7f6e2f8f3b80 20 client.25597 trim_cache size 0 max 16384 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.084+0000 7f6e2f8f3b80 1 Processor -- start 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 -- start start 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819900c0 0x55da8199ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819909e0 0x55da81997d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da81764920 con 0x55da819900c0 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da81988a20 con 0x55da8199b890 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.085+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da81999b40 con 0x55da819909e0 2023-10-06T21:11:14.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.086+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.086+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57024/0 (socket says 172.21.15.26:57024) 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.086+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3176084593 learned_addr learned my addr 172.21.15.26:0/3176084593 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.086+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819900c0 0x55da8199ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.086+0000 7f6e1959a700 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819909e0 0x55da81997d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3176084593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819909e0 msgr2=0x55da81997d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819909e0 0x55da81997d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3176084593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819900c0 msgr2=0x55da8199ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819900c0 0x55da8199ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3176084593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da81735250 con 0x55da8199b890 2023-10-06T21:11:14.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e04004f90 tx=0x7f6e04001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1e2bc61d20ae54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e08ff9700 1 -- 172.21.15.26:0/3176084593 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04005950 con 0x55da8199b890 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e08ff9700 1 -- 172.21.15.26:0/3176084593 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e040051d0 con 0x55da8199b890 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.087+0000 7f6e08ff9700 1 -- 172.21.15.26:0/3176084593 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003680 con 0x55da8199b890 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3176084593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 msgr2=0x55da81998250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e04004f90 tx=0x7f6e04001e70 comp rx=0 tx=0).stop 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3176084593 shutdown_connections 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da81998250 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819900c0 0x55da8199ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3176084593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819909e0 0x55da81997d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3176084593 >> 172.21.15.26:0/3176084593 conn(0x55da81905260 msgr2=0x55da819938b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.089+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3176084593 shutdown_connections 2023-10-06T21:11:14.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.090+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3176084593 wait complete. 2023-10-06T21:11:14.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.090+0000 7f6e2f8f3b80 1 -- start start 2023-10-06T21:11:14.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.090+0000 7f6e2f8f3b80 1 Processor -- start 2023-10-06T21:11:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da819a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da819fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da819a1300 con 0x55da819909e0 2023-10-06T21:11:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da819fe570 con 0x55da8199b890 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da819fe700 con 0x55da819900c0 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.091+0000 7f6e2f8f3b80 10 client.0 _ll_register_callbacks cb 0x7f6e04004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.092+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.092+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da819fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.092+0000 7f6e1a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54634/0 (socket says 172.21.15.26:54634) 2023-10-06T21:11:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.092+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3306103135 learned_addr learned my addr 172.21.15.26:0/3306103135 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 msgr2=0x55da819fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da819fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3306103135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 msgr2=0x55da819a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da819a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 -- 172.21.15.26:0/3306103135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da816f7c60 con 0x55da819900c0 2023-10-06T21:11:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6e1a59c700 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e04004c90 tx=0x7f6e04000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c256a6d4046e0d1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.093+0000 7f6dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003700 con 0x55da819900c0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f6e04003860 con 0x55da819900c0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e04004510 con 0x55da819900c0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e04003a20 con 0x55da819900c0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6e2f8f3b80 10 client.25603 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.094+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55da8170d2b0 con 0x55da819900c0 2023-10-06T21:11:14.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.095+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e04004510 con 0x55da819900c0 2023-10-06T21:11:14.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6e04005340 con 0x55da819900c0 2023-10-06T21:11:14.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 10 client.25603 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:14.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:14.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 20 client.25603 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:14.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ff940 0x55da81a01d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 4 client.25603 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:11:14.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da818dcb30 con 0x55da819ff940 2023-10-06T21:11:14.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.097+0000 7f6e1959a700 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ff940 0x55da81a01d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.099+0000 7f6e1959a700 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ff940 0x55da81a01d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa1d2a7419460f4 server_cookie=4bdd14338f3f85cd in_seq=0 out_seq=0 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.099+0000 7f6dfd7fa700 10 client.25603 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.099+0000 7f6dfd7fa700 1 -- 172.21.15.26:0/3306103135 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55da818dcb30 con 0x55da819ff940 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.099+0000 7f6dfd7fa700 10 client.25603 handle_command_reply: tid=0 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.099+0000 7f6e2f8f3b80 1 client.25603 shutdown 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 msgr2=0x55da819a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e04004c90 tx=0x7f6e04000bc0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 shutdown_connections 2023-10-06T21:11:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55da819ff940 0x55da81a01d30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da8199b890 0x55da819fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da819900c0 0x55da819a0620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 --2- 172.21.15.26:0/3306103135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da819909e0 0x55da819a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 >> 172.21.15.26:0/3306103135 conn(0x55da81905260 msgr2=0x55da81991fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6dfd7fa700 0 client.25603 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 shutdown_connections 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 1 -- 172.21.15.26:0/3306103135 wait complete. 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.101+0000 7f6e2f8f3b80 20 client.25603 trim_cache size 0 max 16384 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:11:14.105 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:11:14.106 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:11:14.106 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:11:14.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 msgr2=0x7f6e000a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 0x7f6e000a3d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0cbd20 tx=0x7f6e0c009920 comp rx=0 tx=0).stop 2023-10-06T21:11:14.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 msgr2=0x7f6e141a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 0x7f6e141a36d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000f80 tx=0x7f6e0c004c80 comp rx=0 tx=0).stop 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 shutdown_connections 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e000a1860 0x7f6e000a3d20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e1410cd00 0x7f6e1410d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e140ff530 0x7f6e141a36d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 --2- 172.21.15.26:0/792998843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e14106230 0x7f6e141a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 >> 172.21.15.26:0/792998843 conn(0x7f6e14077330 msgr2=0x7f6e14104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 shutdown_connections 2023-10-06T21:11:14.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.102+0000 7f6e1bfff700 1 -- 172.21.15.26:0/792998843 wait complete. 2023-10-06T21:11:14.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:11:14.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:11:14.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.455+0000 7fb8155ff700 1 -- 172.21.15.26:0/564901232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81010c7f0 msgr2=0x7fb8101143b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.455+0000 7fb8155ff700 1 --2- 172.21.15.26:0/564901232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81010c7f0 0x7fb8101143b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8080098f0 tx=0x7fb808009c00 comp rx=0 tx=0).stop 2023-10-06T21:11:14.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 -- 172.21.15.26:0/564901232 shutdown_connections 2023-10-06T21:11:14.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 --2- 172.21.15.26:0/564901232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81010c7f0 0x7fb8101143b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 --2- 172.21.15.26:0/564901232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810107f60 0x7fb810108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 --2- 172.21.15.26:0/564901232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb810108910 0x7fb81010c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 -- 172.21.15.26:0/564901232 >> 172.21.15.26:0/564901232 conn(0x7fb810076b70 msgr2=0x7fb810078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.456+0000 7fb8155ff700 1 -- 172.21.15.26:0/564901232 shutdown_connections 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.457+0000 7fb8155ff700 1 -- 172.21.15.26:0/564901232 wait complete. 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.458+0000 7fb8155ff700 1 Processor -- start 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 -- start start 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 0x7fb810072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 0x7fb81006c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81006cd60 con 0x7fb81006c2e0 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81006ced0 con 0x7fb810107f60 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb8155ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81006d090 con 0x7fb81010c7f0 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.459+0000 7fb80effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.460+0000 7fb807fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 0x7fb810072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.460+0000 7fb80effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57042/0 (socket says 172.21.15.26:57042) 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.460+0000 7fb80effd700 1 -- 172.21.15.26:0/1889998820 learned_addr learned my addr 172.21.15.26:0/1889998820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb80f7fe700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 0x7fb81006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb807fff700 1 -- 172.21.15.26:0/1889998820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 msgr2=0x7fb810072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb807fff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb807fff700 1 -- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 msgr2=0x7fb81006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb807fff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 0x7fb81006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb807fff700 1 -- 172.21.15.26:0/1889998820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808009520 con 0x7fb81010c7f0 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb80effd700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:11:14.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.462+0000 7fb80f7fe700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 0x7fb81006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.463+0000 7fb807fff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 0x7fb810072890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.463+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800c9c0 con 0x7fb81010c7f0 2023-10-06T21:11:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.463+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8101b2ad0 con 0x7fb81010c7f0 2023-10-06T21:11:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.464+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f8010430 con 0x7fb81010c7f0 2023-10-06T21:11:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.464+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8101b3000 con 0x7fb81010c7f0 2023-10-06T21:11:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.464+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8018680 con 0x7fb81010c7f0 2023-10-06T21:11:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.465+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0005320 con 0x7fb81010c7f0 2023-10-06T21:11:14.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.469+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fb7f800cb20 con 0x7fb81010c7f0 2023-10-06T21:11:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.473+0000 7fb80cff9700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 0x7fb7fc0a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.473+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb7f8014070 con 0x7fb81010c7f0 2023-10-06T21:11:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.474+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb7f80cd560 con 0x7fb81010c7f0 2023-10-06T21:11:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.474+0000 7fb80effd700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 0x7fb7fc0a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.476+0000 7fb80effd700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 0x7fb7fc0a3d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb800009c10 tx=0x7fb80000b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:14.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:14 smithi167 ceph-mon[134494]: pgmap v30: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-10-06T21:11:14.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:14 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:14.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:14 smithi026 ceph-mon[98490]: pgmap v30: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-10-06T21:11:14.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:14 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:14.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:14 smithi078 ceph-mon[134896]: pgmap v30: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2023-10-06T21:11:14.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:14 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:11:14.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.934+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7f0005190 con 0x7fb81010c7f0 2023-10-06T21:11:14.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.936+0000 7fb80cff9700 1 -- 172.21.15.26:0/1889998820 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fb7f8096490 con 0x7fb81010c7f0 2023-10-06T21:11:14.967 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:11:14.968 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:11:12.794718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/1710830195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":1710830195},{"type":"v1","addr":"172.21.15.167:6839","nonce":1710830195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:11:14.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 msgr2=0x7fb7fc0a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 0x7fb7fc0a3d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb800009c10 tx=0x7fb80000b040 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 msgr2=0x7fb810072890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 0x7fb810072890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f800eca0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 shutdown_connections 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb7fc0a1890 0x7fb7fc0a3d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb81006c2e0 0x7fb81006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb810107f60 0x7fb810072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 --2- 172.21.15.26:0/1889998820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb81010c7f0 0x7fb810072890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.939+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 >> 172.21.15.26:0/1889998820 conn(0x7fb810076b70 msgr2=0x7fb81010a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.940+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 shutdown_connections 2023-10-06T21:11:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:14.940+0000 7fb8155ff700 1 -- 172.21.15.26:0/1889998820 wait complete. 2023-10-06T21:11:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-10-06T21:11:14.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:11:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 -- 172.21.15.26:0/3243598530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c11a690 msgr2=0x7fd61c11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 --2- 172.21.15.26:0/3243598530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c11a690 0x7fd61c11ca80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009f90 tx=0x7fd60c00b330 comp rx=0 tx=0).stop 2023-10-06T21:11:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 -- 172.21.15.26:0/3243598530 shutdown_connections 2023-10-06T21:11:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 --2- 172.21.15.26:0/3243598530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c11a690 0x7fd61c11ca80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 --2- 172.21.15.26:0/3243598530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c10f640 0x7fd61c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 --2- 172.21.15.26:0/3243598530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c1083f0 0x7fd61c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.315+0000 7fd6209ff700 1 -- 172.21.15.26:0/3243598530 >> 172.21.15.26:0/3243598530 conn(0x7fd61c06d1a0 msgr2=0x7fd61c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.316+0000 7fd6209ff700 1 -- 172.21.15.26:0/3243598530 shutdown_connections 2023-10-06T21:11:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.316+0000 7fd6209ff700 1 -- 172.21.15.26:0/3243598530 wait complete. 2023-10-06T21:11:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.317+0000 7fd6209ff700 1 Processor -- start 2023-10-06T21:11:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.318+0000 7fd6209ff700 1 -- start start 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 0x7fd61c113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c11a690 0x7fd61c119680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c119e50 con 0x7fd61c11a690 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c119fc0 con 0x7fd61c1083f0 2023-10-06T21:11:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd6209ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c11a180 con 0x7fd61c10f640 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54664/0 (socket says 172.21.15.26:54664) 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.319+0000 7fd619d9b700 1 -- 172.21.15.26:0/4290471158 learned_addr learned my addr 172.21.15.26:0/4290471158 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c11a690 0x7fd61c119680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd61a59c700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 0x7fd61c113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 -- 172.21.15.26:0/4290471158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 msgr2=0x7fd61c113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 0x7fd61c113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 -- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c11a690 msgr2=0x7fd61c119680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c11a690 0x7fd61c119680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 -- 172.21.15.26:0/4290471158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610009520 con 0x7fd61c10f640 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd61a59c700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 0x7fd61c113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.320+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd61000eb30 tx=0x7fd61000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:15 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1889998820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:11:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.325+0000 7fd60b7fe700 1 -- 172.21.15.26:0/4290471158 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610009880 con 0x7fd61c10f640 2023-10-06T21:11:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.325+0000 7fd60b7fe700 1 -- 172.21.15.26:0/4290471158 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6100044e0 con 0x7fd61c10f640 2023-10-06T21:11:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.325+0000 7fd60b7fe700 1 -- 172.21.15.26:0/4290471158 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610010430 con 0x7fd61c10f640 2023-10-06T21:11:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.325+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c009c40 con 0x7fd61c10f640 2023-10-06T21:11:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.325+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c1bb460 con 0x7fd61c10f640 2023-10-06T21:11:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.332+0000 7fd60b7fe700 1 -- 172.21.15.26:0/4290471158 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7fd610003ee0 con 0x7fd61c10f640 2023-10-06T21:11:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.333+0000 7fd6343cfb80 1 Processor -- start 2023-10-06T21:11:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.336+0000 7fd60b7fe700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 0x7fd6040a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.336+0000 7fd60b7fe700 1 -- 172.21.15.26:0/4290471158 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd610014070 con 0x7fd61c10f640 2023-10-06T21:11:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.336+0000 7fd6343cfb80 1 -- start start 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad60c0 0x5626baad64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 0x7fd6040a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 0x5626baae3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baa3f460 con 0x5626baae1890 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baa30c20 con 0x5626baad60c0 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baa2e540 con 0x5626baad69e0 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd61a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad60c0 0x5626baad64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54674/0 (socket says 172.21.15.26:54674) 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.337+0000 7fd619d9b700 1 -- 172.21.15.26:0/4192411065 learned_addr learned my addr 172.21.15.26:0/4192411065 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 0x5626baae3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 -- 172.21.15.26:0/4192411065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad60c0 msgr2=0x5626baad64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad60c0 0x5626baad64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 -- 172.21.15.26:0/4192411065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 msgr2=0x5626baae3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 0x5626baae3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 -- 172.21.15.26:0/4192411065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ba854de0 con 0x5626baad69e0 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 0x5626baae3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd619d9b700 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd610096f90 tx=0x7fd610099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856cfe03d77f8bea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd608ff9700 1 -- 172.21.15.26:0/4192411065 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000cd40 con 0x5626baad69e0 2023-10-06T21:11:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd608ff9700 1 -- 172.21.15.26:0/4192411065 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd61009a520 con 0x5626baad69e0 2023-10-06T21:11:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.338+0000 7fd608ff9700 1 -- 172.21.15.26:0/4192411065 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61009a780 con 0x5626baad69e0 2023-10-06T21:11:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.339+0000 7fd6343cfb80 1 -- 172.21.15.26:0/4192411065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 msgr2=0x5626baae1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.339+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 0x7fd6040a3d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0060b0 tx=0x7fd60c006020 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.339+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd610096f90 tx=0x7fd610099e80 comp rx=0 tx=0).stop 2023-10-06T21:11:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.343+0000 7fd6343cfb80 1 -- 172.21.15.26:0/4192411065 shutdown_connections 2023-10-06T21:11:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.343+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baae1890 0x5626baae3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.343+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad60c0 0x5626baad64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.343+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/4192411065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad69e0 0x5626baae1350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.343+0000 7fd6343cfb80 1 -- 172.21.15.26:0/4192411065 >> 172.21.15.26:0/4192411065 conn(0x5626baa4b260 msgr2=0x5626baad2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.344+0000 7fd6343cfb80 1 -- 172.21.15.26:0/4192411065 shutdown_connections 2023-10-06T21:11:15.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.344+0000 7fd6343cfb80 1 -- 172.21.15.26:0/4192411065 wait complete. 2023-10-06T21:11:15.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.344+0000 7fd6343cfb80 1 -- start start 2023-10-06T21:11:15.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.345+0000 7fd6343cfb80 1 Processor -- start 2023-10-06T21:11:15.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad69e0 0x5626baadeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baae1890 0x5626baadf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baa29e30 con 0x5626baad69e0 2023-10-06T21:11:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ba9c6710 con 0x5626baae1890 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ba9c63b0 con 0x5626baad60c0 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd6343cfb80 10 client.0 _ll_register_callbacks cb 0x5626ba823800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd61a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd61a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54688/0 (socket says 172.21.15.26:54688) 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.346+0000 7fd61a59c700 1 -- 172.21.15.26:0/1022381584 learned_addr learned my addr 172.21.15.26:0/1022381584 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baae1890 0x5626baadf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61a59c700 1 -- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baae1890 msgr2=0x5626baadf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61a59c700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baae1890 0x5626baadf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61a59c700 1 -- 172.21.15.26:0/1022381584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad69e0 msgr2=0x5626baadeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61a59c700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad69e0 0x5626baadeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.347+0000 7fd61a59c700 1 -- 172.21.15.26:0/1022381584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c1bb460 con 0x5626baad60c0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.348+0000 7fd61a59c700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd614012700 tx=0x7fd614012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d40eabb4ce38503 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.348+0000 7fd601ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.348+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x5626baad60c0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.348+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd614004320 con 0x5626baad60c0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.348+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626baac9f10 con 0x5626baad60c0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.349+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd614004520 con 0x5626baad60c0 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.349+0000 7fd6343cfb80 10 client.25621 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.349+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5626ba83dc60 con 0x5626baad60c0 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.349+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626baac9f10 con 0x5626baad60c0 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd614011950 con 0x5626baad60c0 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 10 client.25621 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 10 client.25621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 20 client.25621 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626baaf1260 0x5626baaf3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 4 client.25621 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.350+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626baa22e10 con 0x5626baaf1260 2023-10-06T21:11:15.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.352+0000 7fd619d9b700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626baaf1260 0x5626baaf3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.353+0000 7fd619d9b700 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626baaf1260 0x5626baaf3680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efe23c6d9c554859 server_cookie=ac27cb7b20a38055 in_seq=0 out_seq=0 2023-10-06T21:11:15.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.353+0000 7fd601ffb700 10 client.25621 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:11:15.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.355+0000 7fd601ffb700 1 -- 172.21.15.26:0/1022381584 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5626baa22e10 con 0x5626baaf1260 2023-10-06T21:11:15.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.355+0000 7fd601ffb700 10 client.25621 handle_command_reply: tid=0 2023-10-06T21:11:15.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.355+0000 7fd6343cfb80 1 client.25621 shutdown 2023-10-06T21:11:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.355+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 msgr2=0x5626baade610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.355+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd614012700 tx=0x7fd614012a10 comp rx=0 tx=0).stop 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 shutdown_connections 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626baaf1260 0x5626baaf3680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baae1890 0x5626baadf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baad60c0 0x5626baade610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1022381584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad69e0 0x5626baadeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 >> 172.21.15.26:0/1022381584 conn(0x5626baa4b260 msgr2=0x5626baad1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd601ffb700 0 client.25621 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 shutdown_connections 2023-10-06T21:11:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1022381584 wait complete. 2023-10-06T21:11:15.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.356+0000 7fd6343cfb80 20 client.25621 trim_cache size 0 max 16384 2023-10-06T21:11:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.387+0000 7fd6343cfb80 1 Processor -- start 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 -- start start 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baad8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baadb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ba8aa920 con 0x5626baad60c0 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baae8910 con 0x5626baad69e0 2023-10-06T21:11:15.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.388+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baadcc20 con 0x5626baae1890 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57080/0 (socket says 172.21.15.26:57080) 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 -- 172.21.15.26:0/1970901363 learned_addr learned my addr 172.21.15.26:0/1970901363 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 -- 172.21.15.26:0/1970901363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 msgr2=0x5626baadb920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baadb920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 -- 172.21.15.26:0/1970901363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 msgr2=0x5626baad8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baad8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.389+0000 7fd619d9b700 1 -- 172.21.15.26:0/1970901363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ba87b250 con 0x5626baad69e0 2023-10-06T21:11:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.390+0000 7fd619d9b700 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd61009a220 tx=0x7fd61009a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3717daa121faa1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.390+0000 7fd6097fa700 1 -- 172.21.15.26:0/1970901363 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610003680 con 0x5626baad69e0 2023-10-06T21:11:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.390+0000 7fd6097fa700 1 -- 172.21.15.26:0/1970901363 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6100caca0 con 0x5626baad69e0 2023-10-06T21:11:15.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.390+0000 7fd6097fa700 1 -- 172.21.15.26:0/1970901363 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610099c70 con 0x5626baad69e0 2023-10-06T21:11:15.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.391+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1970901363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 msgr2=0x5626baae8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.391+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd61009a220 tx=0x7fd61009a110 comp rx=0 tx=0).stop 2023-10-06T21:11:15.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1970901363 shutdown_connections 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baadb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baad8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/1970901363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8370 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1970901363 >> 172.21.15.26:0/1970901363 conn(0x5626baa4b260 msgr2=0x5626baad9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1970901363 shutdown_connections 2023-10-06T21:11:15.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 -- 172.21.15.26:0/1970901363 wait complete. 2023-10-06T21:11:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.392+0000 7fd6343cfb80 1 -- start start 2023-10-06T21:11:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.393+0000 7fd6343cfb80 1 Processor -- start 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baae7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baae6940 con 0x5626baad60c0 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baae6ab0 con 0x5626baad69e0 2023-10-06T21:11:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626baae6c70 con 0x5626baae1890 2023-10-06T21:11:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.394+0000 7fd6343cfb80 10 client.0 _ll_register_callbacks cb 0x7fd614009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.395+0000 7fd61ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.395+0000 7fd61ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54704/0 (socket says 172.21.15.26:54704) 2023-10-06T21:11:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.395+0000 7fd61ad9d700 1 -- 172.21.15.26:0/2346311857 learned_addr learned my addr 172.21.15.26:0/2346311857 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61a59c700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baae7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61ad9d700 1 -- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 msgr2=0x5626baae8150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61ad9d700 1 -- 172.21.15.26:0/2346311857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 msgr2=0x5626baae7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baae7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.396+0000 7fd61ad9d700 1 -- 172.21.15.26:0/2346311857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626ba854de0 con 0x5626baae1890 2023-10-06T21:11:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.398+0000 7fd61ad9d700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c003cb0 tx=0x7fd60c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45952807709e0c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.398+0000 7fd601ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.398+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0052e0 con 0x5626baae1890 2023-10-06T21:11:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.398+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd60c003e40 con 0x5626baae1890 2023-10-06T21:11:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.398+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd61009a520 con 0x5626baae1890 2023-10-06T21:11:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.399+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60c005440 con 0x5626baae1890 2023-10-06T21:11:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.399+0000 7fd6343cfb80 10 client.25624 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.399+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5626ba8532b0 con 0x5626baae1890 2023-10-06T21:11:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd60c036030 con 0x5626baae1890 2023-10-06T21:11:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd60c01dcb0 con 0x5626baae1890 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 10 client.25624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 20 client.25624 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626bab463e0 0x5626bab48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 4 client.25624 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.400+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626baa22b30 con 0x5626bab463e0 2023-10-06T21:11:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.401+0000 7fd61a59c700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626bab463e0 0x5626bab48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.401+0000 7fd61a59c700 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626bab463e0 0x5626bab48800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c357f2a6789ab07e server_cookie=69b421a2bb499524 in_seq=0 out_seq=0 2023-10-06T21:11:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.401+0000 7fd601ffb700 10 client.25624 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:11:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.401+0000 7fd601ffb700 1 -- 172.21.15.26:0/2346311857 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626baa22b30 con 0x5626bab463e0 2023-10-06T21:11:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.401+0000 7fd601ffb700 10 client.25624 handle_command_reply: tid=0 2023-10-06T21:11:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 client.25624 shutdown 2023-10-06T21:11:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 msgr2=0x5626baae6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c003cb0 tx=0x7fd60c009920 comp rx=0 tx=0).stop 2023-10-06T21:11:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 shutdown_connections 2023-10-06T21:11:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5626bab463e0 0x5626bab48800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5626baae1890 0x5626baae6360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5626baad60c0 0x5626baae7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 --2- 172.21.15.26:0/2346311857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626baad69e0 0x5626baae8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 >> 172.21.15.26:0/2346311857 conn(0x5626baa4b260 msgr2=0x5626baadce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.402+0000 7fd601ffb700 0 client.25624 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.403+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 shutdown_connections 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.403+0000 7fd6343cfb80 1 -- 172.21.15.26:0/2346311857 wait complete. 2023-10-06T21:11:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.403+0000 7fd6343cfb80 20 client.25624 trim_cache size 0 max 16384 2023-10-06T21:11:15.407 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:11:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.404+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 msgr2=0x7fd6040a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.404+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 0x7fd6040a3d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0060b0 tx=0x7fd60c006020 comp rx=0 tx=0).stop 2023-10-06T21:11:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.404+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 msgr2=0x7fd61c114320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.404+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd61000eb30 tx=0x7fd61000eef0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 shutdown_connections 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd6040a18a0 0x7fd6040a3d60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c11a690 0x7fd61c119680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd61c1083f0 0x7fd61c113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 --2- 172.21.15.26:0/4290471158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd61c10f640 0x7fd61c114320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.406+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 >> 172.21.15.26:0/4290471158 conn(0x7fd61c06d1a0 msgr2=0x7fd61c11bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.408+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 shutdown_connections 2023-10-06T21:11:15.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.408+0000 7fd6209ff700 1 -- 172.21.15.26:0/4290471158 wait complete. 2023-10-06T21:11:15.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:11:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 -- 172.21.15.26:0/3939761169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c103ca0 msgr2=0x7f372c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 --2- 172.21.15.26:0/3939761169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c103ca0 0x7f372c107ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37200098f0 tx=0x7f3720009c00 comp rx=0 tx=0).stop 2023-10-06T21:11:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 -- 172.21.15.26:0/3939761169 shutdown_connections 2023-10-06T21:11:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 --2- 172.21.15.26:0/3939761169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372c108410 0x7f372c10ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 --2- 172.21.15.26:0/3939761169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c1032f0 0x7f372c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 --2- 172.21.15.26:0/3939761169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c103ca0 0x7f372c107ce0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 -- 172.21.15.26:0/3939761169 >> 172.21.15.26:0/3939761169 conn(0x7f372c0feb30 msgr2=0x7f372c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 -- 172.21.15.26:0/3939761169 shutdown_connections 2023-10-06T21:11:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.478+0000 7f37343ff700 1 -- 172.21.15.26:0/3939761169 wait complete. 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.480+0000 7f37343ff700 1 Processor -- start 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.480+0000 7f37343ff700 1 -- start start 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.480+0000 7f37343ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 0x7f372c072550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.480+0000 7f37343ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372c072a90 0x7f372c06b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.480+0000 7f37343ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 0x7f372c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f37343ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c0701f0 con 0x7f372c072a90 2023-10-06T21:11:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f37343ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c070360 con 0x7f372c108410 2023-10-06T21:11:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f37343ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c070520 con 0x7f372c06b890 2023-10-06T21:11:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f373299c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 0x7f372c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f373299c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 0x7f372c06bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54730/0 (socket says 172.21.15.26:54730) 2023-10-06T21:11:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f373299c700 1 -- 172.21.15.26:0/2972204565 learned_addr learned my addr 172.21.15.26:0/2972204565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.481+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 0x7f372c072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 -- 172.21.15.26:0/2972204565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 msgr2=0x7f372c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 0x7f372c06bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 -- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372c072a90 msgr2=0x7f372c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372c072a90 0x7f372c06b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 -- 172.21.15.26:0/2972204565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720009520 con 0x7f372c108410 2023-10-06T21:11:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 0x7f372c072550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f372c102fb0 tx=0x7f371c00ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.482+0000 7f372b7fe700 1 -- 172.21.15.26:0/2972204565 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c0092e0 con 0x7f372c108410 2023-10-06T21:11:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.483+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c070800 con 0x7f372c108410 2023-10-06T21:11:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.483+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c070d10 con 0x7f372c108410 2023-10-06T21:11:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.483+0000 7f372b7fe700 1 -- 172.21.15.26:0/2972204565 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f371c009440 con 0x7f372c108410 2023-10-06T21:11:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.483+0000 7f372b7fe700 1 -- 172.21.15.26:0/2972204565 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c01e9f0 con 0x7f372c108410 2023-10-06T21:11:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.490+0000 7f372b7fe700 1 -- 172.21.15.26:0/2972204565 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f371c01eb50 con 0x7f372c108410 2023-10-06T21:11:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.490+0000 7f3747d22b80 1 Processor -- start 2023-10-06T21:11:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.491+0000 7f3747d22b80 1 -- start start 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48ca520 0x5650c48d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48d53d0 0x5650c48d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4833400 con 0x5650c48ca520 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4824bc0 con 0x5650c48d53d0 2023-10-06T21:11:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f3747d22b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c48224e0 con 0x5650c48c9c00 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.492+0000 7f373299c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54736/0 (socket says 172.21.15.26:54736) 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 -- 172.21.15.26:0/2153753403 learned_addr learned my addr 172.21.15.26:0/2153753403 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 -- 172.21.15.26:0/2153753403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48d53d0 msgr2=0x5650c48d77c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48d53d0 0x5650c48d77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 -- 172.21.15.26:0/2153753403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48ca520 msgr2=0x5650c48d4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48ca520 0x5650c48d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 -- 172.21.15.26:0/2153753403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c4648d80 con 0x5650c48c9c00 2023-10-06T21:11:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.493+0000 7f373299c700 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f372400bde0 tx=0x7f37240106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf10bc7ccb4cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.494+0000 7f372b7fe700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 0x7f37140a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.494+0000 7f372b7fe700 1 -- 172.21.15.26:0/2972204565 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f371c0d4ff0 con 0x7f372c108410 2023-10-06T21:11:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.495+0000 7f3728bfe700 1 -- 172.21.15.26:0/2153753403 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724010ec0 con 0x5650c48c9c00 2023-10-06T21:11:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.495+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 0x7f37140a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.495+0000 7f3728bfe700 1 -- 172.21.15.26:0/2153753403 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724015070 con 0x5650c48c9c00 2023-10-06T21:11:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.495+0000 7f3728bfe700 1 -- 172.21.15.26:0/2153753403 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724011440 con 0x5650c48c9c00 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 -- 172.21.15.26:0/2153753403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 msgr2=0x5650c48c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f372400bde0 tx=0x7f37240106b0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 -- 172.21.15.26:0/2153753403 shutdown_connections 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48d53d0 0x5650c48d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48c9fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2153753403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48ca520 0x5650c48d4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.497+0000 7f3747d22b80 1 -- 172.21.15.26:0/2153753403 >> 172.21.15.26:0/2153753403 conn(0x5650c483f200 msgr2=0x5650c48c6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.498+0000 7f3747d22b80 1 -- 172.21.15.26:0/2153753403 shutdown_connections 2023-10-06T21:11:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.498+0000 7f3747d22b80 1 -- 172.21.15.26:0/2153753403 wait complete. 2023-10-06T21:11:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.498+0000 7f3747d22b80 1 -- start start 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 Processor -- start 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f373219b700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 0x7f37140a3cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f371c0d25f0 tx=0x7f371c009ee0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c481ddd0 con 0x5650c48d53d0 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c47ba6b0 con 0x5650c48ca520 2023-10-06T21:11:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c47ba350 con 0x5650c48c9c00 2023-10-06T21:11:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.499+0000 7f3747d22b80 10 client.0 _ll_register_callbacks cb 0x5650c46177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57126/0 (socket says 172.21.15.26:57126) 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373199a700 1 -- 172.21.15.26:0/2115414551 learned_addr learned my addr 172.21.15.26:0/2115414551 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 -- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 msgr2=0x5650c48d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 -- 172.21.15.26:0/2115414551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 msgr2=0x5650c48d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48d44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.500+0000 7f373299c700 1 -- 172.21.15.26:0/2115414551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372400ba10 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f373299c700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48d3a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f372400bdb0 tx=0x7f3724005350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814970a473e0cb90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724004a10 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3724017070 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650c48bdd50 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3724004b70 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3747d22b80 10 client.25636 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.501+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650c4631c00 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650c48bdd50 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3724029030 con 0x5650c48c9c00 2023-10-06T21:11:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 10 client.25636 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 20 client.25636 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c48e4fa0 0x5650c48e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 4 client.25636 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650c4816db0 con 0x5650c48e4fa0 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.502+0000 7f373199a700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c48e4fa0 0x5650c48e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.503+0000 7f373199a700 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c48e4fa0 0x5650c48e73c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615d4755e0790b54 server_cookie=138b3f05e7809f6f in_seq=0 out_seq=0 2023-10-06T21:11:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.503+0000 7f3711ffb700 10 client.25636 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.504+0000 7f3711ffb700 1 -- 172.21.15.26:0/2115414551 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650c4816db0 con 0x5650c48e4fa0 2023-10-06T21:11:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.504+0000 7f3711ffb700 10 client.25636 handle_command_reply: tid=0 2023-10-06T21:11:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.504+0000 7f3747d22b80 1 client.25636 shutdown 2023-10-06T21:11:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 msgr2=0x5650c48d3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48d3a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f372400bdb0 tx=0x7f3724005350 comp rx=0 tx=0).stop 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 shutdown_connections 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48d44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48c9c00 0x5650c48d3a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c48e4fa0 0x5650c48e73c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 --2- 172.21.15.26:0/2115414551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 >> 172.21.15.26:0/2115414551 conn(0x5650c483f200 msgr2=0x5650c48c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3711ffb700 0 client.25636 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.507+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 shutdown_connections 2023-10-06T21:11:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.508+0000 7f3747d22b80 1 -- 172.21.15.26:0/2115414551 wait complete. 2023-10-06T21:11:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.508+0000 7f3747d22b80 20 client.25636 trim_cache size 0 max 16384 2023-10-06T21:11:15.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:15 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1889998820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:11:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.537+0000 7f3747d22b80 1 Processor -- start 2023-10-06T21:11:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.538+0000 7f3747d22b80 1 -- start start 2023-10-06T21:11:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.538+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48c9c00 0x5650c48cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.538+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.538+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 0x5650c48cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.538+0000 7f3747d22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c469e8c0 con 0x5650c48c9c00 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f3747d22b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c48c28c0 con 0x5650c48ca520 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f3747d22b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c48cdfa0 con 0x5650c48d53d0 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57128/0 (socket says 172.21.15.26:57128) 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373199a700 1 -- 172.21.15.26:0/3411070955 learned_addr learned my addr 172.21.15.26:0/3411070955 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373299c700 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 0x5650c48cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373299c700 1 -- 172.21.15.26:0/3411070955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 msgr2=0x5650c48d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373299c700 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.539+0000 7f373299c700 1 -- 172.21.15.26:0/3411070955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48c9c00 msgr2=0x5650c48cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f373299c700 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48c9c00 0x5650c48cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f373299c700 1 -- 172.21.15.26:0/3411070955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c466f1f0 con 0x5650c48d53d0 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f373299c700 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 0x5650c48cf3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37240053c0 tx=0x7f3724005950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=897f7189f5f2c879 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f37293ff700 1 -- 172.21.15.26:0/3411070955 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724005510 con 0x5650c48d53d0 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f37293ff700 1 -- 172.21.15.26:0/3411070955 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724034930 con 0x5650c48d53d0 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.540+0000 7f37293ff700 1 -- 172.21.15.26:0/3411070955 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724004fa0 con 0x5650c48d53d0 2023-10-06T21:11:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.541+0000 7f3747d22b80 1 -- 172.21.15.26:0/3411070955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 msgr2=0x5650c48cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.541+0000 7f3747d22b80 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 0x5650c48cf3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37240053c0 tx=0x7f3724005950 comp rx=0 tx=0).stop 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.542+0000 7f3747d22b80 1 -- 172.21.15.26:0/3411070955 shutdown_connections 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.542+0000 7f3747d22b80 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48d53d0 0x5650c48cf3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.542+0000 7f3747d22b80 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48c9c00 0x5650c48cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.542+0000 7f3747d22b80 1 --2- 172.21.15.26:0/3411070955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48ca520 0x5650c48d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.542+0000 7f3747d22b80 1 -- 172.21.15.26:0/3411070955 >> 172.21.15.26:0/3411070955 conn(0x5650c483f200 msgr2=0x5650c48cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.545+0000 7f3747d22b80 1 -- 172.21.15.26:0/3411070955 shutdown_connections 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.545+0000 7f3747d22b80 1 -- 172.21.15.26:0/3411070955 wait complete. 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.546+0000 7f3747d22b80 1 -- start start 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.546+0000 7f3747d22b80 1 Processor -- start 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48ca520 0x5650c48da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c48dae30 con 0x5650c48d53d0 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c48dafa0 con 0x5650c48c9c00 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4938dc0 con 0x5650c48ca520 2023-10-06T21:11:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.547+0000 7f3747d22b80 10 client.0 _ll_register_callbacks cb 0x7f372400bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57130/0 (socket says 172.21.15.26:57130) 2023-10-06T21:11:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 -- 172.21.15.26:0/1012933777 learned_addr learned my addr 172.21.15.26:0/1012933777 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373199a700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48ca520 0x5650c48da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 -- 172.21.15.26:0/1012933777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48ca520 msgr2=0x5650c48da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48ca520 0x5650c48da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 -- 172.21.15.26:0/1012933777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 msgr2=0x5650c48da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.548+0000 7f373299c700 1 -- 172.21.15.26:0/1012933777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650c4631c00 con 0x5650c48c9c00 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.549+0000 7f373299c700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3724005370 tx=0x7f3724034e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33af735eb8c1aec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.550+0000 7f3711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.550+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724005510 con 0x5650c48c9c00 2023-10-06T21:11:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.550+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3724000e40 con 0x5650c48c9c00 2023-10-06T21:11:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.551+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3724015070 con 0x5650c48c9c00 2023-10-06T21:11:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.556+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3724004a20 con 0x5650c48c9c00 2023-10-06T21:11:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.556+0000 7f3747d22b80 10 client.25658 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.556+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650c4647250 con 0x5650c48c9c00 2023-10-06T21:11:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724015070 con 0x5650c48c9c00 2023-10-06T21:11:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3724045050 con 0x5650c48c9c00 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 20 client.25658 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c493a040 0x5650c493c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 4 client.25658 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650c4816ad0 con 0x5650c493a040 2023-10-06T21:11:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.560+0000 7f373199a700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c493a040 0x5650c493c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.561+0000 7f373199a700 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c493a040 0x5650c493c460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85dfc5b6b4dddf1e server_cookie=2bcac53e6b8cafc9 in_seq=0 out_seq=0 2023-10-06T21:11:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.561+0000 7f3711ffb700 10 client.25658 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.574+0000 7f3711ffb700 1 -- 172.21.15.26:0/1012933777 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650c4816ad0 con 0x5650c493a040 2023-10-06T21:11:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.574+0000 7f3711ffb700 10 client.25658 handle_command_reply: tid=0 2023-10-06T21:11:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.575+0000 7f3747d22b80 1 client.25658 shutdown 2023-10-06T21:11:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 msgr2=0x5650c48dbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3724005370 tx=0x7f3724034e00 comp rx=0 tx=0).stop 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 shutdown_connections 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5650c48d53d0 0x5650c48da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5650c493a040 0x5650c493c460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5650c48c9c00 0x5650c48dbf30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 --2- 172.21.15.26:0/1012933777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5650c48ca520 0x5650c48da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 >> 172.21.15.26:0/1012933777 conn(0x5650c483f200 msgr2=0x5650c48d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.576+0000 7f3711ffb700 0 client.25658 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.577+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 shutdown_connections 2023-10-06T21:11:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.577+0000 7f3747d22b80 1 -- 172.21.15.26:0/1012933777 wait complete. 2023-10-06T21:11:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.577+0000 7f3747d22b80 20 client.25658 trim_cache size 0 max 16384 2023-10-06T21:11:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:11:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:11:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:11:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 msgr2=0x7f37140a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 0x7f37140a3cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f371c0d25f0 tx=0x7f371c009ee0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 msgr2=0x7f372c072550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 0x7f372c072550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f372c102fb0 tx=0x7f371c00ca80 comp rx=0 tx=0).stop 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 shutdown_connections 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f372c06b890 0x7f372c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f372c108410 0x7f372c072550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f37140a1810 0x7f37140a3cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 --2- 172.21.15.26:0/2972204565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f372c072a90 0x7f372c06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 >> 172.21.15.26:0/2972204565 conn(0x7f372c0feb30 msgr2=0x7f372c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 shutdown_connections 2023-10-06T21:11:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:15.579+0000 7f37343ff700 1 -- 172.21.15.26:0/2972204565 wait complete. 2023-10-06T21:11:15.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:15 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1889998820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:11:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/3457494141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e78101a70 msgr2=0x7f0e78105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 --2- 172.21.15.26:0/3457494141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e78101a70 0x7f0e78105ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68009ca0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/3457494141 shutdown_connections 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 --2- 172.21.15.26:0/3457494141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781061e0 0x7f0e7810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 --2- 172.21.15.26:0/3457494141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e781010c0 0x7f0e781014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 --2- 172.21.15.26:0/3457494141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e78101a70 0x7f0e78105ab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/3457494141 >> 172.21.15.26:0/3457494141 conn(0x7f0e780fc960 msgr2=0x7f0e780fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/3457494141 shutdown_connections 2023-10-06T21:11:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.824+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/3457494141 wait complete. 2023-10-06T21:11:15.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.826+0000 7f0e7e1ff700 1 Processor -- start 2023-10-06T21:11:15.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.826+0000 7f0e7e1ff700 1 -- start start 2023-10-06T21:11:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 0x7f0e78199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 0x7f0e7819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7819ed00 0x7f0e7819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7819f780 con 0x7f0e781010c0 2023-10-06T21:11:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7819f8f0 con 0x7f0e781061e0 2023-10-06T21:11:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.829+0000 7f0e7e1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7819fab0 con 0x7f0e7819ed00 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 0x7f0e78199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 0x7f0e7819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 0x7f0e78199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43510/0 (socket says 172.21.15.26:43510) 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e777fe700 1 -- 172.21.15.26:0/2729514802 learned_addr learned my addr 172.21.15.26:0/2729514802 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e76ffd700 1 -- 172.21.15.26:0/2729514802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7819ed00 msgr2=0x7f0e7819f180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7819ed00 0x7f0e7819f180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.830+0000 7f0e76ffd700 1 -- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 msgr2=0x7f0e78199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 0x7f0e78199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e76ffd700 1 -- 172.21.15.26:0/2729514802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680095f0 con 0x7f0e781061e0 2023-10-06T21:11:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 0x7f0e7819a3d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e74ff9700 1 -- 172.21.15.26:0/2729514802 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d070 con 0x7f0e781061e0 2023-10-06T21:11:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e74ff9700 1 -- 172.21.15.26:0/2729514802 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e68021410 con 0x7f0e781061e0 2023-10-06T21:11:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e74ff9700 1 -- 172.21.15.26:0/2729514802 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68021ea0 con 0x7f0e781061e0 2023-10-06T21:11:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.831+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/2729514802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e78110b90 con 0x7f0e781061e0 2023-10-06T21:11:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.833+0000 7f0e7e1ff700 1 -- 172.21.15.26:0/2729514802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e78110ff0 con 0x7f0e781061e0 2023-10-06T21:11:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.839+0000 7f0e91b83b80 1 Processor -- start 2023-10-06T21:11:15.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.843+0000 7f0e74ff9700 1 -- 172.21.15.26:0/2729514802 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f0e68021580 con 0x7f0e781061e0 2023-10-06T21:11:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.848+0000 7f0e74ff9700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 0x7f0e640a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.848+0000 7f0e74ff9700 1 -- 172.21.15.26:0/2729514802 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e680ce7b0 con 0x7f0e781061e0 2023-10-06T21:11:15.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.849+0000 7f0e91b83b80 1 -- start start 2023-10-06T21:11:15.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.850+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 0x7f0e640a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.852+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 0x7f0e640a3cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000b910 tx=0x7f0e7000b370 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dc6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852d2f460 con 0x560852dd1a70 2023-10-06T21:11:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852d20c20 con 0x560852dc6270 2023-10-06T21:11:15.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.853+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852d1e540 con 0x560852dc6b90 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.854+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dc6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.854+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dc6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57150/0 (socket says 172.21.15.26:57150) 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.854+0000 7f0e777fe700 1 -- 172.21.15.26:0/2794538870 learned_addr learned my addr 172.21.15.26:0/2794538870 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.854+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 -- 172.21.15.26:0/2794538870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 msgr2=0x560852dd1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 -- 172.21.15.26:0/2794538870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 msgr2=0x560852dc6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dc6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 -- 172.21.15.26:0/2794538870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560852b44de0 con 0x560852dd1a70 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.855+0000 7f0e77fff700 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd3e60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000d5c0 tx=0x7f0e70010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b61554518dc985 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.856+0000 7f0e6d7fa700 1 -- 172.21.15.26:0/2794538870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000b560 con 0x560852dd1a70 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.856+0000 7f0e6d7fa700 1 -- 172.21.15.26:0/2794538870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e70010db0 con 0x560852dd1a70 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.856+0000 7f0e6d7fa700 1 -- 172.21.15.26:0/2794538870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000f3c0 con 0x560852dd1a70 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.857+0000 7f0e91b83b80 1 -- 172.21.15.26:0/2794538870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 msgr2=0x560852dd3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.857+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd3e60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000d5c0 tx=0x7f0e70010000 comp rx=0 tx=0).stop 2023-10-06T21:11:15.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 -- 172.21.15.26:0/2794538870 shutdown_connections 2023-10-06T21:11:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd3e60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dc6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/2794538870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 -- 172.21.15.26:0/2794538870 >> 172.21.15.26:0/2794538870 conn(0x560852d3b260 msgr2=0x560852dc2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.858+0000 7f0e91b83b80 1 -- 172.21.15.26:0/2794538870 shutdown_connections 2023-10-06T21:11:15.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.859+0000 7f0e91b83b80 1 -- 172.21.15.26:0/2794538870 wait complete. 2023-10-06T21:11:15.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.859+0000 7f0e91b83b80 1 -- start start 2023-10-06T21:11:15.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.860+0000 7f0e91b83b80 1 Processor -- start 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dcec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852d19e30 con 0x560852dd1a70 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852cb6710 con 0x560852dc6270 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852cb63b0 con 0x560852dc6b90 2023-10-06T21:11:15.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e91b83b80 10 client.0 _ll_register_callbacks cb 0x560852b13800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57166/0 (socket says 172.21.15.26:57166) 2023-10-06T21:11:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e777fe700 1 -- 172.21.15.26:0/446787188 learned_addr learned my addr 172.21.15.26:0/446787188 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.861+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 -- 172.21.15.26:0/446787188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 msgr2=0x560852dce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 -- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 msgr2=0x560852dcec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dcec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 -- 172.21.15.26:0/446787188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7000d7d0 con 0x560852dc6270 2023-10-06T21:11:15.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e777fe700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007f40 tx=0x7f0e60007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e304bdad0c8551b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.862+0000 7f0e5effd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.863+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000e050 con 0x560852dc6270 2023-10-06T21:11:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.863+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e60013b30 con 0x560852dc6270 2023-10-06T21:11:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.865+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560852db9f10 con 0x560852dc6270 2023-10-06T21:11:15.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.865+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e60013cf0 con 0x560852dc6270 2023-10-06T21:11:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.865+0000 7f0e91b83b80 10 client.25670 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.865+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560852b2dc60 con 0x560852dc6270 2023-10-06T21:11:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.865+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560852db9f10 con 0x560852dc6270 2023-10-06T21:11:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e600203e0 con 0x560852dc6270 2023-10-06T21:11:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 10 client.25670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 20 client.25670 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852df2390 0x560852df47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 4 client.25670 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560852d12e10 con 0x560852df2390 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.866+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852df2390 0x560852df47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.867+0000 7f0e5effd700 10 client.25670 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.867+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852df2390 0x560852df47b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cec17194410cf559 server_cookie=5a08e75e94f99fc4 in_seq=0 out_seq=0 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.868+0000 7f0e5effd700 1 -- 172.21.15.26:0/446787188 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560852d12e10 con 0x560852df2390 2023-10-06T21:11:15.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.868+0000 7f0e5effd700 10 client.25670 handle_command_reply: tid=0 2023-10-06T21:11:15.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.868+0000 7f0e91b83b80 1 client.25670 shutdown 2023-10-06T21:11:15.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 msgr2=0x560852dd13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007f40 tx=0x7f0e60007f70 comp rx=0 tx=0).stop 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 shutdown_connections 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dcec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852dc6270 0x560852dd13d0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007f40 tx=0x7f0e60007f70 comp rx=0 tx=0).stop 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852df2390 0x560852df47b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/446787188 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 >> 172.21.15.26:0/446787188 conn(0x560852d3b260 msgr2=0x560852dc8380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e5effd700 0 client.25670 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 shutdown_connections 2023-10-06T21:11:15.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.873+0000 7f0e91b83b80 1 -- 172.21.15.26:0/446787188 wait complete. 2023-10-06T21:11:15.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.874+0000 7f0e91b83b80 20 client.25670 trim_cache size 0 max 16384 2023-10-06T21:11:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.897+0000 7f0e91b83b80 1 Processor -- start 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.898+0000 7f0e91b83b80 1 -- start start 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.898+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dc6b90 0x560852dcebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.898+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852ddcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852b9a920 con 0x560852dc6b90 2023-10-06T21:11:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852dc98d0 con 0x560852df43a0 2023-10-06T21:11:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852dc74e0 con 0x560852dd1a70 2023-10-06T21:11:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dc6b90 0x560852dcebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54810/0 (socket says 172.21.15.26:54810) 2023-10-06T21:11:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 -- 172.21.15.26:0/828386586 learned_addr learned my addr 172.21.15.26:0/828386586 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 -- 172.21.15.26:0/828386586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 msgr2=0x560852ddcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852ddcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 -- 172.21.15.26:0/828386586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dc6b90 msgr2=0x560852dcebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dc6b90 0x560852dcebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 -- 172.21.15.26:0/828386586 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560852b6b250 con 0x560852dd1a70 2023-10-06T21:11:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.899+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e68097a80 tx=0x7f0e68086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b86cd4dd01d212 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.900+0000 7f0e6dffb700 1 -- 172.21.15.26:0/828386586 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6809b350 con 0x560852dd1a70 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.900+0000 7f0e6dffb700 1 -- 172.21.15.26:0/828386586 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6809bc80 con 0x560852dd1a70 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.900+0000 7f0e6dffb700 1 -- 172.21.15.26:0/828386586 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6809ba40 con 0x560852dd1a70 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.900+0000 7f0e91b83b80 1 -- 172.21.15.26:0/828386586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 msgr2=0x560852dc9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.900+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e68097a80 tx=0x7f0e68086040 comp rx=0 tx=0).stop 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 -- 172.21.15.26:0/828386586 shutdown_connections 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852ddcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dc6b90 0x560852dcebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/828386586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dd1a70 0x560852dc9390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 -- 172.21.15.26:0/828386586 >> 172.21.15.26:0/828386586 conn(0x560852d3b260 msgr2=0x560852dbb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 -- 172.21.15.26:0/828386586 shutdown_connections 2023-10-06T21:11:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 -- 172.21.15.26:0/828386586 wait complete. 2023-10-06T21:11:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 -- start start 2023-10-06T21:11:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.901+0000 7f0e91b83b80 1 Processor -- start 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852dd6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852dd6910 con 0x560852dd1a70 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852dd6a80 con 0x560852df43a0 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560852dd6c40 con 0x560852dc6b90 2023-10-06T21:11:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e91b83b80 10 client.0 _ll_register_callbacks cb 0x7f0e600078a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54824/0 (socket says 172.21.15.26:54824) 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e777fe700 1 -- 172.21.15.26:0/4138923301 learned_addr learned my addr 172.21.15.26:0/4138923301 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e77fff700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852dd6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.902+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 -- 172.21.15.26:0/4138923301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 msgr2=0x560852dd6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852dd6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 -- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 msgr2=0x560852dd83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 -- 172.21.15.26:0/4138923301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e78110ff0 con 0x560852dc6b90 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e77fff700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852dd6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:11:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e777fe700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007c90 tx=0x7f0e60013620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a756d5c1ce90391 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.903+0000 7f0e5effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.904+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000e040 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.904+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e60003a00 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.904+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e70010db0 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.905+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e60003bc0 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.905+0000 7f0e91b83b80 10 client.25654 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.905+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560852b432b0 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.905+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e70010db0 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.905+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e600052c0 con 0x560852dc6b90 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 10 client.25654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 20 client.25654 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852e47460 0x560852e49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 4 client.25654 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:11:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.906+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560852d12b30 con 0x560852e47460 2023-10-06T21:11:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.907+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852e47460 0x560852e49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.907+0000 7f0e5effd700 10 client.25654 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:11:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.907+0000 7f0e76ffd700 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852e47460 0x560852e49880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17181cff03cf380d server_cookie=270456a37f976115 in_seq=0 out_seq=0 2023-10-06T21:11:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.907+0000 7f0e5effd700 1 -- 172.21.15.26:0/4138923301 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560852d12b30 con 0x560852e47460 2023-10-06T21:11:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.907+0000 7f0e5effd700 10 client.25654 handle_command_reply: tid=0 2023-10-06T21:11:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 client.25654 shutdown 2023-10-06T21:11:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 msgr2=0x560852dd7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007c90 tx=0x7f0e60013620 comp rx=0 tx=0).stop 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 shutdown_connections 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560852df43a0 0x560852dd6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560852e47460 0x560852e49880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560852dc6b90 0x560852dd7e80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 --2- 172.21.15.26:0/4138923301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560852dd1a70 0x560852dd83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 >> 172.21.15.26:0/4138923301 conn(0x560852d3b260 msgr2=0x560852dee680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.908+0000 7f0e5effd700 0 client.25654 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.909+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 shutdown_connections 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.909+0000 7f0e91b83b80 1 -- 172.21.15.26:0/4138923301 wait complete. 2023-10-06T21:11:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.909+0000 7f0e91b83b80 20 client.25654 trim_cache size 0 max 16384 2023-10-06T21:11:15.913 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:11:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 msgr2=0x7f0e640a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 0x7f0e640a3cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000b910 tx=0x7f0e7000b370 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 msgr2=0x7f0e7819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 0x7f0e7819a3d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68005ef0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 shutdown_connections 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7819ed00 0x7f0e7819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e640a1810 0x7f0e640a3cd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e781010c0 0x7f0e78199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 --2- 172.21.15.26:0/2729514802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e781061e0 0x7f0e7819a3d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.910+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 >> 172.21.15.26:0/2729514802 conn(0x7f0e780fc960 msgr2=0x7f0e780fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.911+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 shutdown_connections 2023-10-06T21:11:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:15.911+0000 7f0e6e7fc700 1 -- 172.21.15.26:0/2729514802 wait complete. 2023-10-06T21:11:15.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:11:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 -- 172.21.15.26:0/614456933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 msgr2=0x7f3a741163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 --2- 172.21.15.26:0/614456933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a741163f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009f90 tx=0x7f3a6c00b330 comp rx=0 tx=0).stop 2023-10-06T21:11:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 -- 172.21.15.26:0/614456933 shutdown_connections 2023-10-06T21:11:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 --2- 172.21.15.26:0/614456933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a741163f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 --2- 172.21.15.26:0/614456933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a74107eb0 0x7f3a74108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 --2- 172.21.15.26:0/614456933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a741087d0 0x7f3a74113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 -- 172.21.15.26:0/614456933 >> 172.21.15.26:0/614456933 conn(0x7f3a7406d1a0 msgr2=0x7f3a7406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 -- 172.21.15.26:0/614456933 shutdown_connections 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.323+0000 7f3a7abff700 1 -- 172.21.15.26:0/614456933 wait complete. 2023-10-06T21:11:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.324+0000 7f3a7abff700 1 Processor -- start 2023-10-06T21:11:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 -- start start 2023-10-06T21:11:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a74107eb0 0x7f3a74113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a741087d0 0x7f3a74110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74111250 con 0x7f3a741087d0 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a741113c0 con 0x7f3a74107eb0 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.326+0000 7f3a7abff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74111580 con 0x7f3a74114000 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54850/0 (socket says 172.21.15.26:54850) 2023-10-06T21:11:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 -- 172.21.15.26:0/2745308142 learned_addr learned my addr 172.21.15.26:0/2745308142 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 -- 172.21.15.26:0/2745308142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a74107eb0 msgr2=0x7f3a74113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a74107eb0 0x7f3a74113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 -- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a741087d0 msgr2=0x7f3a74110740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a741087d0 0x7f3a74110740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 -- 172.21.15.26:0/2745308142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009c40 con 0x7f3a74114000 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.327+0000 7f3a7919c700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c004cd0 tx=0x7f3a6c004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.328+0000 7f3a71ffb700 1 -- 172.21.15.26:0/2745308142 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00e040 con 0x7f3a74114000 2023-10-06T21:11:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.328+0000 7f3a71ffb700 1 -- 172.21.15.26:0/2745308142 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c007850 con 0x7f3a74114000 2023-10-06T21:11:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.328+0000 7f3a71ffb700 1 -- 172.21.15.26:0/2745308142 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0052a0 con 0x7f3a74114000 2023-10-06T21:11:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.328+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a741b2790 con 0x7f3a74114000 2023-10-06T21:11:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.328+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a741b2bc0 con 0x7f3a74114000 2023-10-06T21:11:16.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.334+0000 7f3a71ffb700 1 -- 172.21.15.26:0/2745308142 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f3a6c01d470 con 0x7f3a74114000 2023-10-06T21:11:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.336+0000 7f3a8e5e9b80 1 Processor -- start 2023-10-06T21:11:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 -- start start 2023-10-06T21:11:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 0x5631599bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159917460 con 0x5631599ae0c0 2023-10-06T21:11:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159908c20 con 0x5631599ae9e0 2023-10-06T21:11:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.337+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159906540 con 0x5631599b9890 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.338+0000 7f3a71ffb700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 0x7f3a5c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.338+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.338+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43584/0 (socket says 172.21.15.26:43584) 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.338+0000 7f3a73fff700 1 -- 172.21.15.26:0/3454012699 learned_addr learned my addr 172.21.15.26:0/3454012699 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.338+0000 7f3a73fff700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 0x5631599bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a71ffb700 1 -- 172.21.15.26:0/2745308142 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a6c0ce7f0 con 0x7f3a74114000 2023-10-06T21:11:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a7899b700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 -- 172.21.15.26:0/3454012699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 msgr2=0x5631599b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 -- 172.21.15.26:0/3454012699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 msgr2=0x5631599ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 -- 172.21.15.26:0/3454012699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56315972cde0 con 0x5631599b9890 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a7899b700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:11:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.339+0000 7f3a73fff700 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 0x5631599bbc80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400df60 tx=0x7f3a640047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8666d4bf7899d343 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.340+0000 7f3a7919c700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 0x7f3a5c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.340+0000 7f3a5affd700 1 -- 172.21.15.26:0/3454012699 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64012c50 con 0x5631599b9890 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.340+0000 7f3a5affd700 1 -- 172.21.15.26:0/3454012699 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a64012db0 con 0x5631599b9890 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.340+0000 7f3a5affd700 1 -- 172.21.15.26:0/3454012699 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640103d0 con 0x5631599b9890 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/3454012699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 msgr2=0x5631599bbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 0x5631599bbc80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400df60 tx=0x7f3a640047c0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/3454012699 shutdown_connections 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599b9890 0x5631599bbc80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/3454012699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.341+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/3454012699 >> 172.21.15.26:0/3454012699 conn(0x563159923260 msgr2=0x5631599aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.342+0000 7f3a7919c700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 0x7f3a5c0a3c50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cbe30 tx=0x7f3a6c009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.343+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/3454012699 shutdown_connections 2023-10-06T21:11:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.343+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/3454012699 wait complete. 2023-10-06T21:11:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 -- start start 2023-10-06T21:11:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 Processor -- start 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 0x5631599b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 0x5631599b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159901e30 con 0x5631599b9890 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56315989e710 con 0x5631599ae9e0 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56315989e3b0 con 0x5631599ae0c0 2023-10-06T21:11:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.344+0000 7f3a8e5e9b80 10 client.0 _ll_register_callbacks cb 0x5631596fb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.345+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 0x5631599b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.346+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 0x5631599b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.346+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 0x5631599b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43586/0 (socket says 172.21.15.26:43586) 2023-10-06T21:11:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.346+0000 7f3a73fff700 1 -- 172.21.15.26:0/2661963902 learned_addr learned my addr 172.21.15.26:0/2661963902 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a73fff700 1 -- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 msgr2=0x5631599b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a73fff700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a73fff700 1 -- 172.21.15.26:0/2661963902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 msgr2=0x5631599b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a73fff700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 0x5631599b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.347+0000 7f3a73fff700 1 -- 172.21.15.26:0/2661963902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6400dd40 con 0x5631599ae0c0 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a73fff700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 0x5631599b66a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400dd10 tx=0x7f3a64005370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05eebcdee1e0a2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a53fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64004a30 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631599a1f10 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.348+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a64000e40 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.349+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64005700 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.349+0000 7f3a8e5e9b80 10 client.25666 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.349+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563159715c60 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.349+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631599a1f10 con 0x5631599ae0c0 2023-10-06T21:11:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.349+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3a6401cbd0 con 0x5631599ae0c0 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.350+0000 7f3a8e5e9b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.350+0000 7f3a8e5e9b80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.351+0000 7f3a8e5e9b80 20 client.25666 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.351+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5631599c9210 0x5631599cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.351+0000 7f3a8e5e9b80 4 client.25666 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.351+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631598fae10 con 0x5631599c9210 2023-10-06T21:11:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.351+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5631599c9210 0x5631599cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.352+0000 7f3a53fff700 10 client.25666 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.352+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5631599c9210 0x5631599cb630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2892b2ffcc012945 server_cookie=ce347cb87703232 in_seq=0 out_seq=0 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.352+0000 7f3a53fff700 1 -- 172.21.15.26:0/2661963902 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5631598fae10 con 0x5631599c9210 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.352+0000 7f3a53fff700 10 client.25666 handle_command_reply: tid=0 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 client.25666 shutdown 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 msgr2=0x5631599b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 0x5631599b66a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400dd10 tx=0x7f3a64005370 comp rx=0 tx=0).stop 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 shutdown_connections 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599b9890 0x5631599b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae0c0 0x5631599b66a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5631599c9210 0x5631599cb630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2661963902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599ae9e0 0x5631599b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a53fff700 0 client.25666 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.353+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 >> 172.21.15.26:0/2661963902 conn(0x563159923260 msgr2=0x5631599aa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.354+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 shutdown_connections 2023-10-06T21:11:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.354+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2661963902 wait complete. 2023-10-06T21:11:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.354+0000 7f3a8e5e9b80 20 client.25666 trim_cache size 0 max 16384 2023-10-06T21:11:16.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.373+0000 7f3a8e5e9b80 1 Processor -- start 2023-10-06T21:11:16.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 -- start start 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159782920 con 0x5631599ae0c0 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599af270 con 0x5631599b9890 2023-10-06T21:11:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.374+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599b4c20 con 0x5631599ae9e0 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54866/0 (socket says 172.21.15.26:54866) 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 -- 172.21.15.26:0/1760585632 learned_addr learned my addr 172.21.15.26:0/1760585632 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 -- 172.21.15.26:0/1760585632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 msgr2=0x5631599b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 -- 172.21.15.26:0/1760585632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 msgr2=0x5631599b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 -- 172.21.15.26:0/1760585632 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563159753250 con 0x5631599ae9e0 2023-10-06T21:11:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.375+0000 7f3a7899b700 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5631599b7be0 tx=0x7f3a68005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ef88be9e3ac29c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.376+0000 7f3a5b7fe700 1 -- 172.21.15.26:0/1760585632 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68017040 con 0x5631599ae9e0 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.376+0000 7f3a5b7fe700 1 -- 172.21.15.26:0/1760585632 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6801e070 con 0x5631599ae9e0 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.376+0000 7f3a5b7fe700 1 -- 172.21.15.26:0/1760585632 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68013520 con 0x5631599ae9e0 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/1760585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 msgr2=0x5631599b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5631599b7be0 tx=0x7f3a68005da0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/1760585632 shutdown_connections 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/1760585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599b5580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/1760585632 >> 172.21.15.26:0/1760585632 conn(0x563159923260 msgr2=0x5631599b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.377+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/1760585632 shutdown_connections 2023-10-06T21:11:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.378+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/1760585632 wait complete. 2023-10-06T21:11:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.378+0000 7f3a8e5e9b80 1 -- start start 2023-10-06T21:11:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.378+0000 7f3a8e5e9b80 1 Processor -- start 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599c00b0 con 0x5631599ae0c0 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599c0220 con 0x5631599b9890 2023-10-06T21:11:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599c03e0 con 0x5631599ae9e0 2023-10-06T21:11:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.379+0000 7f3a8e5e9b80 10 client.0 _ll_register_callbacks cb 0x7f3a6400d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a73fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54876/0 (socket says 172.21.15.26:54876) 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 -- 172.21.15.26:0/2449917415 learned_addr learned my addr 172.21.15.26:0/2449917415 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 -- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 msgr2=0x5631599bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 -- 172.21.15.26:0/2449917415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 msgr2=0x5631599be690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599be690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.380+0000 7f3a7899b700 1 -- 172.21.15.26:0/2449917415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a68009d00 con 0x5631599ae9e0 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.381+0000 7f3a7899b700 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005f30 tx=0x7f3a6801a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98408afe09ca8f23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.381+0000 7f3a53fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.381+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003d40 con 0x5631599ae9e0 2023-10-06T21:11:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.381+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a64012db0 con 0x5631599ae9e0 2023-10-06T21:11:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.381+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a68000e40 con 0x5631599ae9e0 2023-10-06T21:11:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.382+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a68004830 con 0x5631599ae9e0 2023-10-06T21:11:16.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.382+0000 7f3a8e5e9b80 10 client.25672 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:16.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.382+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56315972b2b0 con 0x5631599ae9e0 2023-10-06T21:11:16.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.382+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a64012db0 con 0x5631599ae9e0 2023-10-06T21:11:16.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3a68033070 con 0x5631599ae9e0 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 20 client.25672 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563159a1e350 0x563159a20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 4 client.25672 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631598fab30 con 0x563159a1e350 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a73fff700 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563159a1e350 0x563159a20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a53fff700 10 client.25672 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:16.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.383+0000 7f3a73fff700 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563159a1e350 0x563159a20770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d445d1651658a06c server_cookie=fd75e01515c19cd7 in_seq=0 out_seq=0 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a53fff700 1 -- 172.21.15.26:0/2449917415 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631598fab30 con 0x563159a1e350 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a53fff700 10 client.25672 handle_command_reply: tid=0 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 client.25672 shutdown 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 msgr2=0x5631599bebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005f30 tx=0x7f3a6801a6b0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 shutdown_connections 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563159a1e350 0x563159a20770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5631599b9890 0x5631599bfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5631599ae0c0 0x5631599be690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 --2- 172.21.15.26:0/2449917415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5631599ae9e0 0x5631599bebd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 >> 172.21.15.26:0/2449917415 conn(0x563159923260 msgr2=0x5631599b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a53fff700 0 client.25672 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:16.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.384+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 shutdown_connections 2023-10-06T21:11:16.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.385+0000 7f3a8e5e9b80 1 -- 172.21.15.26:0/2449917415 wait complete. 2023-10-06T21:11:16.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.386+0000 7f3a8e5e9b80 20 client.25672 trim_cache size 0 max 16384 2023-10-06T21:11:16.390 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:11:16.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.388+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 msgr2=0x7f3a5c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.388+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 0x7f3a5c0a3c50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cbe30 tx=0x7f3a6c009920 comp rx=0 tx=0).stop 2023-10-06T21:11:16.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.388+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 msgr2=0x7f3a74110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:16.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.388+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c004cd0 tx=0x7f3a6c004db0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 shutdown_connections 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a74114000 0x7f3a74110c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a74107eb0 0x7f3a74113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3a5c0a1790 0x7f3a5c0a3c50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 --2- 172.21.15.26:0/2745308142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a741087d0 0x7f3a74110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 >> 172.21.15.26:0/2745308142 conn(0x7f3a7406d1a0 msgr2=0x7f3a7406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 shutdown_connections 2023-10-06T21:11:16.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:16.389+0000 7f3a7abff700 1 -- 172.21.15.26:0/2745308142 wait complete. 2023-10-06T21:11:16.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:16 smithi167 ceph-mon[134494]: pgmap v31: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 49 MiB/s wr, 2.24k op/s 2023-10-06T21:11:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:16 smithi026 ceph-mon[98490]: pgmap v31: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 49 MiB/s wr, 2.24k op/s 2023-10-06T21:11:16.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:16 smithi078 ceph-mon[134896]: pgmap v31: 97 pgs: 97 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 49 MiB/s wr, 2.24k op/s 2023-10-06T21:11:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 35.0 s, 1878.2 tps, lat 51.807 ms stddev 70.799 2023-10-06T21:11:17.423 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:11:17.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:11:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.774+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1245521095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac0101a70 msgr2=0x7f8ac0105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.774+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1245521095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac0101a70 0x7f8ac0105ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40098f0 tx=0x7f8ab4009c00 comp rx=0 tx=0).stop 2023-10-06T21:11:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1245521095 shutdown_connections 2023-10-06T21:11:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1245521095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 0x7f8ac010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1245521095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac01010c0 0x7f8ac01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1245521095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac0101a70 0x7f8ac0105ab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1245521095 >> 172.21.15.26:0/1245521095 conn(0x7f8ac00fc960 msgr2=0x7f8ac00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.775+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1245521095 shutdown_connections 2023-10-06T21:11:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.776+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1245521095 wait complete. 2023-10-06T21:11:17.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.777+0000 7f8ac69ff700 1 Processor -- start 2023-10-06T21:11:17.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.777+0000 7f8ac69ff700 1 -- start start 2023-10-06T21:11:17.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac01010c0 0x7f8ac010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 0x7f8ac010a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac010e130 con 0x7f8ac01010c0 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0071920 con 0x7f8ac01061e0 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac69ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0071ae0 con 0x7f8ac010ae80 2023-10-06T21:11:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac4f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac4f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54904/0 (socket says 172.21.15.26:54904) 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8ac4f9c700 1 -- 172.21.15.26:0/1275728987 learned_addr learned my addr 172.21.15.26:0/1275728987 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.778+0000 7f8abffff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac01010c0 0x7f8ac010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 0x7f8ac010a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 -- 172.21.15.26:0/1275728987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 msgr2=0x7f8ac010a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 0x7f8ac010a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 -- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac01010c0 msgr2=0x7f8ac010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac01010c0 0x7f8ac010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 -- 172.21.15.26:0/1275728987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009520 con 0x7f8ac010ae80 2023-10-06T21:11:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8ac4f9c700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800b970 tx=0x7f8ab800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8abd7fa700 1 -- 172.21.15.26:0/1275728987 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800c9b0 con 0x7f8ac010ae80 2023-10-06T21:11:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8abd7fa700 1 -- 172.21.15.26:0/1275728987 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab8014440 con 0x7f8ac010ae80 2023-10-06T21:11:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.779+0000 7f8abd7fa700 1 -- 172.21.15.26:0/1275728987 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8007570 con 0x7f8ac010ae80 2023-10-06T21:11:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.780+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac0071dc0 con 0x7f8ac010ae80 2023-10-06T21:11:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.780+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac0072250 con 0x7f8ac010ae80 2023-10-06T21:11:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.787+0000 7f8ada3edb80 1 Processor -- start 2023-10-06T21:11:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.787+0000 7f8abd7fa700 1 -- 172.21.15.26:0/1275728987 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8ab80145b0 con 0x7f8ac010ae80 2023-10-06T21:11:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.788+0000 7f8ada3edb80 1 -- start start 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0e534e0 con 0x5622d0ef5910 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0e44ca0 con 0x5622d0eea140 2023-10-06T21:11:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0e425c0 con 0x5622d0eeaa60 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57248/0 (socket says 172.21.15.26:57248) 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.790+0000 7f8abf7fe700 1 -- 172.21.15.26:0/536931494 learned_addr learned my addr 172.21.15.26:0/536931494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 -- 172.21.15.26:0/536931494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 msgr2=0x5622d0ef53d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef53d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 -- 172.21.15.26:0/536931494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 msgr2=0x5622d0ef7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abd7fa700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 0x7f8aa80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 -- 172.21.15.26:0/536931494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622d0c68e60 con 0x5622d0eea140 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abd7fa700 1 -- 172.21.15.26:0/1275728987 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8ab80cc920 con 0x7f8ac010ae80 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.791+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40098f0 tx=0x7f8ab400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b76bbc5ba68a4fee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.792+0000 7f8ac4f9c700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 0x7f8aa80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.792+0000 7f8aa67fc700 1 -- 172.21.15.26:0/536931494 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400f900 con 0x5622d0eea140 2023-10-06T21:11:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.793+0000 7f8aa67fc700 1 -- 172.21.15.26:0/536931494 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab400ba70 con 0x5622d0eea140 2023-10-06T21:11:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.793+0000 7f8aa67fc700 1 -- 172.21.15.26:0/536931494 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4003680 con 0x5622d0eea140 2023-10-06T21:11:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.794+0000 7f8ada3edb80 1 -- 172.21.15.26:0/536931494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 msgr2=0x5622d0eea520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.794+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40098f0 tx=0x7f8ab400f460 comp rx=0 tx=0).stop 2023-10-06T21:11:17.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.794+0000 7f8ac4f9c700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 0x7f8aa80a3c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ca070 tx=0x7f8ab800c420 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.795+0000 7f8ada3edb80 1 -- 172.21.15.26:0/536931494 shutdown_connections 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.795+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.795+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0eea520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.795+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/536931494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.795+0000 7f8ada3edb80 1 -- 172.21.15.26:0/536931494 >> 172.21.15.26:0/536931494 conn(0x5622d0e5f2e0 msgr2=0x5622d0ee64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.796+0000 7f8ada3edb80 1 -- 172.21.15.26:0/536931494 shutdown_connections 2023-10-06T21:11:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.796+0000 7f8ada3edb80 1 -- 172.21.15.26:0/536931494 wait complete. 2023-10-06T21:11:17.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.797+0000 7f8ada3edb80 1 -- start start 2023-10-06T21:11:17.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.797+0000 7f8ada3edb80 1 Processor -- start 2023-10-06T21:11:17.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eeaa60 0x5622d0ef2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0ef3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0e3deb0 con 0x5622d0eeaa60 2023-10-06T21:11:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0dda790 con 0x5622d0ef5910 2023-10-06T21:11:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0dda430 con 0x5622d0eea140 2023-10-06T21:11:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.798+0000 7f8ada3edb80 10 client.0 _ll_register_callbacks cb 0x5622d0c37880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.799+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.799+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54914/0 (socket says 172.21.15.26:54914) 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.799+0000 7f8abf7fe700 1 -- 172.21.15.26:0/1594515073 learned_addr learned my addr 172.21.15.26:0/1594515073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.799+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0ef3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.800+0000 7f8abf7fe700 1 -- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 msgr2=0x5622d0ef3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.800+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0ef3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.800+0000 7f8abf7fe700 1 -- 172.21.15.26:0/1594515073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eeaa60 msgr2=0x5622d0ef2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.800+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eeaa60 0x5622d0ef2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.800+0000 7f8abf7fe700 1 -- 172.21.15.26:0/1594515073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac010e5d0 con 0x5622d0eea140 2023-10-06T21:11:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.801+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4005a80 tx=0x7f8ab40053d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1480c9f1d4918ca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.801+0000 7f8a9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.806+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4003880 con 0x5622d0eea140 2023-10-06T21:11:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.806+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8ab400b880 con 0x5622d0eea140 2023-10-06T21:11:17.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.806+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622d0eddf90 con 0x5622d0eea140 2023-10-06T21:11:17.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.810+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab40039e0 con 0x5622d0eea140 2023-10-06T21:11:17.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.811+0000 7f8ada3edb80 10 client.25681 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:17.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.811+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5622d0c51ce0 con 0x5622d0eea140 2023-10-06T21:11:17.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.811+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622d0eddf90 con 0x5622d0eea140 2023-10-06T21:11:17.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8ab4015020 con 0x5622d0eea140 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 10 client.25681 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 20 client.25681 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f15f60 0x5622d0f18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 4 client.25681 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.812+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622d0e36e90 con 0x5622d0f15f60 2023-10-06T21:11:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.813+0000 7f8abffff700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f15f60 0x5622d0f18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.815+0000 7f8abffff700 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f15f60 0x5622d0f18350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bac0e980b4a2a2 server_cookie=91432c405d122cb1 in_seq=0 out_seq=0 2023-10-06T21:11:17.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.815+0000 7f8a9f7fe700 10 client.25681 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.815+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/1594515073 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5622d0e36e90 con 0x5622d0f15f60 2023-10-06T21:11:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.815+0000 7f8a9f7fe700 10 client.25681 handle_command_reply: tid=0 2023-10-06T21:11:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 client.25681 shutdown 2023-10-06T21:11:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 msgr2=0x5622d0ef26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4005a80 tx=0x7f8ab40053d0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 shutdown_connections 2023-10-06T21:11:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f15f60 0x5622d0f18350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0ef3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eea140 0x5622d0ef26d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/1594515073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eeaa60 0x5622d0ef2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.816+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 >> 172.21.15.26:0/1594515073 conn(0x5622d0e5f2e0 msgr2=0x5622d0ee6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.817+0000 7f8a9f7fe700 0 client.25681 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.817+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 shutdown_connections 2023-10-06T21:11:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.817+0000 7f8ada3edb80 1 -- 172.21.15.26:0/1594515073 wait complete. 2023-10-06T21:11:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.819+0000 7f8ada3edb80 20 client.25681 trim_cache size 0 max 16384 2023-10-06T21:11:17.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.842+0000 7f8ada3edb80 1 Processor -- start 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 -- start start 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0cbe9a0 con 0x5622d0ef5910 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0ee5370 con 0x5622d0eea140 2023-10-06T21:11:17.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.843+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0eebc00 con 0x5622d0eeaa60 2023-10-06T21:11:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.844+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.844+0000 7f8abf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57266/0 (socket says 172.21.15.26:57266) 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.844+0000 7f8abf7fe700 1 -- 172.21.15.26:0/4177650256 learned_addr learned my addr 172.21.15.26:0/4177650256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.844+0000 7f8abffff700 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.844+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 -- 172.21.15.26:0/4177650256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 msgr2=0x5622d0ef46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 -- 172.21.15.26:0/4177650256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 msgr2=0x5622d0ef4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 -- 172.21.15.26:0/4177650256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622d0f1cf70 con 0x5622d0eea140 2023-10-06T21:11:17.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4005d60 tx=0x7f8ab4019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b193cfd9781924b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8aa6ffd700 1 -- 172.21.15.26:0/4177650256 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400bbe0 con 0x5622d0eea140 2023-10-06T21:11:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8aa6ffd700 1 -- 172.21.15.26:0/4177650256 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab401b030 con 0x5622d0eea140 2023-10-06T21:11:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.845+0000 7f8aa6ffd700 1 -- 172.21.15.26:0/4177650256 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4003c50 con 0x5622d0eea140 2023-10-06T21:11:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.846+0000 7f8ada3edb80 1 -- 172.21.15.26:0/4177650256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 msgr2=0x5622d0efc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.846+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4005d60 tx=0x7f8ab4019040 comp rx=0 tx=0).stop 2023-10-06T21:11:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.847+0000 7f8ada3edb80 1 -- 172.21.15.26:0/4177650256 shutdown_connections 2023-10-06T21:11:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.847+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0ef5910 0x5622d0ef4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.847+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0eea140 0x5622d0efc3a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.847+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/4177650256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0ef46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.847+0000 7f8ada3edb80 1 -- 172.21.15.26:0/4177650256 >> 172.21.15.26:0/4177650256 conn(0x5622d0e5f2e0 msgr2=0x5622d0ef3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.848+0000 7f8ada3edb80 1 -- 172.21.15.26:0/4177650256 shutdown_connections 2023-10-06T21:11:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.849+0000 7f8ada3edb80 1 -- 172.21.15.26:0/4177650256 wait complete. 2023-10-06T21:11:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.849+0000 7f8ada3edb80 1 -- start start 2023-10-06T21:11:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 Processor -- start 2023-10-06T21:11:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 0x5622d0f63080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0f60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0f61280 con 0x5622d0eea140 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0f613f0 con 0x5622d0ef5910 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622d0f615b0 con 0x5622d0eeaa60 2023-10-06T21:11:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8ada3edb80 10 client.0 _ll_register_callbacks cb 0x7f8ab4005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8abffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8abffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54928/0 (socket says 172.21.15.26:54928) 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8abffff700 1 -- 172.21.15.26:0/3560958400 learned_addr learned my addr 172.21.15.26:0/3560958400 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.850+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 0x5622d0f63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0f60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abffff700 1 -- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 msgr2=0x5622d0f60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abffff700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0f60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abffff700 1 -- 172.21.15.26:0/3560958400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 msgr2=0x5622d0f63080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abffff700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 0x5622d0f63080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abffff700 1 -- 172.21.15.26:0/3560958400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622d0c51ce0 con 0x5622d0eeaa60 2023-10-06T21:11:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.851+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 0x5622d0f63080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.852+0000 7f8abffff700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5622d0f18c50 tx=0x7f8ab0018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a739e1eaf7734b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.852+0000 7f8a9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.852+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0018a20 con 0x5622d0eeaa60 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.852+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ab400ba70 con 0x5622d0eeaa60 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.852+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8ab0018b80 con 0x5622d0eeaa60 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.853+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab0018d40 con 0x5622d0eeaa60 2023-10-06T21:11:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.853+0000 7f8ada3edb80 10 client.25687 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.853+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5622d0c8f2d0 con 0x5622d0eeaa60 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.853+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab400ba70 con 0x5622d0eeaa60 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.853+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8ab0027070 con 0x5622d0eeaa60 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 10 client.25687 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 20 client.25687 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f6b1d0 0x5622d0f6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 4 client.25687 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:11:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5622d0e36bb0 con 0x5622d0f6b1d0 2023-10-06T21:11:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.854+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f6b1d0 0x5622d0f6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.856+0000 7f8abf7fe700 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f6b1d0 0x5622d0f6d5f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837831c302377dc4 server_cookie=160d09090b648fdf in_seq=0 out_seq=0 2023-10-06T21:11:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.856+0000 7f8a9f7fe700 10 client.25687 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.857+0000 7f8a9f7fe700 1 -- 172.21.15.26:0/3560958400 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5622d0e36bb0 con 0x5622d0f6b1d0 2023-10-06T21:11:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.857+0000 7f8a9f7fe700 10 client.25687 handle_command_reply: tid=0 2023-10-06T21:11:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.857+0000 7f8ada3edb80 1 client.25687 shutdown 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.857+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 msgr2=0x5622d0f635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.857+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5622d0f18c50 tx=0x7f8ab0018420 comp rx=0 tx=0).stop 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 shutdown_connections 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622d0f6b1d0 0x5622d0f6d5f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622d0ef5910 0x5622d0f60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8a9f7fe700 0 client.25687 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622d0eea140 0x5622d0f63080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 --2- 172.21.15.26:0/3560958400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622d0eeaa60 0x5622d0f635c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 >> 172.21.15.26:0/3560958400 conn(0x5622d0e5f2e0 msgr2=0x5622d0ef4870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 shutdown_connections 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 1 -- 172.21.15.26:0/3560958400 wait complete. 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.858+0000 7f8ada3edb80 20 client.25687 trim_cache size 0 max 16384 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:11:17.862 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:11:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 msgr2=0x7f8aa80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 0x7f8aa80a3c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ca070 tx=0x7f8ab800c420 comp rx=0 tx=0).stop 2023-10-06T21:11:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 msgr2=0x7f8ac010b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800b970 tx=0x7f8ab800bd30 comp rx=0 tx=0).stop 2023-10-06T21:11:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 shutdown_connections 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ac010ae80 0x7f8ac010b330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8aa80a1790 0x7f8aa80a3c50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac01010c0 0x7f8ac010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.860+0000 7f8ac69ff700 1 --2- 172.21.15.26:0/1275728987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac01061e0 0x7f8ac010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.861+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 >> 172.21.15.26:0/1275728987 conn(0x7f8ac00fc960 msgr2=0x7f8ac0104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.861+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 shutdown_connections 2023-10-06T21:11:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:17.861+0000 7f8ac69ff700 1 -- 172.21.15.26:0/1275728987 wait complete. 2023-10-06T21:11:17.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 -- 172.21.15.26:0/265145197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec10b410 msgr2=0x7f8bec10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/265145197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec10b410 0x7f8bec10d8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8be400b0a0 tx=0x7f8be400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 -- 172.21.15.26:0/265145197 shutdown_connections 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/265145197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec10b410 0x7f8bec10d8d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/265145197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec101ff0 0x7f8bec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/265145197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 0x7f8bec10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 -- 172.21.15.26:0/265145197 >> 172.21.15.26:0/265145197 conn(0x7f8bec0fb830 msgr2=0x7f8bec0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 -- 172.21.15.26:0/265145197 shutdown_connections 2023-10-06T21:11:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.205+0000 7f8bf1fff700 1 -- 172.21.15.26:0/265145197 wait complete. 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.206+0000 7f8bf1fff700 1 Processor -- start 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.206+0000 7f8bf1fff700 1 -- start start 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 0x7f8bec1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 0x7f8bec19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec19c080 con 0x7f8bec102910 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec19c1f0 con 0x7f8bec101ff0 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bf1fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec19c3b0 con 0x7f8bec10b410 2023-10-06T21:11:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57300/0 (socket says 172.21.15.26:57300) 2023-10-06T21:11:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1066096608 learned_addr learned my addr 172.21.15.26:0/1066096608 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.207+0000 7f8bebfff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 0x7f8bec19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1066096608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 msgr2=0x7f8bec19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beaffd700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 0x7f8bec1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 0x7f8bec19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 msgr2=0x7f8bec1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 0x7f8bec1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1066096608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4009d00 con 0x7f8bec101ff0 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8bebfff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 0x7f8bec19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9a0 tx=0x7f8bdc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8be8ff9700 1 -- 172.21.15.26:0/1066096608 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc009790 con 0x7f8bec101ff0 2023-10-06T21:11:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8be8ff9700 1 -- 172.21.15.26:0/1066096608 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc00cbd0 con 0x7f8bec101ff0 2023-10-06T21:11:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.208+0000 7f8be8ff9700 1 -- 172.21.15.26:0/1066096608 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc018680 con 0x7f8bec101ff0 2023-10-06T21:11:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.209+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec19c690 con 0x7f8bec101ff0 2023-10-06T21:11:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.209+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec1a3ed0 con 0x7f8bec101ff0 2023-10-06T21:11:18.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:18 smithi026 ceph-mon[98490]: pgmap v32: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 MiB/s wr, 2.07k op/s 2023-10-06T21:11:18.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:18 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:11:18.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:18 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:18.218 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:18 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:11:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.216+0000 7f8c058b9b80 1 Processor -- start 2023-10-06T21:11:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.225+0000 7f8be8ff9700 1 -- 172.21.15.26:0/1066096608 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f8bdc00cd40 con 0x7f8bec101ff0 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.227+0000 7f8c058b9b80 1 -- start start 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203300c0 0x561c203304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203440a0 0x561c20346490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20299460 con 0x561c203300c0 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2028ac20 con 0x561c203309e0 2023-10-06T21:11:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.228+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20288540 con 0x561c203440a0 2023-10-06T21:11:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.229+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.229+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57316/0 (socket says 172.21.15.26:57316) 2023-10-06T21:11:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.229+0000 7f8beb7fe700 1 -- 172.21.15.26:0/2211006753 learned_addr learned my addr 172.21.15.26:0/2211006753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.230+0000 7f8beb7fe700 1 -- 172.21.15.26:0/2211006753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203440a0 msgr2=0x561c20346490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:11:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.230+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203440a0 0x561c20346490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.230+0000 7f8beb7fe700 1 -- 172.21.15.26:0/2211006753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203300c0 msgr2=0x561c203304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.230+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203300c0 0x561c203304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.230+0000 7f8beb7fe700 1 -- 172.21.15.26:0/2211006753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c200aede0 con 0x561c203309e0 2023-10-06T21:11:18.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.232+0000 7f8be8ff9700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 0x7f8bd40a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.234+0000 7f8be8ff9700 1 -- 172.21.15.26:0/1066096608 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bdc014070 con 0x7f8bec101ff0 2023-10-06T21:11:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.235+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cacc0 tx=0x7f8bdc0d56f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c11278364302224 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.235+0000 7f8bd1ffb700 1 -- 172.21.15.26:0/2211006753 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0d5e70 con 0x561c203309e0 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.235+0000 7f8bd1ffb700 1 -- 172.21.15.26:0/2211006753 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc096730 con 0x561c203309e0 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.235+0000 7f8bd1ffb700 1 -- 172.21.15.26:0/2211006753 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc09a3f0 con 0x561c203309e0 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 -- 172.21.15.26:0/2211006753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 msgr2=0x561c20343b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cacc0 tx=0x7f8bdc0d56f0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8bebfff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 0x7f8bd40a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 -- 172.21.15.26:0/2211006753 shutdown_connections 2023-10-06T21:11:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203440a0 0x561c20346490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203300c0 0x561c203304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/2211006753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20343b60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.236+0000 7f8c058b9b80 1 -- 172.21.15.26:0/2211006753 >> 172.21.15.26:0/2211006753 conn(0x561c202a5260 msgr2=0x561c2032c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.237+0000 7f8c058b9b80 1 -- 172.21.15.26:0/2211006753 shutdown_connections 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.237+0000 7f8c058b9b80 1 -- 172.21.15.26:0/2211006753 wait complete. 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.237+0000 7f8c058b9b80 1 -- start start 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8bebfff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 0x7f8bd40a3cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009850 tx=0x7f8be4009780 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 Processor -- start 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20337c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20338180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20283e30 con 0x561c203440a0 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20220710 con 0x561c203309e0 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c202203b0 con 0x561c203300c0 2023-10-06T21:11:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.238+0000 7f8c058b9b80 10 client.0 _ll_register_callbacks cb 0x561c2007d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54948/0 (socket says 172.21.15.26:54948) 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1352755613 learned_addr learned my addr 172.21.15.26:0/1352755613 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8bebfff700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20338180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 msgr2=0x561c20337c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20337c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1352755613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 msgr2=0x561c20338180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20338180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.239+0000 7f8beb7fe700 1 -- 172.21.15.26:0/1352755613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec1a3ed0 con 0x561c203300c0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.240+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c5b0 tx=0x7f8bdc09a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c65f3abe129d280 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.240+0000 7f8bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.240+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0d5e70 con 0x561c203300c0 2023-10-06T21:11:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.240+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bdc09aaf0 con 0x561c203300c0 2023-10-06T21:11:18.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.241+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c20323f10 con 0x561c203300c0 2023-10-06T21:11:18.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.241+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bdc09a1c0 con 0x561c203300c0 2023-10-06T21:11:18.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.242+0000 7f8c058b9b80 10 client.25696 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:18.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.242+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c20097c60 con 0x561c203300c0 2023-10-06T21:11:18.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.242+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c20323f10 con 0x561c203300c0 2023-10-06T21:11:18.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.243+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8bdc09e150 con 0x561c203300c0 2023-10-06T21:11:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 10 client.25696 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 10 client.25696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 20 client.25696 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c2035c1e0 0x561c2035e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 4 client.25696 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c2027ce10 con 0x561c2035c1e0 2023-10-06T21:11:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.244+0000 7f8beaffd700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c2035c1e0 0x561c2035e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.245+0000 7f8beaffd700 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c2035c1e0 0x561c2035e5d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1569695123d34f server_cookie=8992f639f87d336d in_seq=0 out_seq=0 2023-10-06T21:11:18.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.245+0000 7f8bcaffd700 10 client.25696 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.247+0000 7f8bcaffd700 1 -- 172.21.15.26:0/1352755613 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c2027ce10 con 0x561c2035c1e0 2023-10-06T21:11:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.247+0000 7f8bcaffd700 10 client.25696 handle_command_reply: tid=0 2023-10-06T21:11:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.247+0000 7f8c058b9b80 1 client.25696 shutdown 2023-10-06T21:11:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 msgr2=0x561c20337700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c5b0 tx=0x7f8bdc09a990 comp rx=0 tx=0).stop 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 shutdown_connections 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20338180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203300c0 0x561c20337700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c2035c1e0 0x561c2035e5d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/1352755613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203309e0 0x561c20337c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 >> 172.21.15.26:0/1352755613 conn(0x561c202a5260 msgr2=0x561c2032bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8bcaffd700 0 client.25696 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 shutdown_connections 2023-10-06T21:11:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 1 -- 172.21.15.26:0/1352755613 wait complete. 2023-10-06T21:11:18.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.248+0000 7f8c058b9b80 20 client.25696 trim_cache size 0 max 16384 2023-10-06T21:11:18.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 Processor -- start 2023-10-06T21:11:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 -- start start 2023-10-06T21:11:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20334990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20104920 con 0x561c203440a0 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c20338ce0 con 0x561c203300c0 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.277+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2033ef00 con 0x561c203309e0 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57342/0 (socket says 172.21.15.26:57342) 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 -- 172.21.15.26:0/24818003 learned_addr learned my addr 172.21.15.26:0/24818003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8bebfff700 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20334990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 -- 172.21.15.26:0/24818003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 msgr2=0x561c203420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:18.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 -- 172.21.15.26:0/24818003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 msgr2=0x561c20334990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:18.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20334990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.278+0000 7f8beb7fe700 1 -- 172.21.15.26:0/24818003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c200d5250 con 0x561c203300c0 2023-10-06T21:11:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.281+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc09a0f0 tx=0x7f8bdc0d0d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6a1660361f3dc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.281+0000 7f8bd27fc700 1 -- 172.21.15.26:0/24818003 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0cac80 con 0x561c203300c0 2023-10-06T21:11:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.281+0000 7f8bd27fc700 1 -- 172.21.15.26:0/24818003 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc09a3f0 con 0x561c203300c0 2023-10-06T21:11:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.281+0000 7f8bd27fc700 1 -- 172.21.15.26:0/24818003 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0a2780 con 0x561c203300c0 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 -- 172.21.15.26:0/24818003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 msgr2=0x561c2033e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc09a0f0 tx=0x7f8bdc0d0d10 comp rx=0 tx=0).stop 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 -- 172.21.15.26:0/24818003 shutdown_connections 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c20334990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c2033e990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/24818003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 -- 172.21.15.26:0/24818003 >> 172.21.15.26:0/24818003 conn(0x561c202a5260 msgr2=0x561c2035e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 -- 172.21.15.26:0/24818003 shutdown_connections 2023-10-06T21:11:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.282+0000 7f8c058b9b80 1 -- 172.21.15.26:0/24818003 wait complete. 2023-10-06T21:11:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.283+0000 7f8c058b9b80 1 -- start start 2023-10-06T21:11:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.283+0000 7f8c058b9b80 1 Processor -- start 2023-10-06T21:11:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c203a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c203a9c60 con 0x561c203440a0 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c203a4390 con 0x561c203300c0 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c203a4550 con 0x561c203309e0 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8c058b9b80 10 client.0 _ll_register_callbacks cb 0x7f8bdc0d5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57344/0 (socket says 172.21.15.26:57344) 2023-10-06T21:11:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.285+0000 7f8beb7fe700 1 -- 172.21.15.26:0/6320955 learned_addr learned my addr 172.21.15.26:0/6320955 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8bebfff700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c203a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8beb7fe700 1 -- 172.21.15.26:0/6320955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 msgr2=0x561c203a93e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:11:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203a93e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8beb7fe700 1 -- 172.21.15.26:0/6320955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 msgr2=0x561c203a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c203a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.286+0000 7f8beb7fe700 1 -- 172.21.15.26:0/6320955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c20097c60 con 0x561c203300c0 2023-10-06T21:11:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.287+0000 7f8beb7fe700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc000c00 tx=0x7f8bdc09a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a244780d713c0e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.287+0000 7f8bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.287+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc099f30 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.287+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8bdc0d0e40 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.287+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bdc096730 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bdc09a1c0 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 10 client.25709 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c200ad2b0 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc096730 con 0x561c203300c0 2023-10-06T21:11:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8bdc07a030 con 0x561c203300c0 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 20 client.25709 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c203c1750 0x561c203c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 4 client.25709 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.288+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561c2027cb30 con 0x561c203c1750 2023-10-06T21:11:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.289+0000 7f8beaffd700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c203c1750 0x561c203c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:18.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8beaffd700 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c203c1750 0x561c203c3b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5503fa491c3b11e7 server_cookie=58ee2eb096bf3256 in_seq=0 out_seq=0 2023-10-06T21:11:18.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8bcaffd700 10 client.25709 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:18.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8bcaffd700 1 -- 172.21.15.26:0/6320955 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561c2027cb30 con 0x561c203c1750 2023-10-06T21:11:18.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8bcaffd700 10 client.25709 handle_command_reply: tid=0 2023-10-06T21:11:18.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 client.25709 shutdown 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 msgr2=0x561c203a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc000c00 tx=0x7f8bdc09a990 comp rx=0 tx=0).stop 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 shutdown_connections 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561c203440a0 0x561c203a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561c203c1750 0x561c203c3b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c203300c0 0x561c203a8ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 --2- 172.21.15.26:0/6320955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c203309e0 0x561c203a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.292+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 >> 172.21.15.26:0/6320955 conn(0x561c202a5260 msgr2=0x561c2032b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.293+0000 7f8bcaffd700 0 client.25709 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.293+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 shutdown_connections 2023-10-06T21:11:18.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.293+0000 7f8c058b9b80 1 -- 172.21.15.26:0/6320955 wait complete. 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.293+0000 7f8c058b9b80 20 client.25709 trim_cache size 0 max 16384 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:11:18.297 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:11:18.298 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:11:18.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 msgr2=0x7f8bd40a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 0x7f8bd40a3cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009850 tx=0x7f8be4009780 comp rx=0 tx=0).stop 2023-10-06T21:11:18.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 msgr2=0x7f8bec1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9a0 tx=0x7f8bdc00ed60 comp rx=0 tx=0).stop 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 shutdown_connections 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bec10b410 0x7f8bec19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bec101ff0 0x7f8bec1970a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8bd40a1810 0x7f8bd40a3cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 --2- 172.21.15.26:0/1066096608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bec102910 0x7f8bec1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.295+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 >> 172.21.15.26:0/1066096608 conn(0x7f8bec0fb830 msgr2=0x7f8bec10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.296+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 shutdown_connections 2023-10-06T21:11:18.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:18.296+0000 7f8bf1fff700 1 -- 172.21.15.26:0/1066096608 wait complete. 2023-10-06T21:11:18.328 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:11:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:18 smithi167 ceph-mon[134494]: pgmap v32: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 MiB/s wr, 2.07k op/s 2023-10-06T21:11:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:18 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:11:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:18 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:18.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:18 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:11:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:18 smithi078 ceph-mon[134896]: pgmap v32: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 MiB/s wr, 2.07k op/s 2023-10-06T21:11:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:18 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:11:18.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:18 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:18.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:18 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:11:19.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:19 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:19 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:19.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:19 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:11:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:20 smithi167 ceph-mon[134494]: pgmap v33: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 967 op/s 2023-10-06T21:11:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:20 smithi026 ceph-mon[98490]: pgmap v33: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 967 op/s 2023-10-06T21:11:20.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:20 smithi078 ceph-mon[134896]: pgmap v33: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 967 op/s 2023-10-06T21:11:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 40.0 s, 2277.3 tps, lat 45.065 ms stddev 57.939 2023-10-06T21:11:22.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:22 smithi167 ceph-mon[134494]: pgmap v34: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 52 MiB/s wr, 1.44k op/s 2023-10-06T21:11:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:22 smithi026 ceph-mon[98490]: pgmap v34: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 52 MiB/s wr, 1.44k op/s 2023-10-06T21:11:22.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:22 smithi078 ceph-mon[134896]: pgmap v34: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 52 MiB/s wr, 1.44k op/s 2023-10-06T21:11:24.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:23 smithi167 ceph-mon[134494]: pgmap v35: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 42 MiB/s wr, 1.13k op/s 2023-10-06T21:11:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:23 smithi026 ceph-mon[98490]: pgmap v35: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 42 MiB/s wr, 1.13k op/s 2023-10-06T21:11:24.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:23 smithi078 ceph-mon[134896]: pgmap v35: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 42 MiB/s wr, 1.13k op/s 2023-10-06T21:11:26.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:26 smithi078 ceph-mon[134896]: pgmap v36: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 MiB/s wr, 2.60k op/s 2023-10-06T21:11:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:26 smithi167 ceph-mon[134494]: pgmap v36: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 MiB/s wr, 2.60k op/s 2023-10-06T21:11:26.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:26 smithi026 ceph-mon[98490]: pgmap v36: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 MiB/s wr, 2.60k op/s 2023-10-06T21:11:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 45.0 s, 2495.0 tps, lat 40.039 ms stddev 68.056 2023-10-06T21:11:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:28 smithi167 ceph-mon[134494]: pgmap v37: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 54 MiB/s wr, 2.34k op/s 2023-10-06T21:11:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:28 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:28 smithi026 ceph-mon[98490]: pgmap v37: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 54 MiB/s wr, 2.34k op/s 2023-10-06T21:11:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:28 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:28 smithi078 ceph-mon[134896]: pgmap v37: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 54 MiB/s wr, 2.34k op/s 2023-10-06T21:11:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:28 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 -- 172.21.15.26:0/3225747643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544111e70 msgr2=0x7f7544114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 --2- 172.21.15.26:0/3225747643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544111e70 0x7f7544114260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b0a0 tx=0x7f753c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 -- 172.21.15.26:0/3225747643 shutdown_connections 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 --2- 172.21.15.26:0/3225747643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544111e70 0x7f7544114260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 --2- 172.21.15.26:0/3225747643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544073250 0x7f7544073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 --2- 172.21.15.26:0/3225747643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 0x7f7544111930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.849+0000 7f754bdff700 1 -- 172.21.15.26:0/3225747643 >> 172.21.15.26:0/3225747643 conn(0x7f75440fc930 msgr2=0x7f75440fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.852+0000 7f754bdff700 1 -- 172.21.15.26:0/3225747643 shutdown_connections 2023-10-06T21:11:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.853+0000 7f754bdff700 1 -- 172.21.15.26:0/3225747643 wait complete. 2023-10-06T21:11:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.854+0000 7f754bdff700 1 Processor -- start 2023-10-06T21:11:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 -- start start 2023-10-06T21:11:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 0x7f7544111320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 0x7f75441118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754410aa90 con 0x7f7544111e70 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754410ac00 con 0x7f7544073b70 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.855+0000 7f754bdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754410adc0 con 0x7f7544073250 2023-10-06T21:11:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f7549b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 0x7f7544111320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36388/0 (socket says 172.21.15.26:36388) 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f7549b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 0x7f7544111320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39882/0 (socket says 172.21.15.26:39882) 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 -- 172.21.15.26:0/1808327375 learned_addr learned my addr 172.21.15.26:0/1808327375 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754939a700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 0x7f75441118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 -- 172.21.15.26:0/1808327375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 msgr2=0x7f7544111320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 0x7f7544111320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 -- 172.21.15.26:0/1808327375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 msgr2=0x7f75441118a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 0x7f75441118a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.856+0000 7f754a39c700 1 -- 172.21.15.26:0/1808327375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c009d00 con 0x7f7544111e70 2023-10-06T21:11:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.857+0000 7f754a39c700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.857+0000 7f753affd700 1 -- 172.21.15.26:0/1808327375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c00e040 con 0x7f7544111e70 2023-10-06T21:11:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.857+0000 7f753affd700 1 -- 172.21.15.26:0/1808327375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753c007930 con 0x7f7544111e70 2023-10-06T21:11:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.857+0000 7f753affd700 1 -- 172.21.15.26:0/1808327375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c01b6c0 con 0x7f7544111e70 2023-10-06T21:11:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.858+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754410eeb0 con 0x7f7544111e70 2023-10-06T21:11:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.858+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754410f310 con 0x7f7544111e70 2023-10-06T21:11:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.863+0000 7f753affd700 1 -- 172.21.15.26:0/1808327375 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f753c01d470 con 0x7f7544111e70 2023-10-06T21:11:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.865+0000 7f753affd700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 0x7f75300a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.865+0000 7f753affd700 1 -- 172.21.15.26:0/1808327375 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f753c0cde10 con 0x7f7544111e70 2023-10-06T21:11:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.866+0000 7f7549b9b700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 0x7f75300a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.867+0000 7f7549b9b700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 0x7f75300a3cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7540009a10 tx=0x7f7540009190 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 Processor -- start 2023-10-06T21:11:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 -- start start 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a590fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 0x562a1a59be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.868+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a4fa400 con 0x562a1a591520 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a4ebbc0 con 0x562a1a59c3d0 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a4e94e0 con 0x562a1a590c00 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33824/0 (socket says 172.21.15.26:33824) 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 -- 172.21.15.26:0/3558209012 learned_addr learned my addr 172.21.15.26:0/3558209012 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754939a700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 0x562a1a59be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a590fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 -- 172.21.15.26:0/3558209012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 msgr2=0x562a1a590fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a590fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 -- 172.21.15.26:0/3558209012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 msgr2=0x562a1a59be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 0x562a1a59be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754a39c700 1 -- 172.21.15.26:0/3558209012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1a30fd80 con 0x562a1a59c3d0 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f754939a700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 0x562a1a59be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.869+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a590fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.870+0000 7f754a39c700 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f753c097650 tx=0x7f753c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d682691f3d80ee21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.870+0000 7f752ffff700 1 -- 172.21.15.26:0/3558209012 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c011e50 con 0x562a1a59c3d0 2023-10-06T21:11:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.870+0000 7f752ffff700 1 -- 172.21.15.26:0/3558209012 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753c09b2e0 con 0x562a1a59c3d0 2023-10-06T21:11:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.870+0000 7f752ffff700 1 -- 172.21.15.26:0/3558209012 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c09a0d0 con 0x562a1a59c3d0 2023-10-06T21:11:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.871+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3558209012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 msgr2=0x562a1a59e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.871+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f753c097650 tx=0x7f753c009550 comp rx=0 tx=0).stop 2023-10-06T21:11:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3558209012 shutdown_connections 2023-10-06T21:11:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a59c3d0 0x562a1a59e7c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a590fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3558209012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a591520 0x562a1a59be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3558209012 >> 172.21.15.26:0/3558209012 conn(0x562a1a506200 msgr2=0x562a1a58d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3558209012 shutdown_connections 2023-10-06T21:11:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.875+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3558209012 wait complete. 2023-10-06T21:11:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.876+0000 7f755f6a0b80 1 -- start start 2023-10-06T21:11:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.876+0000 7f755f6a0b80 1 Processor -- start 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a59be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a5991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a4e4dd0 con 0x562a1a59c3d0 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a4816b0 con 0x562a1a591520 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a481350 con 0x562a1a590c00 2023-10-06T21:11:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f755f6a0b80 10 client.0 _ll_register_callbacks cb 0x562a1a2de7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36404/0 (socket says 172.21.15.26:36404) 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f754a39c700 1 -- 172.21.15.26:0/796837436 learned_addr learned my addr 172.21.15.26:0/796837436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.881+0000 7f7549b9b700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a59be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.883+0000 7f754a39c700 1 -- 172.21.15.26:0/796837436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 msgr2=0x562a1a59be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.883+0000 7f754a39c700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a59be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.883+0000 7f754a39c700 1 -- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 msgr2=0x562a1a5991a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:11:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.883+0000 7f754a39c700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a5991a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.883+0000 7f754a39c700 1 -- 172.21.15.26:0/796837436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754410f310 con 0x562a1a59c3d0 2023-10-06T21:11:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.886+0000 7f754a39c700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b070 tx=0x7f753c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73e3d3ff158d15d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.886+0000 7f752d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:11:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.899+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c011e50 con 0x562a1a59c3d0 2023-10-06T21:11:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.899+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f753c09abe0 con 0x562a1a59c3d0 2023-10-06T21:11:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.899+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a1a584d50 con 0x562a1a59c3d0 2023-10-06T21:11:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.900+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f753c0a2ee0 con 0x562a1a59c3d0 2023-10-06T21:11:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.900+0000 7f755f6a0b80 10 client.15597 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.900+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562a1a2f8c00 con 0x562a1a59c3d0 2023-10-06T21:11:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.900+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562a1a584d50 con 0x562a1a59c3d0 2023-10-06T21:11:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.900+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f753c0969d0 con 0x562a1a59c3d0 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 20 client.15597 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a5bc860 0x562a1a5bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 4 client.15597 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.901+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a1a4dddb0 con 0x562a1a5bc860 2023-10-06T21:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.902+0000 7f7549b9b700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a5bc860 0x562a1a5bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.902+0000 7f7549b9b700 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a5bc860 0x562a1a5bec80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a166e5ce4f5c9e2 server_cookie=fe1142eac63f6fad in_seq=0 out_seq=0 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.902+0000 7f752d7fa700 10 client.15597 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.903+0000 7f752d7fa700 1 -- 172.21.15.26:0/796837436 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562a1a4dddb0 con 0x562a1a5bc860 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.903+0000 7f752d7fa700 10 client.15597 handle_command_reply: tid=0 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.903+0000 7f755f6a0b80 1 client.15597 shutdown 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.904+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 msgr2=0x562a1a5998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.904+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f753c00b070 tx=0x7f753c0099e0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 shutdown_connections 2023-10-06T21:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5998e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a59be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a5bc860 0x562a1a5bec80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/796837436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a5991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 >> 172.21.15.26:0/796837436 conn(0x562a1a506200 msgr2=0x562a1a58cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f752d7fa700 0 client.15597 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 shutdown_connections 2023-10-06T21:11:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.905+0000 7f755f6a0b80 1 -- 172.21.15.26:0/796837436 wait complete. 2023-10-06T21:11:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.908+0000 7f755f6a0b80 20 client.15597 trim_cache size 0 max 16384 2023-10-06T21:11:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 Processor -- start 2023-10-06T21:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 -- start start 2023-10-06T21:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a59b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a59be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a3658c0 con 0x562a1a59c3d0 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a5848c0 con 0x562a1a591520 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.936+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a58de40 con 0x562a1a590c00 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39912/0 (socket says 172.21.15.26:39912) 2023-10-06T21:11:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 -- 172.21.15.26:0/4267843856 learned_addr learned my addr 172.21.15.26:0/4267843856 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f754939a700 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a59b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f754a39c700 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a59be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 -- 172.21.15.26:0/4267843856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 msgr2=0x562a1a59b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a59b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 -- 172.21.15.26:0/4267843856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 msgr2=0x562a1a59be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.937+0000 7f7549b9b700 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a59be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.938+0000 7f7549b9b700 1 -- 172.21.15.26:0/4267843856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1a3361f0 con 0x562a1a590c00 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.938+0000 7f7549b9b700 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f754000eb20 tx=0x7f7540011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de84a76e342c881 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.938+0000 7f7538bff700 1 -- 172.21.15.26:0/4267843856 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540027070 con 0x562a1a590c00 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.938+0000 7f7538bff700 1 -- 172.21.15.26:0/4267843856 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7540015940 con 0x562a1a590c00 2023-10-06T21:11:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.938+0000 7f7538bff700 1 -- 172.21.15.26:0/4267843856 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540022650 con 0x562a1a590c00 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 -- 172.21.15.26:0/4267843856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 msgr2=0x562a1a593b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f754000eb20 tx=0x7f7540011000 comp rx=0 tx=0).stop 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 -- 172.21.15.26:0/4267843856 shutdown_connections 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a59be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a590c00 0x562a1a593b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/4267843856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a591520 0x562a1a59b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.939+0000 7f755f6a0b80 1 -- 172.21.15.26:0/4267843856 >> 172.21.15.26:0/4267843856 conn(0x562a1a506200 msgr2=0x562a1a5944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.940+0000 7f755f6a0b80 1 -- 172.21.15.26:0/4267843856 shutdown_connections 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.940+0000 7f755f6a0b80 1 -- 172.21.15.26:0/4267843856 wait complete. 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.940+0000 7f755f6a0b80 1 -- start start 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 Processor -- start 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 0x562a1a5a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a591520 0x562a1a5a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a5a33b0 con 0x562a1a59c3d0 2023-10-06T21:11:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a5a1880 con 0x562a1a590c00 2023-10-06T21:11:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.941+0000 7f755f6a0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1a5a19f0 con 0x562a1a591520 2023-10-06T21:11:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.942+0000 7f755f6a0b80 10 client.0 _ll_register_callbacks cb 0x562a1a2de7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.942+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f754a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36408/0 (socket says 172.21.15.26:36408) 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f754a39c700 1 -- 172.21.15.26:0/3343402602 learned_addr learned my addr 172.21.15.26:0/3343402602 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 0x562a1a5a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f754939a700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a591520 0x562a1a5a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 -- 172.21.15.26:0/3343402602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a591520 msgr2=0x562a1a5a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a591520 0x562a1a5a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 -- 172.21.15.26:0/3343402602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 msgr2=0x562a1a5a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f7549b9b700 1 -- 172.21.15.26:0/3343402602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754001a040 con 0x562a1a590c00 2023-10-06T21:11:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.943+0000 7f754a39c700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.944+0000 7f7549b9b700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 0x562a1a5a2ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7540015fa0 tx=0x7f754000e540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de02b2b80e3f870d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.944+0000 7f752d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.944+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540027040 con 0x562a1a590c00 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.944+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7540015480 con 0x562a1a590c00 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.944+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f753c09b2e0 con 0x562a1a590c00 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.945+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7540015640 con 0x562a1a590c00 2023-10-06T21:11:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.945+0000 7f755f6a0b80 10 client.25724 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.945+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562a1a30e250 con 0x562a1a590c00 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.945+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753c09b2e0 con 0x562a1a590c00 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.945+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7540031490 con 0x562a1a590c00 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 20 client.25724 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a611960 0x562a1a613d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 4 client.25724 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:11:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.946+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a1a4ddad0 con 0x562a1a611960 2023-10-06T21:11:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.948+0000 7f754939a700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a611960 0x562a1a613d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.950+0000 7f754939a700 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a611960 0x562a1a613d80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e9c362148e4a84 server_cookie=1502440b3b79e507 in_seq=0 out_seq=0 2023-10-06T21:11:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.950+0000 7f752d7fa700 10 client.25724 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.964+0000 7f752d7fa700 1 -- 172.21.15.26:0/3343402602 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a1a4ddad0 con 0x562a1a611960 2023-10-06T21:11:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.964+0000 7f752d7fa700 10 client.25724 handle_command_reply: tid=0 2023-10-06T21:11:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.964+0000 7f755f6a0b80 1 client.25724 shutdown 2023-10-06T21:11:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.964+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 msgr2=0x562a1a5a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.964+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 0x562a1a5a2ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7540015fa0 tx=0x7f754000e540 comp rx=0 tx=0).stop 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 shutdown_connections 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a1a59c3d0 0x562a1a5a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x562a1a611960 0x562a1a613d80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a1a590c00 0x562a1a5a2ae0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 --2- 172.21.15.26:0/3343402602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a1a591520 0x562a1a5a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 >> 172.21.15.26:0/3343402602 conn(0x562a1a506200 msgr2=0x562a1a58c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f752d7fa700 0 client.25724 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.965+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 shutdown_connections 2023-10-06T21:11:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.966+0000 7f755f6a0b80 1 -- 172.21.15.26:0/3343402602 wait complete. 2023-10-06T21:11:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.966+0000 7f755f6a0b80 20 client.25724 trim_cache size 0 max 16384 2023-10-06T21:11:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:11:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:11:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:11:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:11:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 msgr2=0x7f75300a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 0x7f75300a3cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7540009a10 tx=0x7f7540009190 comp rx=0 tx=0).stop 2023-10-06T21:11:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 msgr2=0x7f754410a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c00bce0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 shutdown_connections 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75300a1810 0x7f75300a3cd0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7540009a10 tx=0x7f7540009190 comp rx=0 tx=0).stop 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7544111e70 0x7f754410a350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7544073250 0x7f7544111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 --2- 172.21.15.26:0/1808327375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7544073b70 0x7f75441118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.967+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 >> 172.21.15.26:0/1808327375 conn(0x7f75440fc930 msgr2=0x7f75440fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.968+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 shutdown_connections 2023-10-06T21:11:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:28.968+0000 7f754bdff700 1 -- 172.21.15.26:0/1808327375 wait complete. 2023-10-06T21:11:29.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:29 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:11:29.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:29 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:11:29.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:29 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:11:29.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:29 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:11:29.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:29 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:11:29.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:29 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:11:29.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:29 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:11:29.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:29 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:11:29.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:29 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:11:30.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:30 smithi167 ceph-mon[134494]: pgmap v38: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 45 MiB/s wr, 2.01k op/s 2023-10-06T21:11:30.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:30 smithi026 ceph-mon[98490]: pgmap v38: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 45 MiB/s wr, 2.01k op/s 2023-10-06T21:11:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:30 smithi078 ceph-mon[134896]: pgmap v38: 97 pgs: 97 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 45 MiB/s wr, 2.01k op/s 2023-10-06T21:11:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 50.0 s, 3036.5 tps, lat 33.085 ms stddev 37.198 2023-10-06T21:11:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:32 smithi167 ceph-mon[134494]: pgmap v39: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 53 MiB/s wr, 2.32k op/s 2023-10-06T21:11:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:32 smithi026 ceph-mon[98490]: pgmap v39: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 53 MiB/s wr, 2.32k op/s 2023-10-06T21:11:32.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:32 smithi078 ceph-mon[134896]: pgmap v39: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 53 MiB/s wr, 2.32k op/s 2023-10-06T21:11:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:33 smithi167 ceph-mon[134494]: pgmap v40: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-10-06T21:11:34.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:33 smithi026 ceph-mon[98490]: pgmap v40: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-10-06T21:11:34.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:33 smithi078 ceph-mon[134896]: pgmap v40: 97 pgs: 97 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 37 MiB/s wr, 1.85k op/s 2023-10-06T21:11:36.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:36 smithi078 ceph-mon[134896]: pgmap v41: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 45 MiB/s wr, 2.24k op/s 2023-10-06T21:11:36.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:36 smithi167 ceph-mon[134494]: pgmap v41: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 45 MiB/s wr, 2.24k op/s 2023-10-06T21:11:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:36 smithi026 ceph-mon[98490]: pgmap v41: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 45 MiB/s wr, 2.24k op/s 2023-10-06T21:11:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 55.0 s, 3323.3 tps, lat 30.073 ms stddev 31.486 2023-10-06T21:11:38.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:38 smithi167 ceph-mon[134494]: pgmap v42: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 18 MiB/s wr, 777 op/s 2023-10-06T21:11:38.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:38 smithi026 ceph-mon[98490]: pgmap v42: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 18 MiB/s wr, 777 op/s 2023-10-06T21:11:38.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:38 smithi078 ceph-mon[134896]: pgmap v42: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 18 MiB/s wr, 777 op/s 2023-10-06T21:11:40.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:40 smithi167 ceph-mon[134494]: pgmap v43: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 705 op/s 2023-10-06T21:11:40.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:40 smithi026 ceph-mon[98490]: pgmap v43: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 705 op/s 2023-10-06T21:11:40.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:40 smithi078 ceph-mon[134896]: pgmap v43: 97 pgs: 97 active+clean; 2.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 705 op/s 2023-10-06T21:11:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 60.0 s, 2691.7 tps, lat 37.038 ms stddev 40.244 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 -- 172.21.15.26:0/1174506502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 msgr2=0x7f4ef0103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/1174506502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef0103700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00098f0 tx=0x7f4ee0009c00 comp rx=0 tx=0).stop 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 -- 172.21.15.26:0/1174506502 shutdown_connections 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/1174506502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 0x7f4ef010ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/1174506502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef0103700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/1174506502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef0103cd0 0x7f4ef0107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 -- 172.21.15.26:0/1174506502 >> 172.21.15.26:0/1174506502 conn(0x7f4ef00feba0 msgr2=0x7f4ef0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 -- 172.21.15.26:0/1174506502 shutdown_connections 2023-10-06T21:11:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.207+0000 7f4ef61ff700 1 -- 172.21.15.26:0/1174506502 wait complete. 2023-10-06T21:11:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.209+0000 7f4ef61ff700 1 Processor -- start 2023-10-06T21:11:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.210+0000 7f4ef61ff700 1 -- start start 2023-10-06T21:11:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 0x7f4ef010cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef010d020 0x7f4ef010d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a9f90 con 0x7f4ef0108440 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01aa0d0 con 0x7f4ef010d020 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4ef61ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01aa290 con 0x7f4ef0103320 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.211+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44640/0 (socket says 172.21.15.26:44640) 2023-10-06T21:11:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3713716962 learned_addr learned my addr 172.21.15.26:0/3713716962 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eeffff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef010d020 0x7f4ef010d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 0x7f4ef010cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3713716962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef010d020 msgr2=0x7f4ef010d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef010d020 0x7f4ef010d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 msgr2=0x7f4ef010cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 0x7f4ef010cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3713716962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009520 con 0x7f4ef0103320 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.212+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009810 tx=0x7f4ee000b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.214+0000 7f4eecff9700 1 -- 172.21.15.26:0/3713716962 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee001c070 con 0x7f4ef0103320 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.214+0000 7f4eecff9700 1 -- 172.21.15.26:0/3713716962 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee0005040 con 0x7f4ef0103320 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.214+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01aa510 con 0x7f4ef0103320 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.214+0000 7f4eecff9700 1 -- 172.21.15.26:0/3713716962 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0003a70 con 0x7f4ef0103320 2023-10-06T21:11:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.215+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01aa930 con 0x7f4ef0103320 2023-10-06T21:11:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.222+0000 7f4eecff9700 1 -- 172.21.15.26:0/3713716962 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f4ee0020030 con 0x7f4ef0103320 2023-10-06T21:11:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.222+0000 7f4f09b78b80 1 Processor -- start 2023-10-06T21:11:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.225+0000 7f4f09b78b80 1 -- start start 2023-10-06T21:11:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.225+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e93ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8a9400 con 0x55737e93fc00 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e89abc0 con 0x55737e940520 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8984e0 con 0x55737e94b3d0 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52934/0 (socket says 172.21.15.26:52934) 2023-10-06T21:11:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3405502876 learned_addr learned my addr 172.21.15.26:0/3405502876 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3405502876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 msgr2=0x55737e94d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3405502876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 msgr2=0x55737e93ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e93ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.226+0000 7f4eef7fe700 1 -- 172.21.15.26:0/3405502876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e6bed80 con 0x55737e940520 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.227+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00cbd50 tx=0x7f4ee00d4a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ec32c1ac415d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.227+0000 7f4ed5ffb700 1 -- 172.21.15.26:0/3405502876 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00d4e70 con 0x55737e940520 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.227+0000 7f4ed5ffb700 1 -- 172.21.15.26:0/3405502876 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee00cece0 con 0x55737e940520 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.227+0000 7f4ed5ffb700 1 -- 172.21.15.26:0/3405502876 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00d9940 con 0x55737e940520 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.228+0000 7f4eecff9700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 0x7f4ed80a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.228+0000 7f4eecff9700 1 -- 172.21.15.26:0/3713716962 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4ee00ce660 con 0x7f4ef0103320 2023-10-06T21:11:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.228+0000 7f4eeffff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 0x7f4ed80a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.228+0000 7f4f09b78b80 1 -- 172.21.15.26:0/3405502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 msgr2=0x55737e94ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.228+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00cbd50 tx=0x7f4ee00d4a80 comp rx=0 tx=0).stop 2023-10-06T21:11:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4f09b78b80 1 -- 172.21.15.26:0/3405502876 shutdown_connections 2023-10-06T21:11:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e93ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/3405502876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94ae90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4f09b78b80 1 -- 172.21.15.26:0/3405502876 >> 172.21.15.26:0/3405502876 conn(0x55737e8b5200 msgr2=0x55737e93c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.234+0000 7f4eeffff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 0x7f4ed80a3d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009bf0 tx=0x7f4ee8009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.235+0000 7f4f09b78b80 1 -- 172.21.15.26:0/3405502876 shutdown_connections 2023-10-06T21:11:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.236+0000 7f4f09b78b80 1 -- 172.21.15.26:0/3405502876 wait complete. 2023-10-06T21:11:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.238+0000 7f4f09b78b80 1 -- start start 2023-10-06T21:11:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.239+0000 7f4f09b78b80 1 Processor -- start 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e956830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e893dd0 con 0x55737e93fc00 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8306b0 con 0x55737e940520 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e830350 con 0x55737e94b3d0 2023-10-06T21:11:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.240+0000 7f4f09b78b80 10 client.0 _ll_register_callbacks cb 0x55737e68d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.241+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.241+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40318/0 (socket says 172.21.15.26:40318) 2023-10-06T21:11:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.241+0000 7f4eef7fe700 1 -- 172.21.15.26:0/1217311919 learned_addr learned my addr 172.21.15.26:0/1217311919 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eeffff700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e956830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eef7fe700 1 -- 172.21.15.26:0/1217311919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 msgr2=0x55737e956830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e956830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eef7fe700 1 -- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 msgr2=0x55737e94bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.242+0000 7f4eef7fe700 1 -- 172.21.15.26:0/1217311919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01aa930 con 0x55737e93fc00 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0097f00 tx=0x7f4ee0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cb7aaa1497d5d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00cee50 con 0x55737e93fc00 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4ee009bb90 con 0x55737e93fc00 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55737e933d50 con 0x55737e93fc00 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee00977d0 con 0x55737e93fc00 2023-10-06T21:11:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4f09b78b80 10 client.15612 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.243+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55737e6a7c00 con 0x55737e93fc00 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55737e933d50 con 0x55737e93fc00 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4ee00051b0 con 0x55737e93fc00 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4f09b78b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4f09b78b80 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4f09b78b80 20 client.15612 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.244+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e96b8e0 0x55737e96dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.245+0000 7f4f09b78b80 4 client.15612 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.245+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55737e88cdb0 con 0x55737e96b8e0 2023-10-06T21:11:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.245+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e96b8e0 0x55737e96dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.247+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e96b8e0 0x55737e96dd00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9300400fa288f1 server_cookie=c91796cab2fe94cb in_seq=0 out_seq=0 2023-10-06T21:11:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.247+0000 7f4eceffd700 10 client.15612 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4eceffd700 1 -- 172.21.15.26:0/1217311919 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55737e88cdb0 con 0x55737e96b8e0 2023-10-06T21:11:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4eceffd700 10 client.15612 handle_command_reply: tid=0 2023-10-06T21:11:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 client.15612 shutdown 2023-10-06T21:11:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 msgr2=0x55737e94b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0097f00 tx=0x7f4ee0005c80 comp rx=0 tx=0).stop 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 shutdown_connections 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e956830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e94b7b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e96b8e0 0x55737e96dd00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/1217311919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.249+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 >> 172.21.15.26:0/1217311919 conn(0x55737e8b5200 msgr2=0x55737e93bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.250+0000 7f4eceffd700 0 client.15612 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.250+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 shutdown_connections 2023-10-06T21:11:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.250+0000 7f4f09b78b80 1 -- 172.21.15.26:0/1217311919 wait complete. 2023-10-06T21:11:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.251+0000 7f4f09b78b80 20 client.15612 trim_cache size 0 max 16384 2023-10-06T21:11:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.284+0000 7f4f09b78b80 1 Processor -- start 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.284+0000 7f4f09b78b80 1 -- start start 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e96d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e7148c0 con 0x55737e93fc00 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e93cdc0 con 0x55737e940520 2023-10-06T21:11:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.285+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e946430 con 0x55737e94b3d0 2023-10-06T21:11:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52956/0 (socket says 172.21.15.26:52956) 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 -- 172.21.15.26:0/2650461382 learned_addr learned my addr 172.21.15.26:0/2650461382 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeffff700 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e96d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 -- 172.21.15.26:0/2650461382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 msgr2=0x55737e94bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 -- 172.21.15.26:0/2650461382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 msgr2=0x55737e96d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e96d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.286+0000 7f4eeeffd700 1 -- 172.21.15.26:0/2650461382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e6e51f0 con 0x55737e940520 2023-10-06T21:11:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.287+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005b40 tx=0x7f4ee401cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7078aa9043c8e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.288+0000 7f4ed67fc700 1 -- 172.21.15.26:0/2650461382 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4017040 con 0x55737e940520 2023-10-06T21:11:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.288+0000 7f4ed67fc700 1 -- 172.21.15.26:0/2650461382 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee4020070 con 0x55737e940520 2023-10-06T21:11:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.288+0000 7f4ed67fc700 1 -- 172.21.15.26:0/2650461382 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40116a0 con 0x55737e940520 2023-10-06T21:11:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2650461382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 msgr2=0x55737e94b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005b40 tx=0x7f4ee401cbe0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2650461382 shutdown_connections 2023-10-06T21:11:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e94bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e93fc00 0x55737e96d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2650461382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e940520 0x55737e94b7b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.289+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2650461382 >> 172.21.15.26:0/2650461382 conn(0x55737e8b5200 msgr2=0x55737e944de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.291+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2650461382 shutdown_connections 2023-10-06T21:11:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.291+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2650461382 wait complete. 2023-10-06T21:11:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.292+0000 7f4f09b78b80 1 -- start start 2023-10-06T21:11:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.292+0000 7f4f09b78b80 1 Processor -- start 2023-10-06T21:11:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.294+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 0x55737e950760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.294+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e940520 0x55737e9baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4f09b78b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e9bafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9b7680 con 0x55737e940520 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9b77c0 con 0x55737e93fc00 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4f09b78b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e9b7980 con 0x55737e94b3d0 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4f09b78b80 10 client.0 _ll_register_callbacks cb 0x7f4ee4009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 0x55737e950760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e9bafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4eeffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e9bafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44684/0 (socket says 172.21.15.26:44684) 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.295+0000 7f4eeffff700 1 -- 172.21.15.26:0/2100176975 learned_addr learned my addr 172.21.15.26:0/2100176975 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e940520 0x55737e9baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eef7fe700 1 -- 172.21.15.26:0/2100176975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 msgr2=0x55737e9bafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e9bafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eef7fe700 1 -- 172.21.15.26:0/2100176975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e940520 msgr2=0x55737e9baaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e940520 0x55737e9baaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.296+0000 7f4eef7fe700 1 -- 172.21.15.26:0/2100176975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee4009c70 con 0x55737e93fc00 2023-10-06T21:11:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.297+0000 7f4eef7fe700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 0x55737e950760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4ee009b420 tx=0x7f4ee00d45c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4005261da3ff654b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.297+0000 7f4eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.297+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee009b260 con 0x55737e93fc00 2023-10-06T21:11:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.297+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4ee000bb60 con 0x55737e93fc00 2023-10-06T21:11:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.297+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee00cece0 con 0x55737e93fc00 2023-10-06T21:11:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee00977d0 con 0x55737e93fc00 2023-10-06T21:11:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 10 client.25739 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55737e6bd250 con 0x55737e93fc00 2023-10-06T21:11:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee00cece0 con 0x55737e93fc00 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4ee00051b0 con 0x55737e93fc00 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 10 client.25739 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 20 client.25739 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e9d0bc0 0x55737e9d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 4 client.25739 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.298+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55737e88cad0 con 0x55737e9d0bc0 2023-10-06T21:11:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.299+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e9d0bc0 0x55737e9d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.299+0000 7f4eeeffd700 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e9d0bc0 0x55737e9d2fb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efaff619d3d03983 server_cookie=49900461f5e2c9a in_seq=0 out_seq=0 2023-10-06T21:11:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.299+0000 7f4eceffd700 10 client.25739 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.338+0000 7f4eceffd700 1 -- 172.21.15.26:0/2100176975 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55737e88cad0 con 0x55737e9d0bc0 2023-10-06T21:11:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.338+0000 7f4eceffd700 10 client.25739 handle_command_reply: tid=0 2023-10-06T21:11:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.339+0000 7f4f09b78b80 1 client.25739 shutdown 2023-10-06T21:11:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.339+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 msgr2=0x55737e950760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.339+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 0x55737e950760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4ee009b420 tx=0x7f4ee00d45c0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 shutdown_connections 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55737e94b3d0 0x55737e9bafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55737e9d0bc0 0x55737e9d2fb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55737e93fc00 0x55737e950760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 --2- 172.21.15.26:0/2100176975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55737e940520 0x55737e9baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 >> 172.21.15.26:0/2100176975 conn(0x55737e8b5200 msgr2=0x55737e945920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4eceffd700 0 client.25739 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 shutdown_connections 2023-10-06T21:11:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 1 -- 172.21.15.26:0/2100176975 wait complete. 2023-10-06T21:11:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.340+0000 7f4f09b78b80 20 client.25739 trim_cache size 0 max 16384 2023-10-06T21:11:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:11:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:11:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:11:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:11:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.345+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 msgr2=0x7f4ed80a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.345+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 0x7f4ed80a3d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009bf0 tx=0x7f4ee8009040 comp rx=0 tx=0).stop 2023-10-06T21:11:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 msgr2=0x7f4ef010fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009810 tx=0x7f4ee000b5b0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 shutdown_connections 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ef010d020 0x7f4ef010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ef0103320 0x7f4ef010fce0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4ed80a1860 0x7f4ed80a3d20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 --2- 172.21.15.26:0/3713716962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ef0108440 0x7f4ef010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 >> 172.21.15.26:0/3713716962 conn(0x7f4ef00feba0 msgr2=0x7f4ef00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 shutdown_connections 2023-10-06T21:11:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:42.346+0000 7f4ef61ff700 1 -- 172.21.15.26:0/3713716962 wait complete. 2023-10-06T21:11:42.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:42 smithi167 ceph-mon[134494]: pgmap v44: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-06T21:11:42.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:42 smithi026 ceph-mon[98490]: pgmap v44: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-06T21:11:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:42 smithi078 ceph-mon[134896]: pgmap v44: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 23 MiB/s wr, 1.02k op/s 2023-10-06T21:11:44.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:43 smithi167 ceph-mon[134494]: pgmap v45: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-06T21:11:44.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:43 smithi026 ceph-mon[98490]: pgmap v45: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-06T21:11:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:43 smithi078 ceph-mon[134896]: pgmap v45: 97 pgs: 97 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 712 op/s 2023-10-06T21:11:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:46 smithi167 ceph-mon[134494]: pgmap v46: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 27 MiB/s wr, 1.28k op/s 2023-10-06T21:11:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:46 smithi026 ceph-mon[98490]: pgmap v46: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 27 MiB/s wr, 1.28k op/s 2023-10-06T21:11:46.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:46 smithi078 ceph-mon[134896]: pgmap v46: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 27 MiB/s wr, 1.28k op/s 2023-10-06T21:11:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 65.0 s, 2486.4 tps, lat 40.224 ms stddev 41.702 2023-10-06T21:11:48.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:11:48.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:48 smithi026 ceph-mon[98490]: pgmap v47: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:48.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:48 smithi167 ceph-mon[134494]: pgmap v47: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:48.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:48 smithi078 ceph-mon[134896]: pgmap v47: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.697+0000 7f35fe3ff700 1 -- 172.21.15.26:0/2784654793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f8107eb0 msgr2=0x7f35f8108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.697+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/2784654793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f8107eb0 0x7f35f8108290 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f35e800faf0 tx=0x7f35e800fe00 comp rx=0 tx=0).stop 2023-10-06T21:11:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 -- 172.21.15.26:0/2784654793 shutdown_connections 2023-10-06T21:11:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/2784654793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8114000 0x7f35f81163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/2784654793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f8107eb0 0x7f35f8108290 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/2784654793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f81087d0 0x7f35f8113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 -- 172.21.15.26:0/2784654793 >> 172.21.15.26:0/2784654793 conn(0x7f35f806d1a0 msgr2=0x7f35f806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 -- 172.21.15.26:0/2784654793 shutdown_connections 2023-10-06T21:11:48.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.699+0000 7f35fe3ff700 1 -- 172.21.15.26:0/2784654793 wait complete. 2023-10-06T21:11:48.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.701+0000 7f35fe3ff700 1 Processor -- start 2023-10-06T21:11:48.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.702+0000 7f35fe3ff700 1 -- start start 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8107eb0 0x7f35f819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f81087d0 0x7f35f819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81a2610 con 0x7f35f81087d0 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81a2780 con 0x7f35f8107eb0 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.703+0000 7f35fe3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81a2940 con 0x7f35f8114000 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8107eb0 0x7f35f819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44708/0 (socket says 172.21.15.26:44708) 2023-10-06T21:11:48.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 -- 172.21.15.26:0/3720791866 learned_addr learned my addr 172.21.15.26:0/3720791866 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:48.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 -- 172.21.15.26:0/3720791866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8107eb0 msgr2=0x7f35f819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8107eb0 0x7f35f819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 -- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f81087d0 msgr2=0x7f35f819db70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f81087d0 0x7f35f819db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.704+0000 7f35fc99c700 1 -- 172.21.15.26:0/3720791866 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec009520 con 0x7f35f8114000 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.705+0000 7f35fc99c700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35f000efc0 tx=0x7f35f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.705+0000 7f35f57fa700 1 -- 172.21.15.26:0/3720791866 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0018070 con 0x7f35f8114000 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.705+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e800f750 con 0x7f35f8114000 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.706+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f81b2a90 con 0x7f35f8114000 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.706+0000 7f35f57fa700 1 -- 172.21.15.26:0/3720791866 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35f000f040 con 0x7f35f8114000 2023-10-06T21:11:48.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.706+0000 7f35f57fa700 1 -- 172.21.15.26:0/3720791866 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f000cd70 con 0x7f35f8114000 2023-10-06T21:11:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.711+0000 7f35f57fa700 1 -- 172.21.15.26:0/3720791866 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) v1 ==== 136236+0+0 (secure 0 0 0) 0x7f35f0003a00 con 0x7f35f8114000 2023-10-06T21:11:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.713+0000 7f3611dd1b80 1 Processor -- start 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.713+0000 7f3611dd1b80 1 -- start start 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b58644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b5871c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b57cd460 con 0x55c6b58640c0 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b57bec20 con 0x55c6b58649e0 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b57bc540 con 0x55c6b586f890 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52992/0 (socket says 172.21.15.26:52992) 2023-10-06T21:11:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.714+0000 7f35f7fff700 1 -- 172.21.15.26:0/3464052506 learned_addr learned my addr 172.21.15.26:0/3464052506 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b5871c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b58644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f57fa700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 0x7f35e00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 -- 172.21.15.26:0/3464052506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 msgr2=0x55c6b586f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 -- 172.21.15.26:0/3464052506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 msgr2=0x55c6b58644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b58644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f77fe700 1 -- 172.21.15.26:0/3464052506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6b55e2de0 con 0x55c6b586f890 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.715+0000 7f35f57fa700 1 -- 172.21.15.26:0/3720791866 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f35f00ccce0 con 0x7f35f8114000 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.716+0000 7f35fc99c700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 0x7f35e00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.716+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b5871c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009e90 tx=0x7f35ec009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32cf08482115f43a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.716+0000 7f35de7fc700 1 -- 172.21.15.26:0/3464052506 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0130f0 con 0x55c6b586f890 2023-10-06T21:11:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.716+0000 7f35de7fc700 1 -- 172.21.15.26:0/3464052506 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ec0044f0 con 0x55c6b586f890 2023-10-06T21:11:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.716+0000 7f35de7fc700 1 -- 172.21.15.26:0/3464052506 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec003e70 con 0x55c6b586f890 2023-10-06T21:11:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3464052506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 msgr2=0x55c6b5871c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b5871c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009e90 tx=0x7f35ec009770 comp rx=0 tx=0).stop 2023-10-06T21:11:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3464052506 shutdown_connections 2023-10-06T21:11:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b5871c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b58644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3464052506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.717+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3464052506 >> 172.21.15.26:0/3464052506 conn(0x55c6b57d9260 msgr2=0x55c6b5860450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.718+0000 7f35fc99c700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 0x7f35e00a3c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35f00ca320 tx=0x7f35f0009de0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.718+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3464052506 shutdown_connections 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.718+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3464052506 wait complete. 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.719+0000 7f3611dd1b80 1 -- start start 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.719+0000 7f3611dd1b80 1 Processor -- start 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b586c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b586cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b57b7e30 con 0x55c6b58640c0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b5754710 con 0x55c6b58649e0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b57543b0 con 0x55c6b586f890 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f3611dd1b80 10 client.0 _ll_register_callbacks cb 0x55c6b55b1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52998/0 (socket says 172.21.15.26:52998) 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f35f7fff700 1 -- 172.21.15.26:0/1732927226 learned_addr learned my addr 172.21.15.26:0/1732927226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.720+0000 7f35f77fe700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b586c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f77fe700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b586cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f7fff700 1 -- 172.21.15.26:0/1732927226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 msgr2=0x55c6b586cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f7fff700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b586cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f7fff700 1 -- 172.21.15.26:0/1732927226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 msgr2=0x55c6b586c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f7fff700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b586c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.721+0000 7f35f7fff700 1 -- 172.21.15.26:0/1732927226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec013460 con 0x55c6b58649e0 2023-10-06T21:11:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.722+0000 7f35f7fff700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35e800f720 tx=0x7f35e8005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01e2c9db991360d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.722+0000 7f35d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.722+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e800f050 con 0x55c6b58649e0 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.722+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f35e800f1b0 con 0x55c6b58649e0 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.723+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6b5857f10 con 0x55c6b58649e0 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.724+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e800f370 con 0x55c6b58649e0 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.725+0000 7f3611dd1b80 10 client.25751 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:48.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.725+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c6b55cbc60 con 0x55c6b58649e0 2023-10-06T21:11:48.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.726+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6b5857f10 con 0x55c6b58649e0 2023-10-06T21:11:48.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.726+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f35e8027070 con 0x55c6b58649e0 2023-10-06T21:11:48.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.726+0000 7f3611dd1b80 10 client.25751 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:48.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.726+0000 7f3611dd1b80 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:48.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.727+0000 7f3611dd1b80 20 client.25751 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:48.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.727+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b587f340 0x55c6b5881760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.727+0000 7f3611dd1b80 4 client.25751 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:48.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.727+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6b57b0e10 con 0x55c6b587f340 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.727+0000 7f35f77fe700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b587f340 0x55c6b5881760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.728+0000 7f35d77fe700 10 client.25751 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.728+0000 7f35f77fe700 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b587f340 0x55c6b5881760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1483337811439f61 server_cookie=97bf72e655259154 in_seq=0 out_seq=0 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.729+0000 7f35d77fe700 1 -- 172.21.15.26:0/1732927226 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6b57b0e10 con 0x55c6b587f340 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f35d77fe700 10 client.25751 handle_command_reply: tid=0 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 client.25751 shutdown 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 msgr2=0x55c6b586cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f35e800f720 tx=0x7f35e8005c70 comp rx=0 tx=0).stop 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 shutdown_connections 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b586cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b58640c0 0x55c6b586c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b587f340 0x55c6b5881760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/1732927226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58649e0 0x55c6b586cab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 >> 172.21.15.26:0/1732927226 conn(0x55c6b57d9260 msgr2=0x55c6b5860040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.730+0000 7f35d77fe700 0 client.25751 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:48.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.731+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 shutdown_connections 2023-10-06T21:11:48.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.732+0000 7f3611dd1b80 1 -- 172.21.15.26:0/1732927226 wait complete. 2023-10-06T21:11:48.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.733+0000 7f3611dd1b80 20 client.25751 trim_cache size 0 max 16384 2023-10-06T21:11:48.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 Processor -- start 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 -- start start 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b5876440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b5867f60 0x55c6b5869d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b5638920 con 0x55c6b5867f60 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b58769b0 con 0x55c6b58640c0 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b585f240 con 0x55c6b586f890 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b5867f60 0x55c6b5869d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44730/0 (socket says 172.21.15.26:44730) 2023-10-06T21:11:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.751+0000 7f35f7fff700 1 -- 172.21.15.26:0/3953185118 learned_addr learned my addr 172.21.15.26:0/3953185118 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b5876440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 -- 172.21.15.26:0/3953185118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 msgr2=0x55c6b5876440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b5876440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 -- 172.21.15.26:0/3953185118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b5867f60 msgr2=0x55c6b5869d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b5867f60 0x55c6b5869d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 -- 172.21.15.26:0/3953185118 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6b5609250 con 0x55c6b586f890 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f77fe700 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b5876440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35f7fff700 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006010 tx=0x7f35e8005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7a4790b9fb5382 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35deffd700 1 -- 172.21.15.26:0/3953185118 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e80058e0 con 0x55c6b586f890 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35deffd700 1 -- 172.21.15.26:0/3953185118 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e8009e10 con 0x55c6b586f890 2023-10-06T21:11:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.752+0000 7f35deffd700 1 -- 172.21.15.26:0/3953185118 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8017420 con 0x55c6b586f890 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3953185118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 msgr2=0x55c6b58697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006010 tx=0x7f35e8005ce0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3953185118 shutdown_connections 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b5867f60 0x55c6b5869d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b5876440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/3953185118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b586f890 0x55c6b58697e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3953185118 >> 172.21.15.26:0/3953185118 conn(0x55c6b57d9260 msgr2=0x55c6b5867850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3953185118 shutdown_connections 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.753+0000 7f3611dd1b80 1 -- 172.21.15.26:0/3953185118 wait complete. 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.754+0000 7f3611dd1b80 1 -- start start 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.754+0000 7f3611dd1b80 1 Processor -- start 2023-10-06T21:11:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.754+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b58ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.754+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f3611dd1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 0x55c6b58c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b58c76e0 con 0x55c6b586f890 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b58c7850 con 0x55c6b58640c0 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f3611dd1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b58c7a10 con 0x55c6b5867f60 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f3611dd1b80 10 client.0 _ll_register_callbacks cb 0x55c6b55b1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b58ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 0x55c6b58c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44738/0 (socket says 172.21.15.26:44738) 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 -- 172.21.15.26:0/964049688 learned_addr learned my addr 172.21.15.26:0/964049688 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 -- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 msgr2=0x55c6b58ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b58ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 -- 172.21.15.26:0/964049688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 msgr2=0x55c6b58c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 0x55c6b58c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f7fff700 1 -- 172.21.15.26:0/964049688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6b55cbc60 con 0x55c6b5867f60 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.755+0000 7f35f77fe700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 0x55c6b58c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35f7fff700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35e800a010 tx=0x7f35e8004240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935664bb4c877360 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8033580 con 0x55c6b5867f60 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35ec0044f0 con 0x55c6b5867f60 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f35e8000e40 con 0x55c6b5867f60 2023-10-06T21:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e8005d50 con 0x55c6b5867f60 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f3611dd1b80 10 client.25747 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c6b55e12b0 con 0x55c6b5867f60 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.756+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ec0044f0 con 0x55c6b5867f60 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f35e8027040 con 0x55c6b5867f60 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 20 client.25747 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b58d4700 0x55c6b58d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 4 client.25747 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6b57b0b30 con 0x55c6b58d4700 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f35f77fe700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b58d4700 0x55c6b58d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f35f77fe700 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b58d4700 0x55c6b58d6b20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a5e44ecdc9ce64 server_cookie=e20046b17844fdc4 in_seq=0 out_seq=0 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.757+0000 7f35d77fe700 10 client.25747 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f35d77fe700 1 -- 172.21.15.26:0/964049688 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c6b57b0b30 con 0x55c6b58d4700 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f35d77fe700 10 client.25747 handle_command_reply: tid=0 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 client.25747 shutdown 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 msgr2=0x55c6b58cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35e800a010 tx=0x7f35e8004240 comp rx=0 tx=0).stop 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 shutdown_connections 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c6b586f890 0x55c6b58c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c6b58d4700 0x55c6b58d6b20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6b58640c0 0x55c6b58ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 --2- 172.21.15.26:0/964049688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6b5867f60 0x55c6b58cd1e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f35d77fe700 0 client.25747 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 >> 172.21.15.26:0/964049688 conn(0x55c6b57d9260 msgr2=0x55c6b5865f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 shutdown_connections 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 1 -- 172.21.15.26:0/964049688 wait complete. 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.758+0000 7f3611dd1b80 20 client.25747 trim_cache size 0 max 16384 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:11:48.769 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:11:48.770 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:11:48.770 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:11:48.771 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:11:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 msgr2=0x7f35e00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 0x7f35e00a3c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35f00ca320 tx=0x7f35f0009de0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 msgr2=0x7f35f81a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35f000efc0 tx=0x7f35f000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 shutdown_connections 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35f8114000 0x7f35f81a1fd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8107eb0 0x7f35f819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f35e00a1790 0x7f35e00a3c50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 --2- 172.21.15.26:0/3720791866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f81087d0 0x7f35f819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 >> 172.21.15.26:0/3720791866 conn(0x7f35f806d1a0 msgr2=0x7f35f806f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 shutdown_connections 2023-10-06T21:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:11:48.759+0000 7f35fe3ff700 1 -- 172.21.15.26:0/3720791866 wait complete. 2023-10-06T21:11:48.801 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:11:50.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:50 smithi167 ceph-mon[134494]: pgmap v48: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:50.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:50 smithi026 ceph-mon[98490]: pgmap v48: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:50.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:50 smithi078 ceph-mon[134896]: pgmap v48: 97 pgs: 97 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 MiB/s wr, 881 op/s 2023-10-06T21:11:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 70.0 s, 2708.7 tps, lat 36.366 ms stddev 36.406 2023-10-06T21:11:52.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:52 smithi167 ceph-mon[134494]: pgmap v49: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.25k op/s 2023-10-06T21:11:52.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:52 smithi167 ceph-mon[134494]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-10-06T21:11:52.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:52 smithi026 ceph-mon[98490]: pgmap v49: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.25k op/s 2023-10-06T21:11:52.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:52 smithi026 ceph-mon[98490]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-10-06T21:11:52.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:52 smithi078 ceph-mon[134896]: pgmap v49: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.25k op/s 2023-10-06T21:11:52.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:52 smithi078 ceph-mon[134896]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-10-06T21:11:54.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:53 smithi167 ceph-mon[134494]: pgmap v50: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 933 op/s 2023-10-06T21:11:54.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:53 smithi026 ceph-mon[98490]: pgmap v50: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 933 op/s 2023-10-06T21:11:54.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:53 smithi078 ceph-mon[134896]: pgmap v50: 97 pgs: 97 active+clean; 2.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 933 op/s 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 -- 172.21.15.26:0/4631239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 msgr2=0x7f725c069870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 --2- 172.21.15.26:0/4631239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c069870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72580099c0 tx=0x7f7258009cd0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 -- 172.21.15.26:0/4631239 shutdown_connections 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 --2- 172.21.15.26:0/4631239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 0x7f725c10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 --2- 172.21.15.26:0/4631239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c069870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 --2- 172.21.15.26:0/4631239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c069e40 0x7f725c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.586+0000 7f72643ff700 1 -- 172.21.15.26:0/4631239 >> 172.21.15.26:0/4631239 conn(0x7f725c076ea0 msgr2=0x7f725c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.587+0000 7f72643ff700 1 -- 172.21.15.26:0/4631239 shutdown_connections 2023-10-06T21:11:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.588+0000 7f72643ff700 1 -- 172.21.15.26:0/4631239 wait complete. 2023-10-06T21:11:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.590+0000 7f72643ff700 1 Processor -- start 2023-10-06T21:11:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.590+0000 7f72643ff700 1 -- start start 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 0x7f725c0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c072cf0 0x7f725c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c100bb0 con 0x7f725c072cf0 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1ae500 con 0x7f725c106220 2023-10-06T21:11:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f72643ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1ae640 con 0x7f725c069490 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42364/0 (socket says 172.21.15.26:42364) 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.591+0000 7f726219b700 1 -- 172.21.15.26:0/77973646 learned_addr learned my addr 172.21.15.26:0/77973646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726199a700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 0x7f725c0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726219b700 1 -- 172.21.15.26:0/77973646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 msgr2=0x7f725c0ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726219b700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 0x7f725c0ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726219b700 1 -- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c072cf0 msgr2=0x7f725c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726219b700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c072cf0 0x7f725c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.592+0000 7f726219b700 1 -- 172.21.15.26:0/77973646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580095f0 con 0x7f725c069490 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.593+0000 7f726219b700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f725800b580 tx=0x7f725800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.593+0000 7f72537fe700 1 -- 172.21.15.26:0/77973646 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725801d070 con 0x7f725c069490 2023-10-06T21:11:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.593+0000 7f72537fe700 1 -- 172.21.15.26:0/77973646 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7258005510 con 0x7f725c069490 2023-10-06T21:11:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.593+0000 7f72537fe700 1 -- 172.21.15.26:0/77973646 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258004b30 con 0x7f725c069490 2023-10-06T21:11:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.594+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c1ae870 con 0x7f725c069490 2023-10-06T21:11:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.594+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c1aebe0 con 0x7f725c069490 2023-10-06T21:11:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.599+0000 7f72537fe700 1 -- 172.21.15.26:0/77973646 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f725800fb80 con 0x7f725c069490 2023-10-06T21:11:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.600+0000 7f7277c81b80 1 Processor -- start 2023-10-06T21:11:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 -- start start 2023-10-06T21:11:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4bd9400 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4bcabc0 con 0x55ecd4c6fc00 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f7277c81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4bc84e0 con 0x55ecd4c70520 2023-10-06T21:11:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.601+0000 7f72537fe700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 0x7f72480a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.602+0000 7f72537fe700 1 -- 172.21.15.26:0/77973646 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f72580cdf60 con 0x7f725c069490 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.602+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.602+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42378/0 (socket says 172.21.15.26:42378) 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.602+0000 7f726219b700 1 -- 172.21.15.26:0/2575347557 learned_addr learned my addr 172.21.15.26:0/2575347557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 -- 172.21.15.26:0/2575347557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c6fc00 msgr2=0x55ecd4c6ffe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c6ffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 -- 172.21.15.26:0/2575347557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 msgr2=0x55ecd4c7d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 -- 172.21.15.26:0/2575347557 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd49eed80 con 0x55ecd4c70520 2023-10-06T21:11:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726219b700 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72580a2d50 tx=0x7f725809ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c3ae3264b7aa2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f726299c700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 0x7f72480a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.603+0000 7f7250bfe700 1 -- 172.21.15.26:0/2575347557 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725809af80 con 0x55ecd4c70520 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.604+0000 7f7250bfe700 1 -- 172.21.15.26:0/2575347557 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725809f0d0 con 0x55ecd4c70520 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.604+0000 7f7250bfe700 1 -- 172.21.15.26:0/2575347557 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725809b440 con 0x55ecd4c70520 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 -- 172.21.15.26:0/2575347557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 msgr2=0x55ecd4c7ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f726299c700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 0x7f72480a3bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f725400e2b0 tx=0x7f725400d040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72580a2d50 tx=0x7f725809ab90 comp rx=0 tx=0).stop 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 -- 172.21.15.26:0/2575347557 shutdown_connections 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c7d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 --2- 172.21.15.26:0/2575347557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c70520 0x55ecd4c7ae90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 -- 172.21.15.26:0/2575347557 >> 172.21.15.26:0/2575347557 conn(0x55ecd4be5200 msgr2=0x55ecd4c6c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 -- 172.21.15.26:0/2575347557 shutdown_connections 2023-10-06T21:11:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.605+0000 7f7277c81b80 1 -- 172.21.15.26:0/2575347557 wait complete. 2023-10-06T21:11:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.606+0000 7f7277c81b80 1 -- start start 2023-10-06T21:11:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.607+0000 7f7277c81b80 1 Processor -- start 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 0x55ecd4c84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4bc3dd0 con 0x55ecd4c70520 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4b606b0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4b60350 con 0x55ecd4c6fc00 2023-10-06T21:11:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.608+0000 7f7277c81b80 10 client.0 _ll_register_callbacks cb 0x55ecd49bd7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726299c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42392/0 (socket says 172.21.15.26:42392) 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 -- 172.21.15.26:0/3720107814 learned_addr learned my addr 172.21.15.26:0/3720107814 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 -- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c7b3d0 msgr2=0x55ecd4c8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726199a700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 0x55ecd4c84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 -- 172.21.15.26:0/3720107814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 msgr2=0x55ecd4c84500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 0x55ecd4c84500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726219b700 1 -- 172.21.15.26:0/3720107814 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c1aebe0 con 0x55ecd4c6fc00 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.609+0000 7f726199a700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 0x55ecd4c84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:11:55.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f726219b700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f725809b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf743839d077517 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f7245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72580a3280 con 0x55ecd4c6fc00 2023-10-06T21:11:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f725809af80 con 0x55ecd4c6fc00 2023-10-06T21:11:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecd4c63d50 con 0x55ecd4c6fc00 2023-10-06T21:11:55.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.610+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72580a2df0 con 0x55ecd4c6fc00 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 10 client.25759 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ecd49d7c00 con 0x55ecd4c6fc00 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ecd4c63d50 con 0x55ecd4c6fc00 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f725809d030 con 0x55ecd4c6fc00 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 10 client.25759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 20 client.25759 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4c7f090 0x55ecd4c814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 4 client.25759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecd4bbcdb0 con 0x55ecd4c7f090 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.611+0000 7f726199a700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4c7f090 0x55ecd4c814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.612+0000 7f726199a700 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4c7f090 0x55ecd4c814b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3626191125d7acc1 server_cookie=cda9ffea711c5cbc in_seq=0 out_seq=0 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.612+0000 7f7245ffb700 10 client.25759 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.613+0000 7f7245ffb700 1 -- 172.21.15.26:0/3720107814 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ecd4bbcdb0 con 0x55ecd4c7f090 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.613+0000 7f7245ffb700 10 client.25759 handle_command_reply: tid=0 2023-10-06T21:11:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.613+0000 7f7277c81b80 1 client.25759 shutdown 2023-10-06T21:11:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 msgr2=0x55ecd4c83fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f725809b540 comp rx=0 tx=0).stop 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 shutdown_connections 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4c7f090 0x55ecd4c814b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c83fc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3720107814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c70520 0x55ecd4c84500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.614+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 >> 172.21.15.26:0/3720107814 conn(0x55ecd4be5200 msgr2=0x55ecd4c6bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.615+0000 7f7245ffb700 0 client.25759 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.615+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 shutdown_connections 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.615+0000 7f7277c81b80 1 -- 172.21.15.26:0/3720107814 wait complete. 2023-10-06T21:11:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.615+0000 7f7277c81b80 20 client.25759 trim_cache size 0 max 16384 2023-10-06T21:11:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.638+0000 7f7277c81b80 1 Processor -- start 2023-10-06T21:11:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.638+0000 7f7277c81b80 1 -- start start 2023-10-06T21:11:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c74ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4a448c0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4c848f0 con 0x55ecd4c70520 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f7277c81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4c68860 con 0x55ecd4c6fc00 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.639+0000 7f726299c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c74ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726299c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c74ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38054/0 (socket says 172.21.15.26:38054) 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726299c700 1 -- 172.21.15.26:0/184838635 learned_addr learned my addr 172.21.15.26:0/184838635 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 -- 172.21.15.26:0/184838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 msgr2=0x55ecd4c74ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:11:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c74ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 -- 172.21.15.26:0/184838635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 msgr2=0x55ecd4c74ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c74ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 -- 172.21.15.26:0/184838635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd4c6aa80 con 0x55ecd4c6fc00 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.640+0000 7f726219b700 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c843b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72580992d0 tx=0x7f7258009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683b1ff762749734 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.641+0000 7f72513ff700 1 -- 172.21.15.26:0/184838635 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725809aac0 con 0x55ecd4c6fc00 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.641+0000 7f72513ff700 1 -- 172.21.15.26:0/184838635 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7258084440 con 0x55ecd4c6fc00 2023-10-06T21:11:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.641+0000 7f72513ff700 1 -- 172.21.15.26:0/184838635 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72580a3050 con 0x55ecd4c6fc00 2023-10-06T21:11:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.641+0000 7f7277c81b80 1 -- 172.21.15.26:0/184838635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 msgr2=0x55ecd4c843b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.641+0000 7f7277c81b80 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c843b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72580992d0 tx=0x7f7258009310 comp rx=0 tx=0).stop 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 -- 172.21.15.26:0/184838635 shutdown_connections 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c74ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c843b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 --2- 172.21.15.26:0/184838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 -- 172.21.15.26:0/184838635 >> 172.21.15.26:0/184838635 conn(0x55ecd4be5200 msgr2=0x55ecd4c83ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 -- 172.21.15.26:0/184838635 shutdown_connections 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.642+0000 7f7277c81b80 1 -- 172.21.15.26:0/184838635 wait complete. 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.643+0000 7f7277c81b80 1 -- start start 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.643+0000 7f7277c81b80 1 Processor -- start 2023-10-06T21:11:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4c87cc0 con 0x55ecd4c6fc00 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4cee650 con 0x55ecd4c70520 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4cee790 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f7277c81b80 10 client.0 _ll_register_callbacks cb 0x7f725809ade0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f726199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f726199a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c86ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33640/0 (socket says 172.21.15.26:33640) 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.644+0000 7f726199a700 1 -- 172.21.15.26:0/3013389973 learned_addr learned my addr 172.21.15.26:0/3013389973 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726219b700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 -- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 msgr2=0x55ecd4c86ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c86ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 -- 172.21.15.26:0/3013389973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c6fc00 msgr2=0x55ecd4c891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 -- 172.21.15.26:0/3013389973 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725807f070 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f726299c700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c87620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f725400ea00 tx=0x7f7254004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57129433ad6b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.645+0000 7f7245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254010040 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f725809f0d0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7254009020 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7254016e90 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7277c81b80 10 client.25765 fetch_fsmap learned FSMap version 29 2023-10-06T21:11:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ecd4a151f0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.646+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725809f0d0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f72540088a0 con 0x55ecd4c7b3d0 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 20 client.25765 populate_metadata read hostname 'smithi026' 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4ceffd0 0x55ecd4cf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 4 client.25765 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecd4bbcad0 con 0x55ecd4ceffd0 2023-10-06T21:11:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.647+0000 7f726219b700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4ceffd0 0x55ecd4cf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:11:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.650+0000 7f726219b700 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4ceffd0 0x55ecd4cf23f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61a6334a4685940 server_cookie=aca9fab7e4715f7d in_seq=0 out_seq=0 2023-10-06T21:11:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.651+0000 7f7245ffb700 10 client.25765 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.688+0000 7f7245ffb700 1 -- 172.21.15.26:0/3013389973 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecd4bbcad0 con 0x55ecd4ceffd0 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.688+0000 7f7245ffb700 10 client.25765 handle_command_reply: tid=0 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.688+0000 7f7277c81b80 1 client.25765 shutdown 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.690+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 msgr2=0x55ecd4c87620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.690+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c87620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f725400ea00 tx=0x7f7254004ad0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 shutdown_connections 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ecd4c7b3d0 0x55ecd4c87620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecd4c6fc00 0x55ecd4c891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ecd4ceffd0 0x55ecd4cf23f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 --2- 172.21.15.26:0/3013389973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecd4c70520 0x55ecd4c86ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 >> 172.21.15.26:0/3013389973 conn(0x55ecd4be5200 msgr2=0x55ecd4c78dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7245ffb700 0 client.25765 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 shutdown_connections 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.692+0000 7f7277c81b80 1 -- 172.21.15.26:0/3013389973 wait complete. 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:11:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.693+0000 7f7277c81b80 20 client.25765 trim_cache size 0 max 16384 2023-10-06T21:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.693+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 msgr2=0x7f72480a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.693+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 0x7f72480a3bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f725400e2b0 tx=0x7f725400d040 comp rx=0 tx=0).stop 2023-10-06T21:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.693+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 msgr2=0x7f725c072760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:11:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.693+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f725800b580 tx=0x7f725800f690 comp rx=0 tx=0).stop 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 shutdown_connections 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72480a16f0 0x7f72480a3bb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f725c072cf0 0x7f725c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c069490 0x7f725c072760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 --2- 172.21.15.26:0/77973646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f725c106220 0x7f725c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 >> 172.21.15.26:0/77973646 conn(0x7f725c076ea0 msgr2=0x7f725c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 shutdown_connections 2023-10-06T21:11:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:11:55.694+0000 7f72643ff700 1 -- 172.21.15.26:0/77973646 wait complete. 2023-10-06T21:11:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:56 smithi167 ceph-mon[134494]: pgmap v51: 97 pgs: 97 active+clean; 2.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.56k op/s 2023-10-06T21:11:56.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:56 smithi026 ceph-mon[98490]: pgmap v51: 97 pgs: 97 active+clean; 2.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.56k op/s 2023-10-06T21:11:56.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:56 smithi078 ceph-mon[134896]: pgmap v51: 97 pgs: 97 active+clean; 2.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.56k op/s 2023-10-06T21:11:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 75.0 s, 2450.4 tps, lat 41.439 ms stddev 65.579 2023-10-06T21:11:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:11:58 smithi167 ceph-mon[134494]: pgmap v52: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:11:58.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:11:58 smithi026 ceph-mon[98490]: pgmap v52: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:11:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:11:58 smithi078 ceph-mon[134896]: pgmap v52: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:12:00.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:00 smithi167 ceph-mon[134494]: pgmap v53: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:12:00.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:00 smithi026 ceph-mon[98490]: pgmap v53: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:12:00.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:00 smithi078 ceph-mon[134896]: pgmap v53: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 40 MiB/s wr, 1.88k op/s 2023-10-06T21:12:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 80.0 s, 2725.5 tps, lat 35.929 ms stddev 36.367 2023-10-06T21:12:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:02 smithi167 ceph-mon[134494]: pgmap v54: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 MiB/s wr, 2.20k op/s 2023-10-06T21:12:02.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:02 smithi026 ceph-mon[98490]: pgmap v54: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 MiB/s wr, 2.20k op/s 2023-10-06T21:12:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:02 smithi078 ceph-mon[134896]: pgmap v54: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 MiB/s wr, 2.20k op/s 2023-10-06T21:12:04.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:03 smithi167 ceph-mon[134494]: pgmap v55: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-10-06T21:12:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:03 smithi026 ceph-mon[98490]: pgmap v55: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-10-06T21:12:04.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:03 smithi078 ceph-mon[134896]: pgmap v55: 97 pgs: 97 active+clean; 2.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-10-06T21:12:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:06 smithi167 ceph-mon[134494]: pgmap v56: 97 pgs: 97 active+clean; 2.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 48 MiB/s wr, 2.68k op/s 2023-10-06T21:12:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:06 smithi026 ceph-mon[98490]: pgmap v56: 97 pgs: 97 active+clean; 2.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 48 MiB/s wr, 2.68k op/s 2023-10-06T21:12:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:06 smithi078 ceph-mon[134896]: pgmap v56: 97 pgs: 97 active+clean; 2.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 48 MiB/s wr, 2.68k op/s 2023-10-06T21:12:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 85.0 s, 2648.5 tps, lat 38.591 ms stddev 64.633 2023-10-06T21:12:08.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:08 smithi167 ceph-mon[134494]: pgmap v57: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 2.45k op/s 2023-10-06T21:12:08.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:08 smithi026 ceph-mon[98490]: pgmap v57: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 2.45k op/s 2023-10-06T21:12:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:08 smithi078 ceph-mon[134896]: pgmap v57: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 2.45k op/s 2023-10-06T21:12:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.933+0000 7fdc331ff700 1 -- 172.21.15.26:0/97092549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0fef20 msgr2=0x7fdc2c0ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.933+0000 7fdc331ff700 1 --2- 172.21.15.26:0/97092549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c0ff3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b0a0 tx=0x7fdc1400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 -- 172.21.15.26:0/97092549 shutdown_connections 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 --2- 172.21.15.26:0/97092549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0ff8e0 0x7fdc2c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 --2- 172.21.15.26:0/97092549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 --2- 172.21.15.26:0/97092549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c0ff3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 -- 172.21.15.26:0/97092549 >> 172.21.15.26:0/97092549 conn(0x7fdc2c076e70 msgr2=0x7fdc2c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 -- 172.21.15.26:0/97092549 shutdown_connections 2023-10-06T21:12:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.934+0000 7fdc331ff700 1 -- 172.21.15.26:0/97092549 wait complete. 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.935+0000 7fdc331ff700 1 Processor -- start 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.935+0000 7fdc331ff700 1 -- start start 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0ff8e0 0x7fdc2c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c19c290 con 0x7fdc2c105eb0 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c19c400 con 0x7fdc2c0ff8e0 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc331ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c19c5c0 con 0x7fdc2c0fef20 2023-10-06T21:12:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc30f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc30f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43472/0 (socket says 172.21.15.26:43472) 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc30f9b700 1 -- 172.21.15.26:0/2529868008 learned_addr learned my addr 172.21.15.26:0/2529868008 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0ff8e0 0x7fdc2c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.936+0000 7fdc3179c700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.937+0000 7fdc3179c700 1 -- 172.21.15.26:0/2529868008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 msgr2=0x7fdc2c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.937+0000 7fdc3179c700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.937+0000 7fdc3179c700 1 -- 172.21.15.26:0/2529868008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0ff8e0 msgr2=0x7fdc2c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.937+0000 7fdc3179c700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0ff8e0 0x7fdc2c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.937+0000 7fdc3179c700 1 -- 172.21.15.26:0/2529868008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009d00 con 0x7fdc2c105eb0 2023-10-06T21:12:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.938+0000 7fdc3179c700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c19bc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400e970 tx=0x7fdc2400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.938+0000 7fdc2a7fc700 1 -- 172.21.15.26:0/2529868008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24004220 con 0x7fdc2c105eb0 2023-10-06T21:12:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.938+0000 7fdc2a7fc700 1 -- 172.21.15.26:0/2529868008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc24004cf0 con 0x7fdc2c105eb0 2023-10-06T21:12:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.938+0000 7fdc2a7fc700 1 -- 172.21.15.26:0/2529868008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24005720 con 0x7fdc2c105eb0 2023-10-06T21:12:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.938+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c19c8a0 con 0x7fdc2c105eb0 2023-10-06T21:12:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.939+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c100aa0 con 0x7fdc2c105eb0 2023-10-06T21:12:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.944+0000 7fdc2a7fc700 1 -- 172.21.15.26:0/2529868008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdc24004750 con 0x7fdc2c105eb0 2023-10-06T21:12:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.945+0000 7fdc46c08b80 1 Processor -- start 2023-10-06T21:12:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.946+0000 7fdc2a7fc700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 0x7fdc0c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.946+0000 7fdc2a7fc700 1 -- 172.21.15.26:0/2529868008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc24014070 con 0x7fdc2c105eb0 2023-10-06T21:12:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.947+0000 7fdc23fff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 0x7fdc0c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.947+0000 7fdc46c08b80 1 -- start start 2023-10-06T21:12:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.947+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118017c00 0x55d118017fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d11802dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f81400 con 0x55d11802bbe0 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f72bc0 con 0x55d118018520 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f704e0 con 0x55d118017c00 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 0x7fdc0c0a3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009fd0 tx=0x7fdc14006210 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52976/0 (socket says 172.21.15.26:52976) 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 -- 172.21.15.26:0/723061100 learned_addr learned my addr 172.21.15.26:0/723061100 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc3179c700 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d11802dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118017c00 0x55d118017fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 -- 172.21.15.26:0/723061100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118017c00 msgr2=0x55d118017fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118017c00 0x55d118017fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 -- 172.21.15.26:0/723061100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 msgr2=0x55d11802dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d11802dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.948+0000 7fdc23fff700 1 -- 172.21.15.26:0/723061100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d117d96d80 con 0x55d118018520 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.950+0000 7fdc23fff700 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b0a0 tx=0x7fdc1401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abc8afd35a2500b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.951+0000 7fdc22ffd700 1 -- 172.21.15.26:0/723061100 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400e040 con 0x55d118018520 2023-10-06T21:12:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.951+0000 7fdc22ffd700 1 -- 172.21.15.26:0/723061100 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc140079a0 con 0x55d118018520 2023-10-06T21:12:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.951+0000 7fdc22ffd700 1 -- 172.21.15.26:0/723061100 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1401c440 con 0x55d118018520 2023-10-06T21:12:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.952+0000 7fdc46c08b80 1 -- 172.21.15.26:0/723061100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 msgr2=0x55d11802b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.952+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b0a0 tx=0x7fdc1401c000 comp rx=0 tx=0).stop 2023-10-06T21:12:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.953+0000 7fdc46c08b80 1 -- 172.21.15.26:0/723061100 shutdown_connections 2023-10-06T21:12:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.953+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d11802dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.953+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118017c00 0x55d118017fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.953+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/723061100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118018520 0x55d11802b6a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.953+0000 7fdc46c08b80 1 -- 172.21.15.26:0/723061100 >> 172.21.15.26:0/723061100 conn(0x55d117f8d200 msgr2=0x55d118014080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.954+0000 7fdc46c08b80 1 -- 172.21.15.26:0/723061100 shutdown_connections 2023-10-06T21:12:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.954+0000 7fdc46c08b80 1 -- 172.21.15.26:0/723061100 wait complete. 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.954+0000 7fdc46c08b80 1 -- start start 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.954+0000 7fdc46c08b80 1 Processor -- start 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d118017c00 0x55d118028950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 0x55d1180293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f6bdd0 con 0x55d118017c00 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f086b0 con 0x55d11802bbe0 2023-10-06T21:12:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117f08350 con 0x55d118018520 2023-10-06T21:12:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.955+0000 7fdc46c08b80 10 client.0 _ll_register_callbacks cb 0x55d117d657a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc3179c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 0x55d1180293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43490/0 (socket says 172.21.15.26:43490) 2023-10-06T21:12:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc23fff700 1 -- 172.21.15.26:0/4028984095 learned_addr learned my addr 172.21.15.26:0/4028984095 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc3179c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 0x55d1180293d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52978/0 (socket says 172.21.15.26:52978) 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.956+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d118017c00 0x55d118028950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 -- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 msgr2=0x55d1180293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 0x55d1180293d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 -- 172.21.15.26:0/4028984095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d118017c00 msgr2=0x55d118028950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d118017c00 0x55d118028950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 -- 172.21.15.26:0/4028984095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c10e5d0 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc23fff700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b3f0 tx=0x7fdc1400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee582c0a0446663 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.957+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400e070 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11800bd50 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc14022040 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc140050c0 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc46c08b80 10 client.25777 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d117d7fc00 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d11800bd50 con 0x55d118018520 2023-10-06T21:12:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdc14027430 con 0x55d118018520 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc46c08b80 10 client.25777 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.958+0000 7fdc46c08b80 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.959+0000 7fdc46c08b80 20 client.25777 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.959+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d11803b470 0x55d11803d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.959+0000 7fdc46c08b80 4 client.25777 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.959+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d117f64db0 con 0x55d11803b470 2023-10-06T21:12:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.959+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d11803b470 0x55d11803d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.960+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d11803b470 0x55d11803d890 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bd10e6104ff3e8 server_cookie=c3436217ecafba12 in_seq=0 out_seq=0 2023-10-06T21:12:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.960+0000 7fdc1bfff700 10 client.25777 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.961+0000 7fdc1bfff700 1 -- 172.21.15.26:0/4028984095 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d117f64db0 con 0x55d11803b470 2023-10-06T21:12:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.961+0000 7fdc1bfff700 10 client.25777 handle_command_reply: tid=0 2023-10-06T21:12:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.961+0000 7fdc46c08b80 1 client.25777 shutdown 2023-10-06T21:12:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.962+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 msgr2=0x55d118028e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.962+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400b3f0 tx=0x7fdc1400b760 comp rx=0 tx=0).stop 2023-10-06T21:12:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 shutdown_connections 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d11803b470 0x55d11803d890 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d11802bbe0 0x55d1180293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d118017c00 0x55d118028950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4028984095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118028e90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 >> 172.21.15.26:0/4028984095 conn(0x55d117f8d200 msgr2=0x55d118013c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc1bfff700 0 client.25777 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 shutdown_connections 2023-10-06T21:12:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4028984095 wait complete. 2023-10-06T21:12:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.963+0000 7fdc46c08b80 20 client.25777 trim_cache size 0 max 16384 2023-10-06T21:12:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 Processor -- start 2023-10-06T21:12:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 -- start start 2023-10-06T21:12:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118024180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118025bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117dec8c0 con 0x55d11802bbe0 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d118010860 con 0x55d118017c00 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.988+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d118014dc0 con 0x55d118018520 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52984/0 (socket says 172.21.15.26:52984) 2023-10-06T21:12:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 -- 172.21.15.26:0/4148728194 learned_addr learned my addr 172.21.15.26:0/4148728194 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc23fff700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118024180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc3179c700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118025bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 -- 172.21.15.26:0/4148728194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 msgr2=0x55d118024180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118024180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 -- 172.21.15.26:0/4148728194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 msgr2=0x55d118025bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118025bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 -- 172.21.15.26:0/4148728194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d117dbd1f0 con 0x55d118017c00 2023-10-06T21:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc23fff700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118024180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.989+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d11803e130 tx=0x7fdc1c004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cb7ef2e6cc8357 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.990+0000 7fdc237fe700 1 -- 172.21.15.26:0/4148728194 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c017040 con 0x55d118017c00 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.990+0000 7fdc237fe700 1 -- 172.21.15.26:0/4148728194 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1c022020 con 0x55d118017c00 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.990+0000 7fdc237fe700 1 -- 172.21.15.26:0/4148728194 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c003ea0 con 0x55d118017c00 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4148728194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 msgr2=0x55d118023c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d11803e130 tx=0x7fdc1c004320 comp rx=0 tx=0).stop 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4148728194 shutdown_connections 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118025bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d118023c40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/4148728194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d118024180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4148728194 >> 172.21.15.26:0/4148728194 conn(0x55d117f8d200 msgr2=0x55d118036e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4148728194 shutdown_connections 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.991+0000 7fdc46c08b80 1 -- 172.21.15.26:0/4148728194 wait complete. 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.992+0000 7fdc46c08b80 1 -- start start 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.992+0000 7fdc46c08b80 1 Processor -- start 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.992+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d1180323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118030c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d118032cc0 con 0x55d11802bbe0 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d118031160 con 0x55d118017c00 2023-10-06T21:12:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1180312d0 con 0x55d118018520 2023-10-06T21:12:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc46c08b80 10 client.0 _ll_register_callbacks cb 0x7fdc240961c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43500/0 (socket says 172.21.15.26:43500) 2023-10-06T21:12:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 -- 172.21.15.26:0/1368598119 learned_addr learned my addr 172.21.15.26:0/1368598119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc3179c700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118030c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 -- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 msgr2=0x55d1180323b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:12:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d1180323b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 -- 172.21.15.26:0/1368598119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 msgr2=0x55d118030c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118030c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.993+0000 7fdc23fff700 1 -- 172.21.15.26:0/1368598119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c00bd80 con 0x55d118018520 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc23fff700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d010 tx=0x7fdc140078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13220f1e75954f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400e040 con 0x55d118018520 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc140079a0 con 0x55d118018520 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc140074a0 con 0x55d118018520 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc14005040 con 0x55d118018520 2023-10-06T21:12:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc46c08b80 10 client.25780 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.994+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d117d95250 con 0x55d118018520 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc140079a0 con 0x55d118018520 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdc1404b070 con 0x55d118018520 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 10 client.25780 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 20 client.25780 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d118090140 0x55d118092560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 4 client.25780 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d117f64ad0 con 0x55d118090140 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.995+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d118090140 0x55d118092560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.996+0000 7fdc30f9b700 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d118090140 0x55d118092560 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb1e144b1b48dc1 server_cookie=94d655d033e0e3fc in_seq=0 out_seq=0 2023-10-06T21:12:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:08.996+0000 7fdc1bfff700 10 client.25780 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.008+0000 7fdc1bfff700 1 -- 172.21.15.26:0/1368598119 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d117f64ad0 con 0x55d118090140 2023-10-06T21:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.008+0000 7fdc1bfff700 10 client.25780 handle_command_reply: tid=0 2023-10-06T21:12:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.008+0000 7fdc46c08b80 1 client.25780 shutdown 2023-10-06T21:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.009+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 msgr2=0x55d1180306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.009+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400d010 tx=0x7fdc140078e0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 shutdown_connections 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d11802bbe0 0x55d118030c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d118090140 0x55d118092560 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d118017c00 0x55d1180323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 --2- 172.21.15.26:0/1368598119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d118018520 0x55d1180306e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.010+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 >> 172.21.15.26:0/1368598119 conn(0x55d117f8d200 msgr2=0x55d118023c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.011+0000 7fdc1bfff700 0 client.25780 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.011+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 shutdown_connections 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.011+0000 7fdc46c08b80 1 -- 172.21.15.26:0/1368598119 wait complete. 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.011+0000 7fdc46c08b80 20 client.25780 trim_cache size 0 max 16384 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:12:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:12:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.012+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 msgr2=0x7fdc0c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.012+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 0x7fdc0c0a3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009fd0 tx=0x7fdc14006210 comp rx=0 tx=0).stop 2023-10-06T21:12:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.012+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 msgr2=0x7fdc2c19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.012+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c19bc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400e970 tx=0x7fdc2400ec80 comp rx=0 tx=0).stop 2023-10-06T21:12:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 shutdown_connections 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdc0c0a1670 0x7fdc0c0a3b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2c105eb0 0x7fdc2c19bc50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc2c0fef20 0x7fdc2c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 --2- 172.21.15.26:0/2529868008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0ff8e0 0x7fdc2c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 >> 172.21.15.26:0/2529868008 conn(0x7fdc2c076e70 msgr2=0x7fdc2c103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 shutdown_connections 2023-10-06T21:12:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:09.013+0000 7fdc331ff700 1 -- 172.21.15.26:0/2529868008 wait complete. 2023-10-06T21:12:10.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:10 smithi167 ceph-mon[134494]: pgmap v58: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-10-06T21:12:10.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:10 smithi026 ceph-mon[98490]: pgmap v58: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-10-06T21:12:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:10 smithi078 ceph-mon[134896]: pgmap v58: 97 pgs: 97 active+clean; 2.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-10-06T21:12:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 90.0 s, 2850.5 tps, lat 35.040 ms stddev 38.396 2023-10-06T21:12:12.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:12 smithi167 ceph-mon[134494]: pgmap v59: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 30 MiB/s wr, 2.02k op/s 2023-10-06T21:12:12.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:12 smithi026 ceph-mon[98490]: pgmap v59: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 30 MiB/s wr, 2.02k op/s 2023-10-06T21:12:12.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:12 smithi078 ceph-mon[134896]: pgmap v59: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 30 MiB/s wr, 2.02k op/s 2023-10-06T21:12:14.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:13 smithi167 ceph-mon[134494]: pgmap v60: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2023-10-06T21:12:14.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:13 smithi026 ceph-mon[98490]: pgmap v60: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2023-10-06T21:12:14.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:13 smithi078 ceph-mon[134896]: pgmap v60: 97 pgs: 97 active+clean; 2.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 1.70k op/s 2023-10-06T21:12:16.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:16 smithi167 ceph-mon[134494]: pgmap v61: 97 pgs: 97 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-10-06T21:12:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:16 smithi026 ceph-mon[98490]: pgmap v61: 97 pgs: 97 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-10-06T21:12:16.642 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:16 smithi078 ceph-mon[134896]: pgmap v61: 97 pgs: 97 active+clean; 2.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2023-10-06T21:12:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 95.0 s, 2630.2 tps, lat 37.957 ms stddev 47.823 2023-10-06T21:12:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:18 smithi167 ceph-mon[134494]: pgmap v62: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-10-06T21:12:18.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:18 smithi026 ceph-mon[98490]: pgmap v62: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-10-06T21:12:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:18 smithi078 ceph-mon[134896]: pgmap v62: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-10-06T21:12:18.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 -- 172.21.15.26:0/1422936117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 msgr2=0x7f81940ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1422936117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f81940ff460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8184009990 tx=0x7f8184009ca0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 -- 172.21.15.26:0/1422936117 shutdown_connections 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1422936117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8194104220 0x7f81941046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1422936117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f81940ff460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1422936117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81940ff9a0 0x7f8194103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.144+0000 7f819b5ff700 1 -- 172.21.15.26:0/1422936117 >> 172.21.15.26:0/1422936117 conn(0x7f8194076ed0 msgr2=0x7f81940fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.145+0000 7f819b5ff700 1 -- 172.21.15.26:0/1422936117 shutdown_connections 2023-10-06T21:12:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.145+0000 7f819b5ff700 1 -- 172.21.15.26:0/1422936117 wait complete. 2023-10-06T21:12:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.146+0000 7f819b5ff700 1 Processor -- start 2023-10-06T21:12:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.146+0000 7f819b5ff700 1 -- start start 2023-10-06T21:12:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81940ff9a0 0x7f8194197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8194104220 0x7f819419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419c000 con 0x7f8194104220 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419c170 con 0x7f81940ff080 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819b5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419c330 con 0x7f81940ff9a0 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f8199b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8194104220 0x7f819419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54516/0 (socket says 172.21.15.26:54516) 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f819939b700 1 -- 172.21.15.26:0/1749457572 learned_addr learned my addr 172.21.15.26:0/1749457572 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.147+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81940ff9a0 0x7f8194197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 -- 172.21.15.26:0/1749457572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81940ff9a0 msgr2=0x7f8194197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81940ff9a0 0x7f8194197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 -- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8194104220 msgr2=0x7f819419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8194104220 0x7f819419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 -- 172.21.15.26:0/1749457572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840095f0 con 0x7f81940ff080 2023-10-06T21:12:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.148+0000 7f819939b700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81840095c0 tx=0x7f818400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.150+0000 7f818a7fc700 1 -- 172.21.15.26:0/1749457572 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818401d070 con 0x7f81940ff080 2023-10-06T21:12:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.150+0000 7f818a7fc700 1 -- 172.21.15.26:0/1749457572 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8184005050 con 0x7f81940ff080 2023-10-06T21:12:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.150+0000 7f818a7fc700 1 -- 172.21.15.26:0/1749457572 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184004a90 con 0x7f81940ff080 2023-10-06T21:12:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.151+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819419c5b0 con 0x7f81940ff080 2023-10-06T21:12:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.151+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194107070 con 0x7f81940ff080 2023-10-06T21:12:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.155+0000 7f818a7fc700 1 -- 172.21.15.26:0/1749457572 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8184005430 con 0x7f81940ff080 2023-10-06T21:12:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.157+0000 7f818a7fc700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 0x7f81800a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.158+0000 7f81aef64b80 1 Processor -- start 2023-10-06T21:12:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.158+0000 7f818a7fc700 1 -- 172.21.15.26:0/1749457572 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f81840cdee0 con 0x7f81940ff080 2023-10-06T21:12:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.158+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 0x7f81800a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.158+0000 7f81aef64b80 1 -- start start 2023-10-06T21:12:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd20c0 0x56129ccd24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccdd890 0x56129ccdfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cc3b460 con 0x56129ccdd890 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cc2cc20 con 0x56129ccd20c0 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f81aef64b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cc2a540 con 0x56129ccd29e0 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 0x7f81800a3c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8194198630 tx=0x7f819000b410 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54724/0 (socket says 172.21.15.26:54724) 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 -- 172.21.15.26:0/3039414392 learned_addr learned my addr 172.21.15.26:0/3039414392 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f819939b700 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd20c0 0x56129ccd24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 -- 172.21.15.26:0/3039414392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd20c0 msgr2=0x56129ccd24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd20c0 0x56129ccd24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 -- 172.21.15.26:0/3039414392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccdd890 msgr2=0x56129ccdfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccdd890 0x56129ccdfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.159+0000 7f8198b9a700 1 -- 172.21.15.26:0/3039414392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56129ca50de0 con 0x56129ccd29e0 2023-10-06T21:12:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.165+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8190010f70 tx=0x7f8190013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1829b36752dc2cdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.165+0000 7f817f7fe700 1 -- 172.21.15.26:0/3039414392 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190011e60 con 0x56129ccd29e0 2023-10-06T21:12:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.165+0000 7f817f7fe700 1 -- 172.21.15.26:0/3039414392 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8190015070 con 0x56129ccd29e0 2023-10-06T21:12:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.165+0000 7f817f7fe700 1 -- 172.21.15.26:0/3039414392 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819001d630 con 0x56129ccd29e0 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 -- 172.21.15.26:0/3039414392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 msgr2=0x56129ccdd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8190010f70 tx=0x7f8190013000 comp rx=0 tx=0).stop 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 -- 172.21.15.26:0/3039414392 shutdown_connections 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccdd890 0x56129ccdfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd20c0 0x56129ccd24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3039414392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd29e0 0x56129ccdd350 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 -- 172.21.15.26:0/3039414392 >> 172.21.15.26:0/3039414392 conn(0x56129cc47260 msgr2=0x56129ccce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 -- 172.21.15.26:0/3039414392 shutdown_connections 2023-10-06T21:12:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.166+0000 7f81aef64b80 1 -- 172.21.15.26:0/3039414392 wait complete. 2023-10-06T21:12:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.167+0000 7f81aef64b80 1 -- start start 2023-10-06T21:12:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.167+0000 7f81aef64b80 1 Processor -- start 2023-10-06T21:12:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129ccda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129ccdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cc25e30 con 0x56129ccd20c0 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cbc2710 con 0x56129ccd29e0 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cbc23b0 con 0x56129ccdd890 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f81aef64b80 10 client.0 _ll_register_callbacks cb 0x56129ca1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54526/0 (socket says 172.21.15.26:54526) 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f8198b9a700 1 -- 172.21.15.26:0/1534355741 learned_addr learned my addr 172.21.15.26:0/1534355741 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.168+0000 7f819939b700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129ccda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8199b9c700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129ccdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 -- 172.21.15.26:0/1534355741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 msgr2=0x56129ccdb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129ccdb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 -- 172.21.15.26:0/1534355741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 msgr2=0x56129ccda5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129ccda5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 -- 172.21.15.26:0/1534355741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8190010c20 con 0x56129ccd29e0 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f819939b700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129ccda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:12:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8199b9c700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129ccdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:12:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8190005950 tx=0x7f8190012f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1523bc79679289 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f817cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.169+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819002b070 con 0x56129ccd29e0 2023-10-06T21:12:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.170+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56129ccc5f10 con 0x56129ccd29e0 2023-10-06T21:12:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.170+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8190017070 con 0x56129ccd29e0 2023-10-06T21:12:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.170+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f819000bee0 con 0x56129ccd29e0 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.170+0000 7f81aef64b80 10 client.25796 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.170+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56129ca39c60 con 0x56129ccd29e0 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819003e020 con 0x56129ccd29e0 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f8190024020 con 0x56129ccd29e0 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 10 client.25796 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 20 client.25796 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129ccfe140 0x56129cd00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 4 client.25796 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.171+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56129cc1ee10 con 0x56129ccfe140 2023-10-06T21:12:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.175+0000 7f819939b700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129ccfe140 0x56129cd00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.175+0000 7f819939b700 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129ccfe140 0x56129cd00580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b54a10574de22d7 server_cookie=365b847d8b413fd5 in_seq=0 out_seq=0 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.176+0000 7f817cff9700 10 client.25796 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f817cff9700 1 -- 172.21.15.26:0/1534355741 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56129cc1ee10 con 0x56129ccfe140 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f817cff9700 10 client.25796 handle_command_reply: tid=0 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 client.25796 shutdown 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 msgr2=0x56129ccdaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8190005950 tx=0x7f8190012f50 comp rx=0 tx=0).stop 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 shutdown_connections 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129ccdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129ccda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129ccfe140 0x56129cd00580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1534355741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129ccdaaf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 >> 172.21.15.26:0/1534355741 conn(0x56129cc47260 msgr2=0x56129cccdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f817cff9700 0 client.25796 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.177+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 shutdown_connections 2023-10-06T21:12:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.178+0000 7f81aef64b80 1 -- 172.21.15.26:0/1534355741 wait complete. 2023-10-06T21:12:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.178+0000 7f81aef64b80 20 client.25796 trim_cache size 0 max 16384 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 Processor -- start 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 -- start start 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 0x56129ccd4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 0x56129cce4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129caa6920 con 0x56129ccd29e0 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129ccc5b00 con 0x56129ccdd890 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.207+0000 7f81aef64b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cce4940 con 0x56129ccd20c0 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8199b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 0x56129ccd4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8199b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54534/0 (socket says 172.21.15.26:54534) 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8199b9c700 1 -- 172.21.15.26:0/1170736563 learned_addr learned my addr 172.21.15.26:0/1170736563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 0x56129cce4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 -- 172.21.15.26:0/1170736563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 msgr2=0x56129ccdae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 -- 172.21.15.26:0/1170736563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 msgr2=0x56129cce4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 0x56129cce4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 -- 172.21.15.26:0/1170736563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56129ca77250 con 0x56129ccd20c0 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8199b9c700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f8198b9a700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 0x56129cce4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.208+0000 7f819939b700 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 0x56129ccd4f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81840cb5e0 tx=0x7f8184009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7829925c0212e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.211+0000 7f817ffff700 1 -- 172.21.15.26:0/1170736563 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818409f050 con 0x56129ccd20c0 2023-10-06T21:12:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.211+0000 7f817ffff700 1 -- 172.21.15.26:0/1170736563 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81840a3200 con 0x56129ccd20c0 2023-10-06T21:12:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.211+0000 7f817ffff700 1 -- 172.21.15.26:0/1170736563 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184004bf0 con 0x56129ccd20c0 2023-10-06T21:12:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 -- 172.21.15.26:0/1170736563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 msgr2=0x56129ccd4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 0x56129ccd4f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81840cb5e0 tx=0x7f8184009ca0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 -- 172.21.15.26:0/1170736563 shutdown_connections 2023-10-06T21:12:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccdd890 0x56129ccdae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccd20c0 0x56129ccd4f60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 --2- 172.21.15.26:0/1170736563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd29e0 0x56129cce4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 -- 172.21.15.26:0/1170736563 >> 172.21.15.26:0/1170736563 conn(0x56129cc47260 msgr2=0x56129ccd58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 -- 172.21.15.26:0/1170736563 shutdown_connections 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.212+0000 7f81aef64b80 1 -- 172.21.15.26:0/1170736563 wait complete. 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.213+0000 7f81aef64b80 1 -- start start 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.213+0000 7f81aef64b80 1 Processor -- start 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129cce25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129cd50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cce3230 con 0x56129ccd20c0 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cd514d0 con 0x56129ccd29e0 2023-10-06T21:12:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56129cd51640 con 0x56129ccdd890 2023-10-06T21:12:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f81aef64b80 10 client.0 _ll_register_callbacks cb 0x56129ca1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f8198b9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54544/0 (socket says 172.21.15.26:54544) 2023-10-06T21:12:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f8198b9a700 1 -- 172.21.15.26:0/3391869414 learned_addr learned my addr 172.21.15.26:0/3391869414 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.214+0000 7f8199b9c700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129cd50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 -- 172.21.15.26:0/3391869414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 msgr2=0x56129cd50f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129cd50f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 -- 172.21.15.26:0/3391869414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 msgr2=0x56129cce25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129cce25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 -- 172.21.15.26:0/3391869414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c003b00 con 0x56129ccd29e0 2023-10-06T21:12:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f8198b9a700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8190005d90 tx=0x7f819001e950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1830aaf8658eefcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.215+0000 7f817cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819002b030 con 0x56129ccd29e0 2023-10-06T21:12:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f819002c070 con 0x56129ccd29e0 2023-10-06T21:12:19.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8190015070 con 0x56129ccd29e0 2023-10-06T21:12:19.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f819000be60 con 0x56129ccd29e0 2023-10-06T21:12:19.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f81aef64b80 10 client.25802 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.216+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8194107070 con 0x56129ccd29e0 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8190015070 con 0x56129ccd29e0 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f81900117d0 con 0x56129ccd29e0 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 10 client.25802 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 20 client.25802 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129cd52970 0x56129cd54d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 4 client.25802 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:12:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56129cc1eb30 con 0x56129cd52970 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.217+0000 7f819939b700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129cd52970 0x56129cd54d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f819939b700 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129cd52970 0x56129cd54d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bfe6314443d8663 server_cookie=601870d548cf4762 in_seq=0 out_seq=0 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f817cff9700 10 client.25802 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f817cff9700 1 -- 172.21.15.26:0/3391869414 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56129cc1eb30 con 0x56129cd52970 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f817cff9700 10 client.25802 handle_command_reply: tid=0 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f81aef64b80 1 client.25802 shutdown 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 msgr2=0x56129cce2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.218+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8190005d90 tx=0x7f819001e950 comp rx=0 tx=0).stop 2023-10-06T21:12:19.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 shutdown_connections 2023-10-06T21:12:19.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56129ccdd890 0x56129cd50f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56129ccd20c0 0x56129cce25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56129cd52970 0x56129cd54d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 --2- 172.21.15.26:0/3391869414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56129ccd29e0 0x56129cce2b20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 >> 172.21.15.26:0/3391869414 conn(0x56129cc47260 msgr2=0x56129ccd8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.219+0000 7f817cff9700 0 client.25802 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.221+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 shutdown_connections 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.221+0000 7f81aef64b80 1 -- 172.21.15.26:0/3391869414 wait complete. 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f81aef64b80 20 client.25802 trim_cache size 0 max 16384 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:12:19.227 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:12:19.228 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 msgr2=0x7f81800a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 0x7f81800a3c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8194198630 tx=0x7f819000b410 comp rx=0 tx=0).stop 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 msgr2=0x7f8194197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81840095c0 tx=0x7f818400f690 comp rx=0 tx=0).stop 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 shutdown_connections 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81800a17c0 0x7f81800a3c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8194104220 0x7f819419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81940ff080 0x7f8194197020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.223+0000 7f819b5ff700 1 --2- 172.21.15.26:0/1749457572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81940ff9a0 0x7f8194197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:19.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.224+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 >> 172.21.15.26:0/1749457572 conn(0x7f8194076ed0 msgr2=0x7f8194101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:19.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.224+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 shutdown_connections 2023-10-06T21:12:19.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:19.224+0000 7f819b5ff700 1 -- 172.21.15.26:0/1749457572 wait complete. 2023-10-06T21:12:19.258 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:12:20.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:20 smithi167 ceph-mon[134494]: pgmap v63: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 777 op/s 2023-10-06T21:12:20.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:20 smithi026 ceph-mon[98490]: pgmap v63: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 777 op/s 2023-10-06T21:12:20.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:20 smithi078 ceph-mon[134896]: pgmap v63: 97 pgs: 97 active+clean; 2.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 777 op/s 2023-10-06T21:12:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 100.0 s, 2689.8 tps, lat 37.232 ms stddev 38.064 2023-10-06T21:12:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.263+0000 7fd3251ff700 1 -- 172.21.15.26:0/186824806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320073250 msgr2=0x7fd320073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.263+0000 7fd3251ff700 1 --2- 172.21.15.26:0/186824806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320073250 0x7fd320073630 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd314009990 tx=0x7fd314009ca0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 -- 172.21.15.26:0/186824806 shutdown_connections 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 --2- 172.21.15.26:0/186824806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320111e60 0x7fd320114250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 --2- 172.21.15.26:0/186824806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320073250 0x7fd320073630 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 --2- 172.21.15.26:0/186824806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd320111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 -- 172.21.15.26:0/186824806 >> 172.21.15.26:0/186824806 conn(0x7fd3200fc940 msgr2=0x7fd3200fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 -- 172.21.15.26:0/186824806 shutdown_connections 2023-10-06T21:12:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.264+0000 7fd3251ff700 1 -- 172.21.15.26:0/186824806 wait complete. 2023-10-06T21:12:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.265+0000 7fd3251ff700 1 Processor -- start 2023-10-06T21:12:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 -- start start 2023-10-06T21:12:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 0x7fd32019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd32019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320111e60 0x7fd3201a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3201a2920 con 0x7fd320073b70 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3201a2a90 con 0x7fd320111e60 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.266+0000 7fd3251ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3201a2c50 con 0x7fd320073250 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.267+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd32019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.267+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd32019dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55462/0 (socket says 172.21.15.26:55462) 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.267+0000 7fd31e59c700 1 -- 172.21.15.26:0/1237124610 learned_addr learned my addr 172.21.15.26:0/1237124610 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.267+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 0x7fd32019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.267+0000 7fd31f59e700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320111e60 0x7fd3201a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 -- 172.21.15.26:0/1237124610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320111e60 msgr2=0x7fd3201a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320111e60 0x7fd3201a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 -- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 msgr2=0x7fd32019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd32019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 -- 172.21.15.26:0/1237124610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140095f0 con 0x7fd320073250 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.268+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 0x7fd32019d770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.269+0000 7fd307fff700 1 -- 172.21.15.26:0/1237124610 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31401d070 con 0x7fd320073250 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.269+0000 7fd307fff700 1 -- 172.21.15.26:0/1237124610 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd314005580 con 0x7fd320073250 2023-10-06T21:12:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.269+0000 7fd307fff700 1 -- 172.21.15.26:0/1237124610 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31400fd40 con 0x7fd320073250 2023-10-06T21:12:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.269+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3201a2ed0 con 0x7fd320073250 2023-10-06T21:12:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.269+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3201aa600 con 0x7fd320073250 2023-10-06T21:12:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.274+0000 7fd307fff700 1 -- 172.21.15.26:0/1237124610 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd31400f460 con 0x7fd320073250 2023-10-06T21:12:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.275+0000 7fd338ad1b80 1 Processor -- start 2023-10-06T21:12:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.276+0000 7fd338ad1b80 1 -- start start 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.276+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec6333fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec63417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec629d400 con 0x55dec6333c00 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec628ebc0 con 0x55dec6334520 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec628c4e0 con 0x55dec633f3d0 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd31ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd31ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43536/0 (socket says 172.21.15.26:43536) 2023-10-06T21:12:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd31ed9d700 1 -- 172.21.15.26:0/59714427 learned_addr learned my addr 172.21.15.26:0/59714427 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd31e59c700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec63417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.277+0000 7fd31f59e700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec6333fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31ed9d700 1 -- 172.21.15.26:0/59714427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 msgr2=0x55dec63417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec63417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31ed9d700 1 -- 172.21.15.26:0/59714427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 msgr2=0x55dec6333fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec6333fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31ed9d700 1 -- 172.21.15.26:0/59714427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec60b2d80 con 0x55dec6334520 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31e59c700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec63417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:22.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.278+0000 7fd31f59e700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec6333fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3140cb7d0 tx=0x7fd3140d61b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb848eb8095d3d51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd307fff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 0x7fd3080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd3057fa700 1 -- 172.21.15.26:0/59714427 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3140d6930 con 0x55dec6334520 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd3057fa700 1 -- 172.21.15.26:0/59714427 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3140ce570 con 0x55dec6334520 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd3057fa700 1 -- 172.21.15.26:0/59714427 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3140dc660 con 0x55dec6334520 2023-10-06T21:12:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.280+0000 7fd307fff700 1 -- 172.21.15.26:0/1237124610 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd3140ce100 con 0x7fd320073250 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.281+0000 7fd31f59e700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 0x7fd3080a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 -- 172.21.15.26:0/59714427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 msgr2=0x55dec633ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3140cb7d0 tx=0x7fd3140d61b0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd31f59e700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 0x7fd3080a3b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd32006a4d0 tx=0x7fd310008040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 -- 172.21.15.26:0/59714427 shutdown_connections 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec63417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec6333fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/59714427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec633ee90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 -- 172.21.15.26:0/59714427 >> 172.21.15.26:0/59714427 conn(0x55dec62a9200 msgr2=0x55dec6330080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.282+0000 7fd338ad1b80 1 -- 172.21.15.26:0/59714427 shutdown_connections 2023-10-06T21:12:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.283+0000 7fd338ad1b80 1 -- 172.21.15.26:0/59714427 wait complete. 2023-10-06T21:12:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.283+0000 7fd338ad1b80 1 -- start start 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.284+0000 7fd338ad1b80 1 Processor -- start 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.284+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6333c00 0x55dec633ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.284+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.284+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec6287dd0 con 0x55dec633f3d0 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec62246b0 con 0x55dec6333c00 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec6224350 con 0x55dec6334520 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd338ad1b80 10 client.0 _ll_register_callbacks cb 0x55dec60817a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34622/0 (socket says 172.21.15.26:34622) 2023-10-06T21:12:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.285+0000 7fd31e59c700 1 -- 172.21.15.26:0/443656635 learned_addr learned my addr 172.21.15.26:0/443656635 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6333c00 0x55dec633ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31f59e700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31e59c700 1 -- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6333c00 msgr2=0x55dec633ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31e59c700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6333c00 0x55dec633ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31e59c700 1 -- 172.21.15.26:0/443656635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 msgr2=0x55dec633c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31e59c700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.286+0000 7fd31e59c700 1 -- 172.21.15.26:0/443656635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3201aa600 con 0x55dec6334520 2023-10-06T21:12:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.287+0000 7fd31e59c700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd318007cf0 tx=0x7fd3180104a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187537c20d6abed1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.287+0000 7fd302ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:12:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.287+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800e070 con 0x55dec6334520 2023-10-06T21:12:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.287+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd318006e40 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dec6327d50 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd318009e50 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 10 client.25804 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dec609bc00 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dec6327d50 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd31801e3f0 con 0x55dec6334520 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 20 client.25804 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec635fcb0 0x55dec63620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 4 client.25804 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.288+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dec6280db0 con 0x55dec635fcb0 2023-10-06T21:12:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.289+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec635fcb0 0x55dec63620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.290+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec635fcb0 0x55dec63620d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac726d4a672a2e1 server_cookie=633129ffd4b59058 in_seq=0 out_seq=0 2023-10-06T21:12:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.290+0000 7fd302ff5700 10 client.25804 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.290+0000 7fd302ff5700 1 -- 172.21.15.26:0/443656635 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dec6280db0 con 0x55dec635fcb0 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.290+0000 7fd302ff5700 10 client.25804 handle_command_reply: tid=0 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.291+0000 7fd338ad1b80 1 client.25804 shutdown 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.291+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 msgr2=0x55dec633bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.291+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd318007cf0 tx=0x7fd3180104a0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 shutdown_connections 2023-10-06T21:12:22.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec635fcb0 0x55dec63620d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6333c00 0x55dec633ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/443656635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6334520 0x55dec633bfd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 >> 172.21.15.26:0/443656635 conn(0x55dec62a9200 msgr2=0x55dec632fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd302ff5700 0 client.25804 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 shutdown_connections 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 1 -- 172.21.15.26:0/443656635 wait complete. 2023-10-06T21:12:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.292+0000 7fd338ad1b80 20 client.25804 trim_cache size 0 max 16384 2023-10-06T21:12:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.314+0000 7fd338ad1b80 1 Processor -- start 2023-10-06T21:12:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.314+0000 7fd338ad1b80 1 -- start start 2023-10-06T21:12:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6333c00 0x55dec633bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec61088c0 con 0x55dec633f3d0 2023-10-06T21:12:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec633a530 con 0x55dec6334520 2023-10-06T21:12:22.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec6337e20 con 0x55dec6333c00 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd31ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6333c00 0x55dec633bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43558/0 (socket says 172.21.15.26:43558) 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd31e59c700 1 -- 172.21.15.26:0/3935674732 learned_addr learned my addr 172.21.15.26:0/3935674732 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.315+0000 7fd31f59e700 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.316+0000 7fd31e59c700 1 -- 172.21.15.26:0/3935674732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6333c00 msgr2=0x55dec633bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.316+0000 7fd31e59c700 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6333c00 0x55dec633bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.316+0000 7fd31e59c700 1 -- 172.21.15.26:0/3935674732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 msgr2=0x55dec633b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.316+0000 7fd31e59c700 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.316+0000 7fd31e59c700 1 -- 172.21.15.26:0/3935674732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dec60d91f0 con 0x55dec6334520 2023-10-06T21:12:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.317+0000 7fd31e59c700 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd318008010 tx=0x7fd3180108a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f9246e81ab696f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.318+0000 7fd305ffb700 1 -- 172.21.15.26:0/3935674732 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800e070 con 0x55dec6334520 2023-10-06T21:12:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.318+0000 7fd305ffb700 1 -- 172.21.15.26:0/3935674732 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd31800f040 con 0x55dec6334520 2023-10-06T21:12:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.318+0000 7fd305ffb700 1 -- 172.21.15.26:0/3935674732 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3180156a0 con 0x55dec6334520 2023-10-06T21:12:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.318+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3935674732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 msgr2=0x55dec6356d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.318+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd318008010 tx=0x7fd3180108a0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3935674732 shutdown_connections 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec633f3d0 0x55dec633b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec6333c00 0x55dec633bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3935674732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec6356d30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3935674732 >> 172.21.15.26:0/3935674732 conn(0x55dec62a9200 msgr2=0x55dec63669f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3935674732 shutdown_connections 2023-10-06T21:12:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3935674732 wait complete. 2023-10-06T21:12:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.319+0000 7fd338ad1b80 1 -- start start 2023-10-06T21:12:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 Processor -- start 2023-10-06T21:12:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec63564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec63569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec6352d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec63533b0 con 0x55dec6333c00 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec63534f0 con 0x55dec6334520 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dec63536b0 con 0x55dec633f3d0 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd338ad1b80 10 client.0 _ll_register_callbacks cb 0x7fd318007630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec63569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd31e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec63569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43560/0 (socket says 172.21.15.26:43560) 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd31e59c700 1 -- 172.21.15.26:0/3947130558 learned_addr learned my addr 172.21.15.26:0/3947130558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.320+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec63564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec6352d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 -- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 msgr2=0x55dec63569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec63569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 -- 172.21.15.26:0/3947130558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 msgr2=0x55dec63564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec63564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.321+0000 7fd31f59e700 1 -- 172.21.15.26:0/3947130558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dec609bc00 con 0x55dec633f3d0 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.322+0000 7fd31f59e700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec6352d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55dec632a240 tx=0x7fd310004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f6a80273468fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.322+0000 7fd302ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:12:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.323+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310015e80 con 0x55dec633f3d0 2023-10-06T21:12:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.323+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd310011e40 con 0x55dec633f3d0 2023-10-06T21:12:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.323+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3140ce570 con 0x55dec633f3d0 2023-10-06T21:12:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd310005a00 con 0x55dec633f3d0 2023-10-06T21:12:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd338ad1b80 10 client.25810 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dec60b1250 con 0x55dec633f3d0 2023-10-06T21:12:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3140ce570 con 0x55dec633f3d0 2023-10-06T21:12:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fd310006020 con 0x55dec633f3d0 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd338ad1b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd338ad1b80 10 client.25810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.324+0000 7fd338ad1b80 20 client.25810 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec63b4c40 0x55dec63b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd338ad1b80 4 client.25810 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dec6280ad0 con 0x55dec63b4c40 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec63b4c40 0x55dec63b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd31ed9d700 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec63b4c40 0x55dec63b7060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b369c254fee7e3 server_cookie=6f47296d14c67ae5 in_seq=0 out_seq=0 2023-10-06T21:12:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.325+0000 7fd302ff5700 10 client.25810 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.336+0000 7fd302ff5700 1 -- 172.21.15.26:0/3947130558 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dec6280ad0 con 0x55dec63b4c40 2023-10-06T21:12:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.336+0000 7fd302ff5700 10 client.25810 handle_command_reply: tid=0 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 client.25810 shutdown 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 msgr2=0x55dec6352d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec6352d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55dec632a240 tx=0x7fd310004770 comp rx=0 tx=0).stop 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 shutdown_connections 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dec633f3d0 0x55dec6352d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dec6333c00 0x55dec63564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55dec63b4c40 0x55dec63b7060 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 --2- 172.21.15.26:0/3947130558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dec6334520 0x55dec63569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd302ff5700 0 client.25810 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 >> 172.21.15.26:0/3947130558 conn(0x55dec62a9200 msgr2=0x55dec6335a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 shutdown_connections 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 1 -- 172.21.15.26:0/3947130558 wait complete. 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.337+0000 7fd338ad1b80 20 client.25810 trim_cache size 0 max 16384 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:12:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:12:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.339+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 msgr2=0x7fd3080a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.339+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 0x7fd3080a3b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd32006a4d0 tx=0x7fd310008040 comp rx=0 tx=0).stop 2023-10-06T21:12:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.339+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 msgr2=0x7fd32019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.339+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 0x7fd32019d770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140037c0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 shutdown_connections 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd320111e60 0x7fd3201a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd320073250 0x7fd32019d770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd3080a1670 0x7fd3080a3b30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 --2- 172.21.15.26:0/1237124610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd320073b70 0x7fd32019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 >> 172.21.15.26:0/1237124610 conn(0x7fd3200fc940 msgr2=0x7fd3200fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 shutdown_connections 2023-10-06T21:12:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:22.340+0000 7fd3251ff700 1 -- 172.21.15.26:0/1237124610 wait complete. 2023-10-06T21:12:22.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:22 smithi167 ceph-mon[134494]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.30k op/s 2023-10-06T21:12:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:22 smithi026 ceph-mon[98490]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.30k op/s 2023-10-06T21:12:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:22 smithi078 ceph-mon[134896]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 1.30k op/s 2023-10-06T21:12:24.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:23 smithi167 ceph-mon[134494]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 843 op/s 2023-10-06T21:12:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:23 smithi026 ceph-mon[98490]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 843 op/s 2023-10-06T21:12:24.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:23 smithi078 ceph-mon[134896]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 843 op/s 2023-10-06T21:12:26.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:26 smithi167 ceph-mon[134494]: pgmap v66: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-06T21:12:26.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:26 smithi026 ceph-mon[98490]: pgmap v66: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-06T21:12:26.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:26 smithi078 ceph-mon[134896]: pgmap v66: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2023-10-06T21:12:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 105.0 s, 2488.4 tps, lat 38.900 ms stddev 40.451 2023-10-06T21:12:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:28 smithi167 ceph-mon[134494]: pgmap v67: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 983 op/s 2023-10-06T21:12:28.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:28 smithi026 ceph-mon[98490]: pgmap v67: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 983 op/s 2023-10-06T21:12:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:28 smithi078 ceph-mon[134896]: pgmap v67: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 983 op/s 2023-10-06T21:12:29.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:29 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:29.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:29 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:29.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:29 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: pgmap v68: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 847 op/s 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:30 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: pgmap v68: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 847 op/s 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:30 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: pgmap v68: 97 pgs: 97 active+clean; 2.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 847 op/s 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:30 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 110.0 s, 2303.4 tps, lat 44.861 ms stddev 65.728 2023-10-06T21:12:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:32 smithi167 ceph-mon[134494]: pgmap v69: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 MiB/s wr, 2.32k op/s 2023-10-06T21:12:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:32 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:32 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:32 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:32 smithi026 ceph-mon[98490]: pgmap v69: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 MiB/s wr, 2.32k op/s 2023-10-06T21:12:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:32 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:32 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:32 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:32.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:32 smithi078 ceph-mon[134896]: pgmap v69: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 MiB/s wr, 2.32k op/s 2023-10-06T21:12:32.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:32 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:12:32.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:32 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:12:32.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:32 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:12:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:33 smithi167 ceph-mon[134494]: pgmap v70: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-10-06T21:12:34.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:33 smithi026 ceph-mon[98490]: pgmap v70: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-10-06T21:12:34.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:33 smithi078 ceph-mon[134896]: pgmap v70: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2023-10-06T21:12:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 -- 172.21.15.26:0/1768477321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd080fef20 msgr2=0x7fdd080ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 --2- 172.21.15.26:0/1768477321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd080fef20 0x7fdd080ff3a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800dd50 tx=0x7fdcf800f330 comp rx=0 tx=0).stop 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 -- 172.21.15.26:0/1768477321 shutdown_connections 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 --2- 172.21.15.26:0/1768477321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd0810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 --2- 172.21.15.26:0/1768477321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd08105eb0 0x7fdd08106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 --2- 172.21.15.26:0/1768477321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd080fef20 0x7fdd080ff3a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.581+0000 7fdd0efff700 1 -- 172.21.15.26:0/1768477321 >> 172.21.15.26:0/1768477321 conn(0x7fdd08076e70 msgr2=0x7fdd080fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.583+0000 7fdd0efff700 1 -- 172.21.15.26:0/1768477321 shutdown_connections 2023-10-06T21:12:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.583+0000 7fdd0efff700 1 -- 172.21.15.26:0/1768477321 wait complete. 2023-10-06T21:12:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.584+0000 7fdd0efff700 1 Processor -- start 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.585+0000 7fdd0efff700 1 -- start start 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.585+0000 7fdd0efff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.585+0000 7fdd0efff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd081a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.585+0000 7fdd0efff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd08105eb0 0x7fdd0819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0efff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819de00 con 0x7fdd080ff8e0 2023-10-06T21:12:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0efff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819df70 con 0x7fdd080fef20 2023-10-06T21:12:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0efff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819e130 con 0x7fdd08105eb0 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0cd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0cd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35996/0 (socket says 172.21.15.26:35996) 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0cd9b700 1 -- 172.21.15.26:0/12612903 learned_addr learned my addr 172.21.15.26:0/12612903 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.586+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd08105eb0 0x7fdd0819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdcfffff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd081a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 -- 172.21.15.26:0/12612903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd08105eb0 msgr2=0x7fdd0819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd08105eb0 0x7fdd0819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 -- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 msgr2=0x7fdd081a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd081a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 -- 172.21.15.26:0/12612903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf800da00 con 0x7fdd080fef20 2023-10-06T21:12:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdcfffff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd081a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:12:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.587+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400b840 tx=0x7fdd0400bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.588+0000 7fdcfdffb700 1 -- 172.21.15.26:0/12612903 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400f840 con 0x7fdd080fef20 2023-10-06T21:12:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.588+0000 7fdcfdffb700 1 -- 172.21.15.26:0/12612903 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0400fe60 con 0x7fdd080fef20 2023-10-06T21:12:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.588+0000 7fdcfdffb700 1 -- 172.21.15.26:0/12612903 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400d3d0 con 0x7fdd080fef20 2023-10-06T21:12:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.588+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0819e410 con 0x7fdd080fef20 2023-10-06T21:12:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.589+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd081aa7c0 con 0x7fdd080fef20 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.596+0000 7fdd229f8b80 1 Processor -- start 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 -- start start 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b15ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8a63400 con 0x5576a8af9c00 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8a54bc0 con 0x5576a8afa520 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.597+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8a524e0 con 0x5576a8b163e0 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.598+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.598+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57184/0 (socket says 172.21.15.26:57184) 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.598+0000 7fdd0d59c700 1 -- 172.21.15.26:0/2879046992 learned_addr learned my addr 172.21.15.26:0/2879046992 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.601+0000 7fdcfdffb700 1 -- 172.21.15.26:0/12612903 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdd0400f9a0 con 0x7fdd080fef20 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.602+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b15ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.603+0000 7fdd0d59c700 1 -- 172.21.15.26:0/2879046992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 msgr2=0x5576a8b187d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.603+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.603+0000 7fdd0d59c700 1 -- 172.21.15.26:0/2879046992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 msgr2=0x5576a8b15ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.603+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b15ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.603+0000 7fdd0d59c700 1 -- 172.21.15.26:0/2879046992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a8878d80 con 0x5576a8af9c00 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.604+0000 7fdcfdffb700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 0x7fdcf00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.604+0000 7fdcfdffb700 1 -- 172.21.15.26:0/12612903 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdd040cce80 con 0x7fdd080fef20 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.604+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098a0 tx=0x7fdd00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3519b8ab8322e5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.608+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 0x7fdcf00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.609+0000 7fdceeffd700 1 -- 172.21.15.26:0/2879046992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd000306e0 con 0x5576a8af9c00 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.609+0000 7fdceeffd700 1 -- 172.21.15.26:0/2879046992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd00030d00 con 0x5576a8af9c00 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.609+0000 7fdceeffd700 1 -- 172.21.15.26:0/2879046992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00035de0 con 0x5576a8af9c00 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.609+0000 7fdd0cd9b700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 0x7fdcf00a3cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd04099da0 tx=0x7fdd040178f0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.610+0000 7fdd229f8b80 1 -- 172.21.15.26:0/2879046992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 msgr2=0x5576a8af9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.610+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098a0 tx=0x7fdd00009bb0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 -- 172.21.15.26:0/2879046992 shutdown_connections 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8af9fe0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/2879046992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b15ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 -- 172.21.15.26:0/2879046992 >> 172.21.15.26:0/2879046992 conn(0x5576a8a6f200 msgr2=0x5576a8af6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.614+0000 7fdd229f8b80 1 -- 172.21.15.26:0/2879046992 shutdown_connections 2023-10-06T21:12:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.615+0000 7fdd229f8b80 1 -- 172.21.15.26:0/2879046992 wait complete. 2023-10-06T21:12:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.616+0000 7fdd229f8b80 1 -- start start 2023-10-06T21:12:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.616+0000 7fdd229f8b80 1 Processor -- start 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b09630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8a4ddd0 con 0x5576a8af9c00 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a89ea6b0 con 0x5576a8afa520 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a89ea350 con 0x5576a8b163e0 2023-10-06T21:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.617+0000 7fdd229f8b80 10 client.0 _ll_register_callbacks cb 0x5576a88477a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.618+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b09630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.618+0000 7fdcfffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.618+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.618+0000 7fdcfffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36026/0 (socket says 172.21.15.26:36026) 2023-10-06T21:12:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.618+0000 7fdcfffff700 1 -- 172.21.15.26:0/1011828408 learned_addr learned my addr 172.21.15.26:0/1011828408 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 -- 172.21.15.26:0/1011828408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 msgr2=0x5576a8b04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 -- 172.21.15.26:0/1011828408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 msgr2=0x5576a8b09630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b09630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 -- 172.21.15.26:0/1011828408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0002f040 con 0x5576a8afa520 2023-10-06T21:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005b40 tx=0x7fdcf80038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0a12e5b896eef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.620+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8012040 con 0x5576a8afa520 2023-10-06T21:12:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.621+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcf800fb90 con 0x5576a8afa520 2023-10-06T21:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.622+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576a8aedd50 con 0x5576a8afa520 2023-10-06T21:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.622+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcf8005040 con 0x5576a8afa520 2023-10-06T21:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.622+0000 7fdd229f8b80 10 client.25829 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.622+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5576a8861c00 con 0x5576a8afa520 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576a8aedd50 con 0x5576a8afa520 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdcf8026020 con 0x5576a8afa520 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 20 client.25829 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b14bb0 0x5576a8b14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 4 client.25829 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.623+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576a8a46db0 con 0x5576a8b14bb0 2023-10-06T21:12:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.624+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b14bb0 0x5576a8b14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.624+0000 7fdce7fff700 10 client.25829 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.624+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b14bb0 0x5576a8b14fc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d8463fa8799285 server_cookie=25f296981b2ffa29 in_seq=0 out_seq=0 2023-10-06T21:12:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.625+0000 7fdce7fff700 1 -- 172.21.15.26:0/1011828408 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5576a8a46db0 con 0x5576a8b14bb0 2023-10-06T21:12:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.625+0000 7fdce7fff700 10 client.25829 handle_command_reply: tid=0 2023-10-06T21:12:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.626+0000 7fdd229f8b80 1 client.25829 shutdown 2023-10-06T21:12:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.626+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 msgr2=0x5576a8b09b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.626+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005b40 tx=0x7fdcf80038b0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 shutdown_connections 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b09630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b14bb0 0x5576a8b14fc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1011828408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 >> 172.21.15.26:0/1011828408 conn(0x5576a8a6f200 msgr2=0x5576a8af5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdce7fff700 0 client.25829 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 shutdown_connections 2023-10-06T21:12:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.627+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1011828408 wait complete. 2023-10-06T21:12:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.628+0000 7fdd229f8b80 20 client.25829 trim_cache size 0 max 16384 2023-10-06T21:12:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 Processor -- start 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 -- start start 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b2d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a88ce8c0 con 0x5576a8af9c00 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8af6de0 con 0x5576a8afa520 2023-10-06T21:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.651+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8b09230 con 0x5576a8b163e0 2023-10-06T21:12:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.652+0000 7fdcfffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.652+0000 7fdcfffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36042/0 (socket says 172.21.15.26:36042) 2023-10-06T21:12:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.652+0000 7fdcfffff700 1 -- 172.21.15.26:0/3124375947 learned_addr learned my addr 172.21.15.26:0/3124375947 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.652+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b2d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 -- 172.21.15.26:0/3124375947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 msgr2=0x5576a8b2d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b2d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 -- 172.21.15.26:0/3124375947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 msgr2=0x5576a8b111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 -- 172.21.15.26:0/3124375947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576a8af4a80 con 0x5576a8afa520 2023-10-06T21:12:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.653+0000 7fdcfffff700 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8004520 tx=0x7fdcf8004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e5c8d8a43304156 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.654+0000 7fdcef7fe700 1 -- 172.21.15.26:0/3124375947 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8012070 con 0x5576a8afa520 2023-10-06T21:12:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.654+0000 7fdcef7fe700 1 -- 172.21.15.26:0/3124375947 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf8005bc0 con 0x5576a8afa520 2023-10-06T21:12:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.654+0000 7fdcef7fe700 1 -- 172.21.15.26:0/3124375947 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8029a60 con 0x5576a8afa520 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- 172.21.15.26:0/3124375947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 msgr2=0x5576a8b09b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8004520 tx=0x7fdcf8004550 comp rx=0 tx=0).stop 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- 172.21.15.26:0/3124375947 shutdown_connections 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8b163e0 0x5576a8b2d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8af9c00 0x5576a8b111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/3124375947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8afa520 0x5576a8b09b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- 172.21.15.26:0/3124375947 >> 172.21.15.26:0/3124375947 conn(0x5576a8a6f200 msgr2=0x5576a8b09420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- 172.21.15.26:0/3124375947 shutdown_connections 2023-10-06T21:12:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- 172.21.15.26:0/3124375947 wait complete. 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.655+0000 7fdd229f8b80 1 -- start start 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 Processor -- start 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8af9c00 0x5576a8b2d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8afa520 0x5576a8b2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8b01e60 con 0x5576a8afa520 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8b01fd0 con 0x5576a8b163e0 2023-10-06T21:12:35.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576a8b02190 con 0x5576a8af9c00 2023-10-06T21:12:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.656+0000 7fdd229f8b80 10 client.0 _ll_register_callbacks cb 0x7fdcf800dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36048/0 (socket says 172.21.15.26:36048) 2023-10-06T21:12:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 -- 172.21.15.26:0/1770667049 learned_addr learned my addr 172.21.15.26:0/1770667049 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8af9c00 0x5576a8b2d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8afa520 0x5576a8b2d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 -- 172.21.15.26:0/1770667049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8af9c00 msgr2=0x5576a8b2d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8af9c00 0x5576a8b2d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 -- 172.21.15.26:0/1770667049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8afa520 msgr2=0x5576a8b2d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8afa520 0x5576a8b2d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.657+0000 7fdd0d59c700 1 -- 172.21.15.26:0/1770667049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576a8861c00 con 0x5576a8b163e0 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.658+0000 7fdd0d59c700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd00000c00 tx=0x7fdd00040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46338e66ad032f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.658+0000 7fdce7fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.658+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00053040 con 0x5576a8b163e0 2023-10-06T21:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.659+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdd0004ea20 con 0x5576a8b163e0 2023-10-06T21:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.659+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd00030d00 con 0x5576a8b163e0 2023-10-06T21:12:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.659+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd00043040 con 0x5576a8b163e0 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.659+0000 7fdd229f8b80 10 client.25835 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.659+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5576a889f1f0 con 0x5576a8b163e0 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0005d080 con 0x5576a8b163e0 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdd0003e450 con 0x5576a8b163e0 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 10 client.25835 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 20 client.25835 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b7a940 0x5576a8b7cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 4 client.25835 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:12:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576a8a46ad0 con 0x5576a8b7a940 2023-10-06T21:12:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.660+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b7a940 0x5576a8b7cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.661+0000 7fdcfffff700 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b7a940 0x5576a8b7cd60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e61a99ae2060aae server_cookie=cbdbd8aeed71912f in_seq=0 out_seq=0 2023-10-06T21:12:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.661+0000 7fdce7fff700 10 client.25835 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.673+0000 7fdce7fff700 1 -- 172.21.15.26:0/1770667049 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576a8a46ad0 con 0x5576a8b7a940 2023-10-06T21:12:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.673+0000 7fdce7fff700 10 client.25835 handle_command_reply: tid=0 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.673+0000 7fdd229f8b80 1 client.25835 shutdown 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 msgr2=0x5576a8b01750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd00000c00 tx=0x7fdd00040040 comp rx=0 tx=0).stop 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 shutdown_connections 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5576a8b7a940 0x5576a8b7cd60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5576a8b163e0 0x5576a8b01750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576a8af9c00 0x5576a8b2d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdce7fff700 0 client.25835 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 --2- 172.21.15.26:0/1770667049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5576a8afa520 0x5576a8b2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.674+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 >> 172.21.15.26:0/1770667049 conn(0x5576a8a6f200 msgr2=0x5576a8b08220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.675+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 shutdown_connections 2023-10-06T21:12:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.676+0000 7fdd229f8b80 1 -- 172.21.15.26:0/1770667049 wait complete. 2023-10-06T21:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.677+0000 7fdd229f8b80 20 client.25835 trim_cache size 0 max 16384 2023-10-06T21:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:12:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 msgr2=0x7fdcf00a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 0x7fdcf00a3cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd04099da0 tx=0x7fdd040178f0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 msgr2=0x7fdd081a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400b840 tx=0x7fdd0400bb50 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 shutdown_connections 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd08105eb0 0x7fdd0819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd080fef20 0x7fdd081a37c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdcf00a1800 0x7fdcf00a3cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 --2- 172.21.15.26:0/12612903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd080ff8e0 0x7fdd081a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 >> 172.21.15.26:0/12612903 conn(0x7fdd08076e70 msgr2=0x7fdd081045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 shutdown_connections 2023-10-06T21:12:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:35.678+0000 7fdd0efff700 1 -- 172.21.15.26:0/12612903 wait complete. 2023-10-06T21:12:36.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:36 smithi167 ceph-mon[134494]: pgmap v71: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 39 MiB/s wr, 2.00k op/s 2023-10-06T21:12:36.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:36 smithi026 ceph-mon[98490]: pgmap v71: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 39 MiB/s wr, 2.00k op/s 2023-10-06T21:12:36.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:36 smithi078 ceph-mon[134896]: pgmap v71: 97 pgs: 97 active+clean; 3.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 39 MiB/s wr, 2.00k op/s 2023-10-06T21:12:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 115.0 s, 2696.0 tps, lat 37.124 ms stddev 42.910 2023-10-06T21:12:38.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:38 smithi167 ceph-mon[134494]: pgmap v72: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2023-10-06T21:12:38.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:38 smithi026 ceph-mon[98490]: pgmap v72: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2023-10-06T21:12:38.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:38 smithi078 ceph-mon[134896]: pgmap v72: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 37 MiB/s wr, 1.88k op/s 2023-10-06T21:12:40.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:40 smithi167 ceph-mon[134494]: pgmap v73: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-10-06T21:12:40.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:40 smithi026 ceph-mon[98490]: pgmap v73: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-10-06T21:12:40.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:40 smithi078 ceph-mon[134896]: pgmap v73: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 1.78k op/s 2023-10-06T21:12:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 120.0 s, 3037.6 tps, lat 32.860 ms stddev 35.506 2023-10-06T21:12:42.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:42 smithi167 ceph-mon[134494]: pgmap v74: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 41 MiB/s wr, 2.21k op/s 2023-10-06T21:12:42.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:42 smithi026 ceph-mon[98490]: pgmap v74: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 41 MiB/s wr, 2.21k op/s 2023-10-06T21:12:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:42 smithi078 ceph-mon[134896]: pgmap v74: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 41 MiB/s wr, 2.21k op/s 2023-10-06T21:12:44.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:43 smithi167 ceph-mon[134494]: pgmap v75: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 733 op/s 2023-10-06T21:12:44.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:43 smithi026 ceph-mon[98490]: pgmap v75: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 733 op/s 2023-10-06T21:12:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:43 smithi078 ceph-mon[134896]: pgmap v75: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 733 op/s 2023-10-06T21:12:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:46 smithi167 ceph-mon[134494]: pgmap v76: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 948 op/s 2023-10-06T21:12:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:46 smithi026 ceph-mon[98490]: pgmap v76: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 948 op/s 2023-10-06T21:12:46.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:46 smithi078 ceph-mon[134896]: pgmap v76: 97 pgs: 97 active+clean; 3.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 18 MiB/s wr, 948 op/s 2023-10-06T21:12:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 125.0 s, 3042.4 tps, lat 32.903 ms stddev 37.338 2023-10-06T21:12:48.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:48 smithi167 ceph-mon[134494]: pgmap v77: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 844 op/s 2023-10-06T21:12:48.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:48 smithi026 ceph-mon[98490]: pgmap v77: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 844 op/s 2023-10-06T21:12:48.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:48 smithi078 ceph-mon[134896]: pgmap v77: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 844 op/s 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.916+0000 7f36bcfff700 1 -- 172.21.15.26:0/4129926032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b8100300 msgr2=0x7f36b8104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.916+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4129926032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b8100300 0x7f36b8104170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36a000aaa0 tx=0x7f36a000adb0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 -- 172.21.15.26:0/4129926032 shutdown_connections 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4129926032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4129926032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b80ff950 0x7f36b80ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4129926032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b8100300 0x7f36b8104170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 -- 172.21.15.26:0/4129926032 >> 172.21.15.26:0/4129926032 conn(0x7f36b80762e0 msgr2=0x7f36b80766f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 -- 172.21.15.26:0/4129926032 shutdown_connections 2023-10-06T21:12:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.917+0000 7f36bcfff700 1 -- 172.21.15.26:0/4129926032 wait complete. 2023-10-06T21:12:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.918+0000 7f36bcfff700 1 Processor -- start 2023-10-06T21:12:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.921+0000 7f36bcfff700 1 -- start start 2023-10-06T21:12:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b80ff950 0x7f36b810c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 0x7f36b810ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81a9e20 con 0x7f36b810a780 2023-10-06T21:12:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81a9f90 con 0x7f36b81048a0 2023-10-06T21:12:48.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36bcfff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81aa150 con 0x7f36b80ff950 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48592/0 (socket says 172.21.15.26:48592) 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.922+0000 7f36affff700 1 -- 172.21.15.26:0/4211923999 learned_addr learned my addr 172.21.15.26:0/4211923999 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 0x7f36b810ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36affff700 1 -- 172.21.15.26:0/4211923999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b80ff950 msgr2=0x7f36b810c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36b659c700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b80ff950 0x7f36b810c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36affff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b80ff950 0x7f36b810c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36affff700 1 -- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 msgr2=0x7f36b810ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36affff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 0x7f36b810ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36affff700 1 -- 172.21.15.26:0/4211923999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a000a700 con 0x7f36b81048a0 2023-10-06T21:12:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.923+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 0x7f36b810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:48.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.926+0000 7f36affff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a9f0 tx=0x7f36a0003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.926+0000 7f36af7fe700 1 -- 172.21.15.26:0/4211923999 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0004040 con 0x7f36b81048a0 2023-10-06T21:12:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.926+0000 7f36af7fe700 1 -- 172.21.15.26:0/4211923999 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a00041a0 con 0x7f36b81048a0 2023-10-06T21:12:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.926+0000 7f36af7fe700 1 -- 172.21.15.26:0/4211923999 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0016680 con 0x7f36b81048a0 2023-10-06T21:12:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.926+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b81aa3d0 con 0x7f36b81048a0 2023-10-06T21:12:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.927+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81aa7a0 con 0x7f36b81048a0 2023-10-06T21:12:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.933+0000 7f36af7fe700 1 -- 172.21.15.26:0/4211923999 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f36a000e450 con 0x7f36b81048a0 2023-10-06T21:12:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.934+0000 7f36d08b0b80 1 Processor -- start 2023-10-06T21:12:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.935+0000 7f36d08b0b80 1 -- start start 2023-10-06T21:12:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.935+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.935+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.935+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea450400 con 0x55d8ea4e6c00 2023-10-06T21:12:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea441bc0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea43f4e0 con 0x55d8ea4e7520 2023-10-06T21:12:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36b6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36b6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60316/0 (socket says 172.21.15.26:60316) 2023-10-06T21:12:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36b6d9d700 1 -- 172.21.15.26:0/3586236609 learned_addr learned my addr 172.21.15.26:0/3586236609 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.936+0000 7f36b659c700 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 -- 172.21.15.26:0/3586236609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 msgr2=0x55d8ea4f47c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f47c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 -- 172.21.15.26:0/3586236609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 msgr2=0x55d8ea4e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 -- 172.21.15.26:0/3586236609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ea265d80 con 0x55d8ea4e7520 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.937+0000 7f36b659c700 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f1e90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009b60 tx=0x7f36a8009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54dfd1f21122e39c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.938+0000 7f36acff9700 1 -- 172.21.15.26:0/3586236609 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004a30 con 0x55d8ea4e7520 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.938+0000 7f36acff9700 1 -- 172.21.15.26:0/3586236609 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a80054e0 con 0x55d8ea4e7520 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.938+0000 7f36acff9700 1 -- 172.21.15.26:0/3586236609 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800ed10 con 0x55d8ea4e7520 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.938+0000 7f36af7fe700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 0x7f36980a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.939+0000 7f36af7fe700 1 -- 172.21.15.26:0/4211923999 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f36a0012070 con 0x7f36b81048a0 2023-10-06T21:12:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.939+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 0x7f36980a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.939+0000 7f36d08b0b80 1 -- 172.21.15.26:0/3586236609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 msgr2=0x55d8ea4f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.939+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f1e90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009b60 tx=0x7f36a8009e70 comp rx=0 tx=0).stop 2023-10-06T21:12:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36d08b0b80 1 -- 172.21.15.26:0/3586236609 shutdown_connections 2023-10-06T21:12:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/3586236609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f1e90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36d08b0b80 1 -- 172.21.15.26:0/3586236609 >> 172.21.15.26:0/3586236609 conn(0x55d8ea45c200 msgr2=0x55d8ea4e3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.940+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 0x7f36980a3be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009fa0 tx=0x7f36b0009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.943+0000 7f36d08b0b80 1 -- 172.21.15.26:0/3586236609 shutdown_connections 2023-10-06T21:12:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.943+0000 7f36d08b0b80 1 -- 172.21.15.26:0/3586236609 wait complete. 2023-10-06T21:12:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.943+0000 7f36d08b0b80 1 -- start start 2023-10-06T21:12:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 Processor -- start 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea43add0 con 0x55d8ea4e6c00 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea3d76b0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea3d7350 con 0x55d8ea4e7520 2023-10-06T21:12:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.944+0000 7f36d08b0b80 10 client.0 _ll_register_callbacks cb 0x55d8ea2347a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48618/0 (socket says 172.21.15.26:48618) 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2457263390 learned_addr learned my addr 172.21.15.26:0/2457263390 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b659c700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36affff700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2457263390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 msgr2=0x55d8ea4ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2457263390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 msgr2=0x55d8ea4ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2457263390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a8009790 con 0x55d8ea4f23d0 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b659c700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:12:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36b0018ea0 tx=0x7f36b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ed8f41befd96e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.945+0000 7f36a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.946+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0016d00 con 0x55d8ea4f23d0 2023-10-06T21:12:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.946+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f36b0023440 con 0x55d8ea4f23d0 2023-10-06T21:12:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.946+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8ea4dad50 con 0x55d8ea4f23d0 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.947+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b0016e60 con 0x55d8ea4f23d0 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.947+0000 7f36d08b0b80 10 client.25844 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.947+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d8ea24ec00 con 0x55d8ea4f23d0 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.947+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8ea4dad50 con 0x55d8ea4f23d0 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.947+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f36b002d6d0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 20 client.25844 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea502060 0x55d8ea504480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 4 client.25844 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8ea433db0 con 0x55d8ea502060 2023-10-06T21:12:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.948+0000 7f36b659c700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea502060 0x55d8ea504480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.949+0000 7f36b659c700 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea502060 0x55d8ea504480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8568fdef4a873eb server_cookie=a49cb0efc3d869c5 in_seq=0 out_seq=0 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.949+0000 7f36a5ffb700 10 client.25844 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2457263390 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8ea433db0 con 0x55d8ea502060 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36a5ffb700 10 client.25844 handle_command_reply: tid=0 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 client.25844 shutdown 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 msgr2=0x55d8ea4efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f36b0018ea0 tx=0x7f36b0004ad0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 shutdown_connections 2023-10-06T21:12:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea502060 0x55d8ea504480 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4efbe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2457263390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 >> 172.21.15.26:0/2457263390 conn(0x55d8ea45c200 msgr2=0x55d8ea4e2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36a5ffb700 0 client.25844 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 shutdown_connections 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.950+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2457263390 wait complete. 2023-10-06T21:12:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.951+0000 7f36d08b0b80 20 client.25844 trim_cache size 0 max 16384 2023-10-06T21:12:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.975+0000 7f36d08b0b80 1 Processor -- start 2023-10-06T21:12:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.976+0000 7f36d08b0b80 1 -- start start 2023-10-06T21:12:48.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea504070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea2bb8c0 con 0x55d8ea4e6c00 2023-10-06T21:12:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea4e7de0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea4e92c0 con 0x55d8ea4e7520 2023-10-06T21:12:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36affff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.977+0000 7f36affff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42666/0 (socket says 172.21.15.26:42666) 2023-10-06T21:12:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 -- 172.21.15.26:0/493432339 learned_addr learned my addr 172.21.15.26:0/493432339 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea504070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 -- 172.21.15.26:0/493432339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 msgr2=0x55d8ea504070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea504070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 -- 172.21.15.26:0/493432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 msgr2=0x55d8ea4f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 -- 172.21.15.26:0/493432339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ea28c1f0 con 0x55d8ea4e7520 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36affff700 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36a0098600 tx=0x7f36a00a0f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d36793d51f0de8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36ad7fa700 1 -- 172.21.15.26:0/493432339 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0004310 con 0x55d8ea4e7520 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36ad7fa700 1 -- 172.21.15.26:0/493432339 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a0014370 con 0x55d8ea4e7520 2023-10-06T21:12:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.978+0000 7f36ad7fa700 1 -- 172.21.15.26:0/493432339 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00a4400 con 0x55d8ea4e7520 2023-10-06T21:12:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 -- 172.21.15.26:0/493432339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 msgr2=0x55d8ea4ef920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36a0098600 tx=0x7f36a00a0f90 comp rx=0 tx=0).stop 2023-10-06T21:12:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 -- 172.21.15.26:0/493432339 shutdown_connections 2023-10-06T21:12:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4f23d0 0x55d8ea504070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/493432339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4ef920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 -- 172.21.15.26:0/493432339 >> 172.21.15.26:0/493432339 conn(0x55d8ea45c200 msgr2=0x55d8ea4e8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.979+0000 7f36d08b0b80 1 -- 172.21.15.26:0/493432339 shutdown_connections 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.980+0000 7f36d08b0b80 1 -- 172.21.15.26:0/493432339 wait complete. 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.980+0000 7f36d08b0b80 1 -- start start 2023-10-06T21:12:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.980+0000 7f36d08b0b80 1 Processor -- start 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea554f00 con 0x55d8ea4f23d0 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea555070 con 0x55d8ea4e6c00 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ea566240 con 0x55d8ea4e7520 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.981+0000 7f36d08b0b80 10 client.0 _ll_register_callbacks cb 0x7f36b0018810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.982+0000 7f36b659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.982+0000 7f36b659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48644/0 (socket says 172.21.15.26:48644) 2023-10-06T21:12:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.982+0000 7f36b659c700 1 -- 172.21.15.26:0/2731942429 learned_addr learned my addr 172.21.15.26:0/2731942429 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.982+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.983+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2731942429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 msgr2=0x55d8ea4f69c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.983+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f69c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.983+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 msgr2=0x55d8ea4f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.983+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.983+0000 7f36b6d9d700 1 -- 172.21.15.26:0/2731942429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81aa7a0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.984+0000 7f36b6d9d700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f6f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36b002dae0 tx=0x7f36b0007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bb3a125eda02bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.984+0000 7f36a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:12:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.984+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0009d20 con 0x55d8ea4f23d0 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.984+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36a80054e0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.984+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f36b002d450 con 0x55d8ea4f23d0 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.985+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36b002d650 con 0x55d8ea4f23d0 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.985+0000 7f36d08b0b80 10 client.15711 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.985+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d8ea264250 con 0x55d8ea4f23d0 2023-10-06T21:12:48.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.985+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a80054e0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.985+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f36b00058a0 con 0x55d8ea4f23d0 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 20 client.15711 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea567a10 0x55d8ea569e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 4 client.15711 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:12:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8ea433ad0 con 0x55d8ea567a10 2023-10-06T21:12:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.986+0000 7f36b659c700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea567a10 0x55d8ea569e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.987+0000 7f36b659c700 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea567a10 0x55d8ea569e30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a636028daf019c1 server_cookie=da27d366f0b44109 in_seq=0 out_seq=0 2023-10-06T21:12:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:48.987+0000 7f36a5ffb700 10 client.15711 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.003+0000 7f36a5ffb700 1 -- 172.21.15.26:0/2731942429 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8ea433ad0 con 0x55d8ea567a10 2023-10-06T21:12:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.003+0000 7f36a5ffb700 10 client.15711 handle_command_reply: tid=0 2023-10-06T21:12:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.003+0000 7f36d08b0b80 1 client.15711 shutdown 2023-10-06T21:12:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 msgr2=0x55d8ea4f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f6f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36b002dae0 tx=0x7f36b0007490 comp rx=0 tx=0).stop 2023-10-06T21:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 shutdown_connections 2023-10-06T21:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d8ea4f23d0 0x55d8ea4f6f00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d8ea567a10 0x55d8ea569e30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8ea4e6c00 0x55d8ea4f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 --2- 172.21.15.26:0/2731942429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8ea4e7520 0x55d8ea4f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.004+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 >> 172.21.15.26:0/2731942429 conn(0x55d8ea45c200 msgr2=0x55d8ea504070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.005+0000 7f36a5ffb700 0 client.15711 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.005+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 shutdown_connections 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.005+0000 7f36d08b0b80 1 -- 172.21.15.26:0/2731942429 wait complete. 2023-10-06T21:12:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36d08b0b80 20 client.15711 trim_cache size 0 max 16384 2023-10-06T21:12:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:12:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:12:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:12:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 msgr2=0x7f36980a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 0x7f36980a3be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009fa0 tx=0x7f36b0009040 comp rx=0 tx=0).stop 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 msgr2=0x7f36b810a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a9f0 tx=0x7f36a0003960 comp rx=0 tx=0).stop 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 shutdown_connections 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f36980a1720 0x7f36980a3be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36b810a780 0x7f36b810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36b80ff950 0x7f36b810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.006+0000 7f36bcfff700 1 --2- 172.21.15.26:0/4211923999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36b81048a0 0x7f36b810a1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.007+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 >> 172.21.15.26:0/4211923999 conn(0x7f36b80762e0 msgr2=0x7f36b80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.007+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 shutdown_connections 2023-10-06T21:12:49.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:12:49.007+0000 7f36bcfff700 1 -- 172.21.15.26:0/4211923999 wait complete. 2023-10-06T21:12:49.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:12:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 -- 172.21.15.26:0/3268949494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 msgr2=0x7f97ac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 --2- 172.21.15.26:0/3268949494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac10dd50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b0a0 tx=0x7f97a400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 -- 172.21.15.26:0/3268949494 shutdown_connections 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 --2- 172.21.15.26:0/3268949494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac10dd50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 --2- 172.21.15.26:0/3268949494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 --2- 172.21.15.26:0/3268949494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac101a60 0x7f97ac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 -- 172.21.15.26:0/3268949494 >> 172.21.15.26:0/3268949494 conn(0x7f97ac0fc930 msgr2=0x7f97ac0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 -- 172.21.15.26:0/3268949494 shutdown_connections 2023-10-06T21:12:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.520+0000 7f97b2fff700 1 -- 172.21.15.26:0/3268949494 wait complete. 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.521+0000 7f97b2fff700 1 Processor -- start 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 -- start start 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac19b820 0x7f97ac19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19c2a0 con 0x7f97ac19b820 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19c410 con 0x7f97ac1061d0 2023-10-06T21:12:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b2fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19c5d0 con 0x7f97ac1010b0 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42706/0 (socket says 172.21.15.26:42706) 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48654/0 (socket says 172.21.15.26:48654) 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2832704900 learned_addr learned my addr 172.21.15.26:0/2832704900 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.522+0000 7f97b159c700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac19b820 0x7f97ac19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2832704900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 msgr2=0x7f97ac1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac19b820 msgr2=0x7f97ac19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac19b820 0x7f97ac19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2832704900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009520 con 0x7f97ac1010b0 2023-10-06T21:12:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eb30 tx=0x7f97a000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97a9ffb700 1 -- 172.21.15.26:0/2832704900 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0009880 con 0x7f97ac1010b0 2023-10-06T21:12:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a4009d00 con 0x7f97ac1010b0 2023-10-06T21:12:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97a9ffb700 1 -- 172.21.15.26:0/2832704900 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a00044e0 con 0x7f97ac1010b0 2023-10-06T21:12:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97a9ffb700 1 -- 172.21.15.26:0/2832704900 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0003d80 con 0x7f97ac1010b0 2023-10-06T21:12:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.523+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1a3ed0 con 0x7f97ac1010b0 2023-10-06T21:12:49.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.527+0000 7f97a9ffb700 1 -- 172.21.15.26:0/2832704900 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f97a0003f20 con 0x7f97ac1010b0 2023-10-06T21:12:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97c69dcb80 1 Processor -- start 2023-10-06T21:12:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97c69dcb80 1 -- start start 2023-10-06T21:12:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97a9ffb700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 0x7f979c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97a9ffb700 1 -- 172.21.15.26:0/2832704900 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97a0014070 con 0x7f97ac1010b0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97b159c700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 0x7f979c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.528+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675ad5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675a33460 con 0x55c675aca9e0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675a24c20 con 0x55c675aca0c0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675a22540 con 0x55c675ad5890 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675ad5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48666/0 (socket says 172.21.15.26:48666) 2023-10-06T21:12:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 -- 172.21.15.26:0/4078143593 learned_addr learned my addr 172.21.15.26:0/4078143593 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 -- 172.21.15.26:0/4078143593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 msgr2=0x55c675ad7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 -- 172.21.15.26:0/4078143593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 msgr2=0x55c675ad5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675ad5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 -- 172.21.15.26:0/4078143593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c675848de0 con 0x55c675aca0c0 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b159c700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 0x7f979c0a3ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009610 tx=0x7f97a400bbf0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.529+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f97a00a2440 tx=0x7f97a009a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a0d9ca45f3f809 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f9796ffd700 1 -- 172.21.15.26:0/4078143593 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a009ace0 con 0x55c675aca0c0 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f9796ffd700 1 -- 172.21.15.26:0/4078143593 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a009a5e0 con 0x55c675aca0c0 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f9796ffd700 1 -- 172.21.15.26:0/4078143593 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0096cf0 con 0x55c675aca0c0 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 -- 172.21.15.26:0/4078143593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 msgr2=0x55c675aca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f97a00a2440 tx=0x7f97a009a4e0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 -- 172.21.15.26:0/4078143593 shutdown_connections 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675aca4a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/4078143593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675ad5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 -- 172.21.15.26:0/4078143593 >> 172.21.15.26:0/4078143593 conn(0x55c675a3f260 msgr2=0x55c675ac6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 -- 172.21.15.26:0/4078143593 shutdown_connections 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.530+0000 7f97c69dcb80 1 -- 172.21.15.26:0/4078143593 wait complete. 2023-10-06T21:12:49.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.531+0000 7f97c69dcb80 1 -- start start 2023-10-06T21:12:49.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.531+0000 7f97c69dcb80 1 Processor -- start 2023-10-06T21:12:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675adf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675a1de30 con 0x55c675aca9e0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6759ba710 con 0x55c675aca0c0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6759ba3b0 con 0x55c675ad5890 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97c69dcb80 10 client.0 _ll_register_callbacks cb 0x55c675817800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48674/0 (socket says 172.21.15.26:48674) 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2603385032 learned_addr learned my addr 172.21.15.26:0/2603385032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b159c700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97abfff700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675adf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2603385032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 msgr2=0x55c675ad9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2603385032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 msgr2=0x55c675adf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675adf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.532+0000 7f97b0d9b700 1 -- 172.21.15.26:0/2603385032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1a3ed0 con 0x55c675aca0c0 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eb00 tx=0x7f97a0003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da82ba230f0d27be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a009e730 con 0x55c675aca0c0 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f97a000cdd0 con 0x55c675aca0c0 2023-10-06T21:12:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c675abdf10 con 0x55c675aca0c0 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97a0096cf0 con 0x55c675aca0c0 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f97c69dcb80 10 client.25862 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c675831c60 con 0x55c675aca0c0 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c675abdf10 con 0x55c675aca0c0 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.533+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f97a00a24e0 con 0x55c675aca0c0 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 10 client.25862 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 20 client.25862 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675ae9390 0x55c675aeb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 4 client.25862 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c675a16e10 con 0x55c675ae9390 2023-10-06T21:12:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97abfff700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675ae9390 0x55c675aeb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f97abfff700 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675ae9390 0x55c675aeb7b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbd3bec0260cddb server_cookie=5c3fa208ba50d6d3 in_seq=0 out_seq=0 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.534+0000 7f978bfff700 10 client.25862 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f978bfff700 1 -- 172.21.15.26:0/2603385032 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c675a16e10 con 0x55c675ae9390 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f978bfff700 10 client.25862 handle_command_reply: tid=0 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 client.25862 shutdown 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 msgr2=0x55c675adf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eb00 tx=0x7f97a0003820 comp rx=0 tx=0).stop 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 shutdown_connections 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675ad9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675ae9390 0x55c675aeb7b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca0c0 0x55c675adf200 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2603385032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca9e0 0x55c675adf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f978bfff700 0 client.25862 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 >> 172.21.15.26:0/2603385032 conn(0x55c675a3f260 msgr2=0x55c675ac5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 shutdown_connections 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.535+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2603385032 wait complete. 2023-10-06T21:12:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.536+0000 7f97c69dcb80 20 client.25862 trim_cache size 0 max 16384 2023-10-06T21:12:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.553+0000 7f97c69dcb80 1 Processor -- start 2023-10-06T21:12:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.553+0000 7f97c69dcb80 1 -- start start 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675ad1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675ad5890 0x55c675ad8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67589e920 con 0x55c675aca0c0 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675ada920 con 0x55c675ad5890 2023-10-06T21:12:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675ad2810 con 0x55c675aca9e0 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675ad1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97b159c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675ad5890 0x55c675ad8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675ad1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60394/0 (socket says 172.21.15.26:60394) 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97b0d9b700 1 -- 172.21.15.26:0/3367269525 learned_addr learned my addr 172.21.15.26:0/3367269525 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42718/0 (socket says 172.21.15.26:42718) 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 -- 172.21.15.26:0/3367269525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675ad5890 msgr2=0x55c675ad8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675ad5890 0x55c675ad8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 -- 172.21.15.26:0/3367269525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 msgr2=0x55c675ad1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675ad1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.554+0000 7f97abfff700 1 -- 172.21.15.26:0/3367269525 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c67586f250 con 0x55c675aca9e0 2023-10-06T21:12:49.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97abfff700 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55c675aec0b0 tx=0x7f9798009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc7f5636461a818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97977fe700 1 -- 172.21.15.26:0/3367269525 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979801f330 con 0x55c675aca9e0 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97977fe700 1 -- 172.21.15.26:0/3367269525 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9798018040 con 0x55c675aca9e0 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97977fe700 1 -- 172.21.15.26:0/3367269525 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798004ea0 con 0x55c675aca9e0 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97c69dcb80 1 -- 172.21.15.26:0/3367269525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 msgr2=0x55c675ad21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.555+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55c675aec0b0 tx=0x7f9798009260 comp rx=0 tx=0).stop 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 -- 172.21.15.26:0/3367269525 shutdown_connections 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675ad5890 0x55c675ad8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675ad1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/3367269525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675aca9e0 0x55c675ad21e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 -- 172.21.15.26:0/3367269525 >> 172.21.15.26:0/3367269525 conn(0x55c675a3f260 msgr2=0x55c675ad9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 -- 172.21.15.26:0/3367269525 shutdown_connections 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 -- 172.21.15.26:0/3367269525 wait complete. 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 -- start start 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.556+0000 7f97c69dcb80 1 Processor -- start 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675b31360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675b36230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675b367a0 con 0x55c675aca0c0 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675b36910 con 0x55c675aca9e0 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c675b36ad0 con 0x55c675ad5890 2023-10-06T21:12:49.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97c69dcb80 10 client.0 _ll_register_callbacks cb 0x7f9798009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97b0d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675b31360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48690/0 (socket says 172.21.15.26:48690) 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 -- 172.21.15.26:0/2192713223 learned_addr learned my addr 172.21.15.26:0/2192713223 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97b159c700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675b36230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 -- 172.21.15.26:0/2192713223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 msgr2=0x55c675b36230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675b36230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 -- 172.21.15.26:0/2192713223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 msgr2=0x55c675b31360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675b31360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.557+0000 7f97abfff700 1 -- 172.21.15.26:0/2192713223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6758472b0 con 0x55c675aca9e0 2023-10-06T21:12:49.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f97abfff700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f979801f3a0 tx=0x7f979801f3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3c3ad05183265d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f978bfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979802c070 con 0x55c675aca9e0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97a009a5e0 con 0x55c675aca9e0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9798004260 con 0x55c675aca9e0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9798004460 con 0x55c675aca9e0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f97c69dcb80 10 client.25868 fetch_fsmap learned FSMap version 29 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c675915a80 con 0x55c675aca9e0 2023-10-06T21:12:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.558+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a009a5e0 con 0x55c675aca9e0 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f979802f020 con 0x55c675aca9e0 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 10 client.25868 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 20 client.25868 populate_metadata read hostname 'smithi026' 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675b3e2d0 0x55c675b406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 4 client.25868 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c675a16b30 con 0x55c675b3e2d0 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675b3e2d0 0x55c675b406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97b0d9b700 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675b3e2d0 0x55c675b406f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cb0a60b1042251 server_cookie=7666c94552b68049 in_seq=0 out_seq=0 2023-10-06T21:12:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f978bfff700 10 client.25868 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f978bfff700 1 -- 172.21.15.26:0/2192713223 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c675a16b30 con 0x55c675b3e2d0 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f978bfff700 10 client.25868 handle_command_reply: tid=0 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.559+0000 7f97c69dcb80 1 client.25868 shutdown 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 msgr2=0x55c675b318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f979801f3a0 tx=0x7f979801f3d0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 shutdown_connections 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c675ad5890 0x55c675b36230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c675aca0c0 0x55c675b31360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c675b3e2d0 0x55c675b406f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 --2- 172.21.15.26:0/2192713223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c675aca9e0 0x55c675b318a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f978bfff700 0 client.25868 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 >> 172.21.15.26:0/2192713223 conn(0x55c675a3f260 msgr2=0x55c675ad1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 shutdown_connections 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 1 -- 172.21.15.26:0/2192713223 wait complete. 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.560+0000 7f97c69dcb80 20 client.25868 trim_cache size 0 max 16384 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:12:49.569 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:12:49.570 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 msgr2=0x7f979c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 0x7f979c0a3ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009610 tx=0x7f97a400bbf0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 msgr2=0x7f97ac196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f97a000eb30 tx=0x7f97a000ee40 comp rx=0 tx=0).stop 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 shutdown_connections 2023-10-06T21:12:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f979c0a15f0 0x7f979c0a3ab0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ac19b820 0x7f97ac19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97ac1010b0 0x7f97ac196e80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 --2- 172.21.15.26:0/2832704900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97ac1061d0 0x7f97ac1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 >> 172.21.15.26:0/2832704900 conn(0x7f97ac0fc930 msgr2=0x7f97ac0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 shutdown_connections 2023-10-06T21:12:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:12:49.561+0000 7f97b2fff700 1 -- 172.21.15.26:0/2832704900 wait complete. 2023-10-06T21:12:49.581 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:12:50.538 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:50 smithi167 ceph-mon[134494]: pgmap v78: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 737 op/s 2023-10-06T21:12:50.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:50 smithi026 ceph-mon[98490]: pgmap v78: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 737 op/s 2023-10-06T21:12:50.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:50 smithi078 ceph-mon[134896]: pgmap v78: 97 pgs: 97 active+clean; 3.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 737 op/s 2023-10-06T21:12:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 130.0 s, 2432.9 tps, lat 41.033 ms stddev 51.994 2023-10-06T21:12:52.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:52 smithi167 ceph-mon[134494]: pgmap v79: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.15k op/s 2023-10-06T21:12:52.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:52 smithi026 ceph-mon[98490]: pgmap v79: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.15k op/s 2023-10-06T21:12:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:52 smithi078 ceph-mon[134896]: pgmap v79: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.15k op/s 2023-10-06T21:12:54.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:53 smithi167 ceph-mon[134494]: pgmap v80: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-06T21:12:54.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:53 smithi026 ceph-mon[98490]: pgmap v80: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-06T21:12:54.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:53 smithi078 ceph-mon[134896]: pgmap v80: 97 pgs: 97 active+clean; 3.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-06T21:12:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:56 smithi167 ceph-mon[134494]: pgmap v81: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 998 op/s 2023-10-06T21:12:56.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:56 smithi026 ceph-mon[98490]: pgmap v81: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 998 op/s 2023-10-06T21:12:56.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:56 smithi078 ceph-mon[134896]: pgmap v81: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 19 MiB/s wr, 998 op/s 2023-10-06T21:12:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 135.0 s, 2398.2 tps, lat 41.615 ms stddev 41.754 2023-10-06T21:12:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:12:58 smithi167 ceph-mon[134494]: pgmap v82: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 20 MiB/s wr, 936 op/s 2023-10-06T21:12:58.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:12:58 smithi026 ceph-mon[98490]: pgmap v82: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 20 MiB/s wr, 936 op/s 2023-10-06T21:12:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:12:58 smithi078 ceph-mon[134896]: pgmap v82: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 20 MiB/s wr, 936 op/s 2023-10-06T21:13:00.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:00 smithi167 ceph-mon[134494]: pgmap v83: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 845 op/s 2023-10-06T21:13:00.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:00 smithi026 ceph-mon[98490]: pgmap v83: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 845 op/s 2023-10-06T21:13:00.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:00 smithi078 ceph-mon[134896]: pgmap v83: 97 pgs: 97 active+clean; 3.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 18 MiB/s wr, 845 op/s 2023-10-06T21:13:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 140.0 s, 2165.2 tps, lat 45.160 ms stddev 59.342 2023-10-06T21:13:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.186+0000 7f5207dff700 1 -- 172.21.15.26:0/2622372607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5200101a80 msgr2=0x7f5200105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.186+0000 7f5207dff700 1 --2- 172.21.15.26:0/2622372607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5200101a80 0x7f5200105ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009990 tx=0x7f51f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 -- 172.21.15.26:0/2622372607 shutdown_connections 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 --2- 172.21.15.26:0/2622372607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 0x7f520010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 --2- 172.21.15.26:0/2622372607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52001010d0 0x7f52001014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 --2- 172.21.15.26:0/2622372607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5200101a80 0x7f5200105ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 -- 172.21.15.26:0/2622372607 >> 172.21.15.26:0/2622372607 conn(0x7f52000fc930 msgr2=0x7f52000fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 -- 172.21.15.26:0/2622372607 shutdown_connections 2023-10-06T21:13:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.187+0000 7f5207dff700 1 -- 172.21.15.26:0/2622372607 wait complete. 2023-10-06T21:13:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.188+0000 7f5207dff700 1 Processor -- start 2023-10-06T21:13:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.188+0000 7f5207dff700 1 -- start start 2023-10-06T21:13:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 0x7f5200197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520019bba0 0x7f520019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019c620 con 0x7f520019bba0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019c790 con 0x7f52001061f0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5207dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019c950 con 0x7f52001010d0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44898/0 (socket says 172.21.15.26:44898) 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 -- 172.21.15.26:0/2826100732 learned_addr learned my addr 172.21.15.26:0/2826100732 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f520639c700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520019bba0 0x7f520019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f520539a700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 0x7f5200197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 -- 172.21.15.26:0/2826100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 msgr2=0x7f5200197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 0x7f5200197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 -- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520019bba0 msgr2=0x7f520019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520019bba0 0x7f520019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.189+0000 7f5205b9b700 1 -- 172.21.15.26:0/2826100732 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00095f0 con 0x7f52001010d0 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9a0 tx=0x7f51fc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f51f6ffd700 1 -- 172.21.15.26:0/2826100732 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc009790 con 0x7f52001010d0 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f51f6ffd700 1 -- 172.21.15.26:0/2826100732 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc010430 con 0x7f52001010d0 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52001a3eb0 con 0x7f52001010d0 2023-10-06T21:13:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f51f6ffd700 1 -- 172.21.15.26:0/2826100732 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc018680 con 0x7f52001010d0 2023-10-06T21:13:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.190+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52001a4380 con 0x7f52001010d0 2023-10-06T21:13:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.193+0000 7f51f6ffd700 1 -- 172.21.15.26:0/2826100732 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f51fc0105a0 con 0x7f52001010d0 2023-10-06T21:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.195+0000 7f51f6ffd700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 0x7f51ec0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.195+0000 7f521b7c0b80 1 Processor -- start 2023-10-06T21:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.195+0000 7f51f6ffd700 1 -- 172.21.15.26:0/2826100732 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51fc014070 con 0x7f52001010d0 2023-10-06T21:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.195+0000 7f520539a700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 0x7f51ec0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.195+0000 7f521b7c0b80 1 -- start start 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 0x5561867542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561867547e0 0x556186756bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561866ae400 con 0x556186745520 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618669fbc0 con 0x5561867547e0 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618669d4e0 con 0x556186744c00 2023-10-06T21:13:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f520539a700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 0x7f51ec0a3b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5200198810 tx=0x7f51f0005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44902/0 (socket says 172.21.15.26:44902) 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f5205b9b700 1 -- 172.21.15.26:0/2094988364 learned_addr learned my addr 172.21.15.26:0/2094988364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f520539a700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 0x5561867542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.196+0000 7f520639c700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561867547e0 0x556186756bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 -- 172.21.15.26:0/2094988364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561867547e0 msgr2=0x556186756bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561867547e0 0x556186756bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 -- 172.21.15.26:0/2094988364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 msgr2=0x5561867542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 0x5561867542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 -- 172.21.15.26:0/2094988364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561864c3d80 con 0x556186744c00 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f520539a700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 0x5561867542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:13:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096c80 tx=0x7f51fc099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50f9d4d34059c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f51ebfff700 1 -- 172.21.15.26:0/2094988364 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09e120 con 0x556186744c00 2023-10-06T21:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f51ebfff700 1 -- 172.21.15.26:0/2094988364 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0a25a0 con 0x556186744c00 2023-10-06T21:13:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.197+0000 7f51ebfff700 1 -- 172.21.15.26:0/2094988364 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc099900 con 0x556186744c00 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2094988364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 msgr2=0x556186744fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096c80 tx=0x7f51fc099e80 comp rx=0 tx=0).stop 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2094988364 shutdown_connections 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561867547e0 0x556186756bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186744fe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2094988364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186745520 0x5561867542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2094988364 >> 172.21.15.26:0/2094988364 conn(0x5561866ba200 msgr2=0x556186741080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2094988364 shutdown_connections 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.198+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2094988364 wait complete. 2023-10-06T21:13:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.199+0000 7f521b7c0b80 1 -- start start 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.199+0000 7f521b7c0b80 1 Processor -- start 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.199+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.199+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.199+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561867547e0 0x556186751bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186698dd0 con 0x5561867547e0 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561866356b0 con 0x556186745520 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186635350 con 0x556186744c00 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f521b7c0b80 10 client.0 _ll_register_callbacks cb 0x5561864927a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44914/0 (socket says 172.21.15.26:44914) 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 -- 172.21.15.26:0/2378147943 learned_addr learned my addr 172.21.15.26:0/2378147943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f520539a700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f520639c700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561867547e0 0x556186751bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 -- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 msgr2=0x5561867514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 -- 172.21.15.26:0/2378147943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561867547e0 msgr2=0x556186751bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561867547e0 0x556186751bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 -- 172.21.15.26:0/2378147943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52001a4380 con 0x556186744c00 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09a1a0 tx=0x7f51fc00c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6fef294bbd7db81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.200+0000 7f51e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc07b070 con 0x556186744c00 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51fc0b82c0 con 0x556186744c00 2023-10-06T21:13:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556186738d50 con 0x556186744c00 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51fc0b8480 con 0x556186744c00 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 10 client.25864 fetch_fsmap learned FSMap version 29 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5561864acc00 con 0x556186744c00 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556186738d50 con 0x556186744c00 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f51fc07e020 con 0x556186744c00 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 10 client.25864 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 10 client.25864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 20 client.25864 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556186763fe0 0x556186766400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 4 client.25864 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.201+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556186691db0 con 0x556186763fe0 2023-10-06T21:13:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.202+0000 7f520539a700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556186763fe0 0x556186766400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.202+0000 7f520539a700 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556186763fe0 0x556186766400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ce66dc7a4deab9 server_cookie=684bfbd207d4f3ee in_seq=0 out_seq=0 2023-10-06T21:13:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.202+0000 7f51e97fa700 10 client.25864 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f51e97fa700 1 -- 172.21.15.26:0/2378147943 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556186691db0 con 0x556186763fe0 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f51e97fa700 10 client.25864 handle_command_reply: tid=0 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 client.25864 shutdown 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 msgr2=0x556186754230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51fc09a1a0 tx=0x7f51fc00c400 comp rx=0 tx=0).stop 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 shutdown_connections 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561867547e0 0x556186751bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556186744c00 0x556186754230 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556186763fe0 0x556186766400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2378147943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f51e97fa700 0 client.25864 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 >> 172.21.15.26:0/2378147943 conn(0x5561866ba200 msgr2=0x556186740c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 shutdown_connections 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2378147943 wait complete. 2023-10-06T21:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.203+0000 7f521b7c0b80 20 client.25864 trim_cache size 0 max 16384 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 Processor -- start 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 -- start start 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x556186747330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186750c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561865198c0 con 0x556186744c00 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186752430 con 0x556186745520 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186752a60 con 0x5561867547e0 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f520639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186750c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.220+0000 7f520639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186750c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44928/0 (socket says 172.21.15.26:44928) 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52322/0 (socket says 172.21.15.26:52322) 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f520639c700 1 -- 172.21.15.26:0/2977082292 learned_addr learned my addr 172.21.15.26:0/2977082292 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f520539a700 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x556186747330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 -- 172.21.15.26:0/2977082292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 msgr2=0x556186750c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186750c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 -- 172.21.15.26:0/2977082292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 msgr2=0x556186747330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x556186747330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 -- 172.21.15.26:0/2977082292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561864ea1f0 con 0x556186744c00 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f5205b9b700 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096710 tx=0x7f51fc099e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf4d0243b48f182 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f51f4bff700 1 -- 172.21.15.26:0/2977082292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc09e120 con 0x556186744c00 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f51f4bff700 1 -- 172.21.15.26:0/2977082292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0994a0 con 0x556186744c00 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.221+0000 7f51f4bff700 1 -- 172.21.15.26:0/2977082292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0837f0 con 0x556186744c00 2023-10-06T21:13:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2977082292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 msgr2=0x55618675b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51fc096710 tx=0x7f51fc099e80 comp rx=0 tx=0).stop 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2977082292 shutdown_connections 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186750c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x55618675b330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/2977082292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x556186747330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2977082292 >> 172.21.15.26:0/2977082292 conn(0x5561866ba200 msgr2=0x556186746c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2977082292 shutdown_connections 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.222+0000 7f521b7c0b80 1 -- 172.21.15.26:0/2977082292 wait complete. 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.223+0000 7f521b7c0b80 1 -- start start 2023-10-06T21:13:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.223+0000 7f521b7c0b80 1 Processor -- start 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x5561867b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186759850 con 0x556186744c00 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186759990 con 0x556186745520 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556186759b20 con 0x5561867547e0 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f521b7c0b80 10 client.0 _ll_register_callbacks cb 0x7f51fc099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f520539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f520639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f520539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56412/0 (socket says 172.21.15.26:56412) 2023-10-06T21:13:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f520639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44938/0 (socket says 172.21.15.26:44938) 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f520539a700 1 -- 172.21.15.26:0/340485115 learned_addr learned my addr 172.21.15.26:0/340485115 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.224+0000 7f5205b9b700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x5561867b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 -- 172.21.15.26:0/340485115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 msgr2=0x556186759310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 -- 172.21.15.26:0/340485115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 msgr2=0x5561867b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x5561867b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 -- 172.21.15.26:0/340485115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561864acc00 con 0x556186745520 2023-10-06T21:13:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520639c700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f520539a700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f520006a460 tx=0x7f51f0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ff787fb685d169 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f51e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00056e0 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f51f0033420 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.225+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51fc0a25a0 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f0033a10 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 10 client.25886 fetch_fsmap learned FSMap version 29 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5561864c2250 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51fc0a25a0 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f51f0004b80 con 0x556186745520 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 10 client.25886 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:13:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 20 client.25886 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561867b8e00 0x5561867bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 4 client.25886 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.226+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556186691ad0 con 0x5561867b8e00 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.227+0000 7f5205b9b700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561867b8e00 0x5561867bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.227+0000 7f5205b9b700 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561867b8e00 0x5561867bb220 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f905570db73e478 server_cookie=64d829cc43e128a7 in_seq=0 out_seq=0 2023-10-06T21:13:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.227+0000 7f51e97fa700 10 client.25886 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:02 smithi167 ceph-mon[134494]: pgmap v84: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-06T21:13:02.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:02 smithi026 ceph-mon[98490]: pgmap v84: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-06T21:13:02.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:02 smithi078 ceph-mon[134896]: pgmap v84: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 27 MiB/s wr, 1.15k op/s 2023-10-06T21:13:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.828+0000 7f51e97fa700 1 -- 172.21.15.26:0/340485115 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556186691ad0 con 0x5561867b8e00 2023-10-06T21:13:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.828+0000 7f51e97fa700 10 client.25886 handle_command_reply: tid=0 2023-10-06T21:13:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.828+0000 7f521b7c0b80 1 client.25886 shutdown 2023-10-06T21:13:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 msgr2=0x5561867b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f520006a460 tx=0x7f51f0009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 shutdown_connections 2023-10-06T21:13:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561867547e0 0x556186759310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556186744c00 0x5561867b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561867b8e00 0x5561867bb220 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 --2- 172.21.15.26:0/340485115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556186745520 0x5561867b15b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f51e97fa700 0 client.25886 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 >> 172.21.15.26:0/340485115 conn(0x5561866ba200 msgr2=0x5561867473f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 shutdown_connections 2023-10-06T21:13:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 1 -- 172.21.15.26:0/340485115 wait complete. 2023-10-06T21:13:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.829+0000 7f521b7c0b80 20 client.25886 trim_cache size 0 max 16384 2023-10-06T21:13:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:13:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:13:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:13:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:13:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 msgr2=0x7f51ec0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 0x7f51ec0a3b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5200198810 tx=0x7f51f0005fd0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 msgr2=0x7f5200196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00e9a0 tx=0x7f51fc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 shutdown_connections 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f51ec0a1670 0x7f51ec0a3b30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520019bba0 0x7f520019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52001010d0 0x7f5200196ff0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 --2- 172.21.15.26:0/2826100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52001061f0 0x7f5200197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 >> 172.21.15.26:0/2826100732 conn(0x7f52000fc930 msgr2=0x7f52000fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 shutdown_connections 2023-10-06T21:13:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:02.830+0000 7f5207dff700 1 -- 172.21.15.26:0/2826100732 wait complete. 2023-10-06T21:13:04.034 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: *** Caught signal (Segmentation fault) ** 2023-10-06T21:13:04.034 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: in thread 7f27ff268700 thread_name:md_log_replay 2023-10-06T21:13:04.034 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f280fe17cf0] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55957016a11f] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55957016c353] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 4: (MDLog::_replay_thread()+0x647) [0x5595701c7bf7] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5595701d1141] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f280fe0d1ca] 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 7: clone() 2023-10-06T21:13:04.035 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:03.729+0000 7f27ff268700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: in thread 7f27ff268700 thread_name:md_log_replay 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f280fe17cf0] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55957016a11f] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55957016c353] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 4: (MDLog::_replay_thread()+0x647) [0x5595701c7bf7] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5595701d1141] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f280fe0d1ca] 2023-10-06T21:13:04.036 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 7: clone() 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 0> 2023-10-06T21:13:03.729+0000 7f27ff268700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: in thread 7f27ff268700 thread_name:md_log_replay 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f280fe17cf0] 2023-10-06T21:13:04.037 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55957016a11f] 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55957016c353] 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 4: (MDLog::_replay_thread()+0x647) [0x5595701c7bf7] 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5595701d1141] 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f280fe0d1ca] 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 7: clone() 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 0> 2023-10-06T21:13:03.729+0000 7f27ff268700 -1 *** Caught signal (Segmentation fault) ** 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: in thread 7f27ff268700 thread_name:md_log_replay 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: ceph version 18.0.0-6616-g14d8d032 (14d8d032507701c76ca224e769ee65a96b660a8c) reef (dev) 2023-10-06T21:13:04.038 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f280fe17cf0] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2: /usr/bin/ceph-mds(+0x4f011f) [0x55957016a11f] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55957016c353] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 4: (MDLog::_replay_thread()+0x647) [0x5595701c7bf7] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5595701d1141] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f280fe0d1ca] 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 7: clone() 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-06T21:13:04.039 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[171099]: 2023-10-06T21:13:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:04 smithi026 ceph-mon[98490]: pgmap v85: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 20 MiB/s wr, 741 op/s 2023-10-06T21:13:04.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:04 smithi078 ceph-mon[134896]: pgmap v85: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 20 MiB/s wr, 741 op/s 2023-10-06T21:13:04.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:04 smithi167 ceph-mon[134494]: pgmap v85: 97 pgs: 97 active+clean; 3.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 20 MiB/s wr, 741 op/s 2023-10-06T21:13:04.443 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:04 smithi167 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service: Main process exited, code=exited, status=139/n/a 2023-10-06T21:13:04.449 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:04 smithi167 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service: Failed with result 'exit-code'. 2023-10-06T21:13:06.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:06 smithi167 ceph-mon[134494]: pgmap v86: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2023-10-06T21:13:06.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:06 smithi026 ceph-mon[98490]: pgmap v86: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2023-10-06T21:13:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:06 smithi078 ceph-mon[134896]: pgmap v86: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 40 MiB/s wr, 1.82k op/s 2023-10-06T21:13:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 145.0 s, 1497.6 tps, lat 68.378 ms stddev 121.707 2023-10-06T21:13:08.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:08 smithi167 ceph-mon[134494]: pgmap v87: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 1.91k op/s 2023-10-06T21:13:08.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:08 smithi026 ceph-mon[98490]: pgmap v87: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 1.91k op/s 2023-10-06T21:13:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:08 smithi078 ceph-mon[134896]: pgmap v87: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 MiB/s wr, 1.91k op/s 2023-10-06T21:13:10.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:10 smithi167 ceph-mon[134494]: pgmap v88: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2023-10-06T21:13:10.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:10 smithi026 ceph-mon[98490]: pgmap v88: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2023-10-06T21:13:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:10 smithi078 ceph-mon[134896]: pgmap v88: 97 pgs: 97 active+clean; 3.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.76k op/s 2023-10-06T21:13:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 150.0 s, 2852.4 tps, lat 35.164 ms stddev 34.301 2023-10-06T21:13:12.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:12 smithi167 ceph-mon[134494]: pgmap v89: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 45 MiB/s wr, 2.40k op/s 2023-10-06T21:13:12.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:12 smithi026 ceph-mon[98490]: pgmap v89: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 45 MiB/s wr, 2.40k op/s 2023-10-06T21:13:12.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:12 smithi078 ceph-mon[134896]: pgmap v89: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 45 MiB/s wr, 2.40k op/s 2023-10-06T21:13:14.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:13 smithi167 ceph-mon[134494]: pgmap v90: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 36 MiB/s wr, 2.10k op/s 2023-10-06T21:13:14.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:13 smithi026 ceph-mon[98490]: pgmap v90: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 36 MiB/s wr, 2.10k op/s 2023-10-06T21:13:14.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:13 smithi078 ceph-mon[134896]: pgmap v90: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 36 MiB/s wr, 2.10k op/s 2023-10-06T21:13:14.865 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:14 smithi167 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service: Service RestartSec=10s expired, scheduling restart. 2023-10-06T21:13:14.865 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:14 smithi167 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service: Scheduled restart job, restart counter is at 1. 2023-10-06T21:13:14.865 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:14 smithi167 systemd[1]: Stopped Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:13:14.865 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:14 smithi167 systemd[1]: Starting Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:13:15.149 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:14 smithi167 podman[174602]: 2023-10-06T21:13:15.149 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:15 smithi167 bash[174602]: 3125f5d161842594b061df8dce52b906952eeeec9ae71b32a0a91396e9684e57 2023-10-06T21:13:15.532 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: starting mds.i at 2023-10-06T21:13:15.533 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:13:15 smithi167 systemd[1]: Started Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:13:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 -- 172.21.15.26:0/1698938854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7011a690 msgr2=0x7f0e7011ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1698938854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7011a690 0x7f0e7011ca80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009990 tx=0x7f0e60009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 -- 172.21.15.26:0/1698938854 shutdown_connections 2023-10-06T21:13:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1698938854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7011a690 0x7f0e7011ca80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1698938854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 0x7f0e70107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1698938854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7010fa90 0x7f0e701083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.069+0000 7f0e74fff700 1 -- 172.21.15.26:0/1698938854 >> 172.21.15.26:0/1698938854 conn(0x7f0e7006cfa0 msgr2=0x7f0e7006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.070+0000 7f0e74fff700 1 -- 172.21.15.26:0/1698938854 shutdown_connections 2023-10-06T21:13:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.070+0000 7f0e74fff700 1 -- 172.21.15.26:0/1698938854 wait complete. 2023-10-06T21:13:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 Processor -- start 2023-10-06T21:13:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 -- start start 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 0x7f0e701a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7011a690 0x7f0e701aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.072+0000 7f0e74fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701ab060 con 0x7f0e7010f620 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e74fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701ab1d0 con 0x7f0e7011a690 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e74fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701ab390 con 0x7f0e7010fa90 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7011a690 0x7f0e701aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47614/0 (socket says 172.21.15.26:47614) 2023-10-06T21:13:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/1406859224 learned_addr learned my addr 172.21.15.26:0/1406859224 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 0x7f0e701a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/1406859224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7011a690 msgr2=0x7f0e701aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7011a690 0x7f0e701aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 msgr2=0x7f0e701a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.073+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 0x7f0e701a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.074+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/1406859224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600095f0 con 0x7f0e7010fa90 2023-10-06T21:13:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.074+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ef10 tx=0x7f0e6800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.076+0000 7f0e5f7fe700 1 -- 172.21.15.26:0/1406859224 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6800ca70 con 0x7f0e7010fa90 2023-10-06T21:13:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.076+0000 7f0e5f7fe700 1 -- 172.21.15.26:0/1406859224 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6800f040 con 0x7f0e7010fa90 2023-10-06T21:13:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.076+0000 7f0e5f7fe700 1 -- 172.21.15.26:0/1406859224 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68014480 con 0x7f0e7010fa90 2023-10-06T21:13:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.077+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e701ab610 con 0x7f0e7010fa90 2023-10-06T21:13:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.077+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e70110050 con 0x7f0e7010fa90 2023-10-06T21:13:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.084+0000 7f0e8889bb80 1 Processor -- start 2023-10-06T21:13:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.084+0000 7f0e8889bb80 1 -- start start 2023-10-06T21:13:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 0x5627a36b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36b63d0 0x5627a36b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a3614400 con 0x5627a36aac00 2023-10-06T21:13:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a3605bc0 con 0x5627a36ab520 2023-10-06T21:13:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36034e0 con 0x5627a36b63d0 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e6ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e6ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46568/0 (socket says 172.21.15.26:46568) 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e6ed9d700 1 -- 172.21.15.26:0/3981478452 learned_addr learned my addr 172.21.15.26:0/3981478452 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.085+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 0x5627a36b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6ed9d700 1 -- 172.21.15.26:0/3981478452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36b63d0 msgr2=0x5627a36b87c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6ed9d700 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36b63d0 0x5627a36b87c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6ed9d700 1 -- 172.21.15.26:0/3981478452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 msgr2=0x5627a36b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6ed9d700 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 0x5627a36b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6ed9d700 1 -- 172.21.15.26:0/3981478452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a3429d80 con 0x5627a36aac00 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e5f7fe700 1 -- 172.21.15.26:0/1406859224 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0e68004ad0 con 0x7f0e7010fa90 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.086+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 0x5627a36b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.088+0000 7f0e5f7fe700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 0x7f0e580a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.088+0000 7f0e6ed9d700 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000b6c0 tx=0x7f0e60005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e74014a8d6772 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.088+0000 7f0e5f7fe700 1 -- 172.21.15.26:0/1406859224 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e680ccea0 con 0x7f0e7010fa90 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.089+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 0x7f0e580a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.089+0000 7f0e5cff9700 1 -- 172.21.15.26:0/3981478452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6001d070 con 0x5627a36aac00 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.089+0000 7f0e5cff9700 1 -- 172.21.15.26:0/3981478452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6001e040 con 0x5627a36aac00 2023-10-06T21:13:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.089+0000 7f0e5cff9700 1 -- 172.21.15.26:0/3981478452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60004a40 con 0x5627a36aac00 2023-10-06T21:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 -- 172.21.15.26:0/3981478452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 msgr2=0x5627a36aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000b6c0 tx=0x7f0e60005be0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 -- 172.21.15.26:0/3981478452 shutdown_connections 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36b63d0 0x5627a36b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36aafe0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/3981478452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36ab520 0x5627a36b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.090+0000 7f0e8889bb80 1 -- 172.21.15.26:0/3981478452 >> 172.21.15.26:0/3981478452 conn(0x5627a3620200 msgr2=0x5627a36a7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.091+0000 7f0e8889bb80 1 -- 172.21.15.26:0/3981478452 shutdown_connections 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.091+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 0x7f0e580a3b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007b60 tx=0x7f0e64008040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.091+0000 7f0e8889bb80 1 -- 172.21.15.26:0/3981478452 wait complete. 2023-10-06T21:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.091+0000 7f0e8889bb80 1 -- start start 2023-10-06T21:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 Processor -- start 2023-10-06T21:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36b63d0 0x5627a36b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a35fedd0 con 0x5627a36aac00 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a359b6b0 con 0x5627a36b63d0 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a359b350 con 0x5627a36ab520 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.092+0000 7f0e8889bb80 10 client.0 _ll_register_callbacks cb 0x5627a33f87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.093+0000 7f0e6dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.093+0000 7f0e6dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47630/0 (socket says 172.21.15.26:47630) 2023-10-06T21:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.093+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/2581194391 learned_addr learned my addr 172.21.15.26:0/2581194391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36b63d0 msgr2=0x5627a36b37c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36b63d0 0x5627a36b37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/2581194391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 msgr2=0x5627a36b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6dd9b700 1 -- 172.21.15.26:0/2581194391 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6400e040 con 0x5627a36ab520 2023-10-06T21:13:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.094+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.095+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009c10 tx=0x7f0e68099820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3027706d99ef3cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.095+0000 7f0e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.095+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68099ec0 con 0x5627a36ab520 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.095+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e6809e010 con 0x5627a36ab520 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.095+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627a369ed50 con 0x5627a36ab520 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e680a21c0 con 0x5627a36ab520 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 10 client.25885 fetch_fsmap learned FSMap version 29 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f0e7011d3b0 con 0x5627a36ab520 2023-10-06T21:13:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627a369ed50 con 0x5627a36ab520 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e68098e20 con 0x5627a36ab520 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 10 client.25885 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 20 client.25885 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a36c5eb0 0x5627a36c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 4 client.25885 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.096+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627a35f7db0 con 0x5627a36c5eb0 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.097+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a36c5eb0 0x5627a36c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.097+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a36c5eb0 0x5627a36c82d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11517a0fb3038a65 server_cookie=5622829c8ed135b0 in_seq=0 out_seq=0 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.097+0000 7f0e55ffb700 10 client.25885 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.098+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2581194391 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5627a35f7db0 con 0x5627a36c5eb0 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.098+0000 7f0e55ffb700 10 client.25885 handle_command_reply: tid=0 2023-10-06T21:13:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.098+0000 7f0e8889bb80 1 client.25885 shutdown 2023-10-06T21:13:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.100+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 msgr2=0x5627a36b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.100+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009c10 tx=0x7f0e68099820 comp rx=0 tx=0).stop 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 shutdown_connections 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a36c5eb0 0x5627a36c82d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36b63d0 0x5627a36b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36aac00 0x5627a36b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2581194391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36b30d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 >> 172.21.15.26:0/2581194391 conn(0x5627a3620200 msgr2=0x5627a36a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e55ffb700 0 client.25885 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 shutdown_connections 2023-10-06T21:13:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.101+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2581194391 wait complete. 2023-10-06T21:13:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.102+0000 7f0e8889bb80 20 client.25885 trim_cache size 0 max 16384 2023-10-06T21:13:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.125+0000 7f0e8889bb80 1 Processor -- start 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.125+0000 7f0e8889bb80 1 -- start start 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a347f8c0 con 0x5627a36b63d0 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36abb00 con 0x5627a36aac00 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36adf50 con 0x5627a36ab520 2023-10-06T21:13:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e6e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e6dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e6e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44042/0 (socket says 172.21.15.26:44042) 2023-10-06T21:13:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.126+0000 7f0e6e59c700 1 -- 172.21.15.26:0/4184064657 learned_addr learned my addr 172.21.15.26:0/4184064657 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6ed9d700 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 -- 172.21.15.26:0/4184064657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 msgr2=0x5627a36ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 -- 172.21.15.26:0/4184064657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 msgr2=0x5627a36b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 -- 172.21.15.26:0/4184064657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a3412c00 con 0x5627a36aac00 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e640109b0 tx=0x7f0e64013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c3bc987c19ed08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.127+0000 7f0e5d7fa700 1 -- 172.21.15.26:0/4184064657 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64026df0 con 0x5627a36aac00 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.128+0000 7f0e5d7fa700 1 -- 172.21.15.26:0/4184064657 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e64016480 con 0x5627a36aac00 2023-10-06T21:13:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.128+0000 7f0e5d7fa700 1 -- 172.21.15.26:0/4184064657 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64014690 con 0x5627a36aac00 2023-10-06T21:13:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 -- 172.21.15.26:0/4184064657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 msgr2=0x5627a36ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e640109b0 tx=0x7f0e64013000 comp rx=0 tx=0).stop 2023-10-06T21:13:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 -- 172.21.15.26:0/4184064657 shutdown_connections 2023-10-06T21:13:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a36ae3d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/4184064657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a36ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 -- 172.21.15.26:0/4184064657 >> 172.21.15.26:0/4184064657 conn(0x5627a3620200 msgr2=0x5627a36c1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 -- 172.21.15.26:0/4184064657 shutdown_connections 2023-10-06T21:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.129+0000 7f0e8889bb80 1 -- 172.21.15.26:0/4184064657 wait complete. 2023-10-06T21:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.130+0000 7f0e8889bb80 1 -- start start 2023-10-06T21:13:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.130+0000 7f0e8889bb80 1 Processor -- start 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a37134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36bb3e0 con 0x5627a36b63d0 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36bb550 con 0x5627a36aac00 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a36bb710 con 0x5627a36ab520 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e8889bb80 10 client.0 _ll_register_callbacks cb 0x7f0e68099d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44050/0 (socket says 172.21.15.26:44050) 2023-10-06T21:13:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6e59c700 1 -- 172.21.15.26:0/2512711176 learned_addr learned my addr 172.21.15.26:0/2512711176 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a37134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6e59c700 1 -- 172.21.15.26:0/2512711176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 msgr2=0x5627a37134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.132+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a37134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e6e59c700 1 -- 172.21.15.26:0/2512711176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 msgr2=0x5627a36baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e6e59c700 1 -- 172.21.15.26:0/2512711176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6401b040 con 0x5627a36aac00 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e6e59c700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e64016d60 tx=0x7f0e64026df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe494c3e8337dc89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64016800 con 0x5627a36aac00 2023-10-06T21:13:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.133+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e6001e040 con 0x5627a36aac00 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.134+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0e64026150 con 0x5627a36aac00 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.134+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e64026330 con 0x5627a36aac00 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.134+0000 7f0e8889bb80 10 client.25907 fetch_fsmap learned FSMap version 29 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.134+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5627a34501f0 con 0x5627a36aac00 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.134+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6001e040 con 0x5627a36aac00 2023-10-06T21:13:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e64030c70 con 0x5627a36aac00 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 10 client.25907 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 20 client.25907 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a371ad90 0x5627a371d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 4 client.25907 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:13:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.135+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5627a35f7ad0 con 0x5627a371ad90 2023-10-06T21:13:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.136+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a371ad90 0x5627a371d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.137+0000 7f0e6dd9b700 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a371ad90 0x5627a371d1b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a2f6ce7b6abfda server_cookie=36631f8c71ce078 in_seq=0 out_seq=0 2023-10-06T21:13:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.137+0000 7f0e55ffb700 10 client.25907 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.152+0000 7f0e55ffb700 1 -- 172.21.15.26:0/2512711176 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5627a35f7ad0 con 0x5627a371ad90 2023-10-06T21:13:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.152+0000 7f0e55ffb700 10 client.25907 handle_command_reply: tid=0 2023-10-06T21:13:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.152+0000 7f0e8889bb80 1 client.25907 shutdown 2023-10-06T21:13:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.152+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 msgr2=0x5627a3712f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.152+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e64016d60 tx=0x7f0e64026df0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 shutdown_connections 2023-10-06T21:13:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a36b63d0 0x5627a36baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5627a371ad90 0x5627a371d1b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5627a36aac00 0x5627a3712f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 --2- 172.21.15.26:0/2512711176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627a36ab520 0x5627a37134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 >> 172.21.15.26:0/2512711176 conn(0x5627a3620200 msgr2=0x5627a36ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e55ffb700 0 client.25907 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 shutdown_connections 2023-10-06T21:13:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.153+0000 7f0e8889bb80 1 -- 172.21.15.26:0/2512711176 wait complete. 2023-10-06T21:13:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:13:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:13:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:13:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:13:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.154+0000 7f0e8889bb80 20 client.25907 trim_cache size 0 max 16384 2023-10-06T21:13:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.156+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 msgr2=0x7f0e580a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.156+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 0x7f0e580a3b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007b60 tx=0x7f0e64008040 comp rx=0 tx=0).stop 2023-10-06T21:13:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.156+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 msgr2=0x7f0e701a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.156+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ef10 tx=0x7f0e6800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 shutdown_connections 2023-10-06T21:13:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e7011a690 0x7f0e701aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0e580a1670 0x7f0e580a3b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7010f620 0x7f0e701a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 --2- 172.21.15.26:0/1406859224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7010fa90 0x7f0e701a63b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 >> 172.21.15.26:0/1406859224 conn(0x7f0e7006cfa0 msgr2=0x7f0e70118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.157+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 shutdown_connections 2023-10-06T21:13:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:16.158+0000 7f0e74fff700 1 -- 172.21.15.26:0/1406859224 wait complete. 2023-10-06T21:13:16.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:16 smithi167 ceph-mon[134494]: pgmap v91: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 2.33k op/s 2023-10-06T21:13:16.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:16 smithi167 ceph-mon[134494]: daemon mds.i restarted 2023-10-06T21:13:16.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:16 smithi026 ceph-mon[98490]: pgmap v91: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 2.33k op/s 2023-10-06T21:13:16.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:16 smithi026 ceph-mon[98490]: daemon mds.i restarted 2023-10-06T21:13:16.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:16 smithi078 ceph-mon[134896]: pgmap v91: 97 pgs: 97 active+clean; 3.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 2.33k op/s 2023-10-06T21:13:16.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:16 smithi078 ceph-mon[134896]: daemon mds.i restarted 2023-10-06T21:13:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 155.0 s, 2667.4 tps, lat 37.388 ms stddev 40.602 2023-10-06T21:13:17.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:17 smithi167 ceph-mon[134494]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:13:17.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:17 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6838/3154348902,v1:172.21.15.167:6839/3154348902] up:boot 2023-10-06T21:13:17.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:17 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:13:17.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:17 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:13:17.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:17 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:17.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:17 smithi026 ceph-mon[98490]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:13:17.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:17 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6838/3154348902,v1:172.21.15.167:6839/3154348902] up:boot 2023-10-06T21:13:17.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:17 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:13:17.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:17 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:13:17.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:17 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:17.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:17 smithi078 ceph-mon[134896]: osdmap e89: 12 total, 12 up, 12 in 2023-10-06T21:13:17.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:17 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6838/3154348902,v1:172.21.15.167:6839/3154348902] up:boot 2023-10-06T21:13:17.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:17 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-06T21:13:17.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:17 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:13:17.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:17 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:18 smithi167 ceph-mon[134494]: pgmap v93: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:18.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:18 smithi026 ceph-mon[98490]: pgmap v93: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:18 smithi078 ceph-mon[134896]: pgmap v93: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:19.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:13:19.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.938+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/4285365879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 msgr2=0x7f2cf8107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.938+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/4285365879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf8107a50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009990 tx=0x7f2cec009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.939+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/4285365879 shutdown_connections 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.939+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/4285365879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf8114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.939+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/4285365879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.939+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/4285365879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf8107a50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.939+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/4285365879 >> 172.21.15.26:0/4285365879 conn(0x7f2cf8076d20 msgr2=0x7f2cf8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.940+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/4285365879 shutdown_connections 2023-10-06T21:13:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.940+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/4285365879 wait complete. 2023-10-06T21:13:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.941+0000 7f2cfd5ff700 1 Processor -- start 2023-10-06T21:13:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 -- start start 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a25d0 con 0x7f2cf81075d0 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a2740 con 0x7f2cf810d520 2023-10-06T21:13:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.942+0000 7f2cfd5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf81a2900 con 0x7f2cf8107f90 2023-10-06T21:13:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47668/0 (socket says 172.21.15.26:47668) 2023-10-06T21:13:19.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1365180314 learned_addr learned my addr 172.21.15.26:0/1365180314 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.943+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf81a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44074/0 (socket says 172.21.15.26:44074) 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1365180314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 msgr2=0x7f2cf81a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf81a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 msgr2=0x7f2cf819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1365180314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0095f0 con 0x7f2cf8107f90 2023-10-06T21:13:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:13:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.944+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005910 tx=0x7f2cec005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.945+0000 7f2cdffff700 1 -- 172.21.15.26:0/1365180314 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec01d070 con 0x7f2cf8107f90 2023-10-06T21:13:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.945+0000 7f2cdffff700 1 -- 172.21.15.26:0/1365180314 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cec022470 con 0x7f2cf8107f90 2023-10-06T21:13:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.945+0000 7f2cdffff700 1 -- 172.21.15.26:0/1365180314 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec017650 con 0x7f2cf8107f90 2023-10-06T21:13:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.945+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf81a2b80 con 0x7f2cf8107f90 2023-10-06T21:13:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.946+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf81a2e00 con 0x7f2cf8107f90 2023-10-06T21:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.952+0000 7f2d10fc1b80 1 Processor -- start 2023-10-06T21:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.955+0000 7f2d10fc1b80 1 -- start start 2023-10-06T21:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.955+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.955+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.955+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x5596691d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.956+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966912c460 con 0x5596691ce890 2023-10-06T21:13:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.956+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966911dc20 con 0x5596691c39e0 2023-10-06T21:13:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.956+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966911b540 con 0x5596691c30c0 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.957+0000 7f2cdffff700 1 -- 172.21.15.26:0/1365180314 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2cec00fad0 con 0x7f2cf8107f90 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691c34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47682/0 (socket says 172.21.15.26:47682) 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf77fe700 1 -- 172.21.15.26:0/1581472052 learned_addr learned my addr 172.21.15.26:0/1581472052 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/1581472052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 msgr2=0x5596691c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/1581472052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 msgr2=0x5596691d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x5596691d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.958+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/1581472052 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559668f41de0 con 0x5596691c39e0 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.959+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691ce350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8011440 tx=0x7f2ce8011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59870613f95cc45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.959+0000 7f2cdffff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 0x7f2ce00a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.960+0000 7f2cdd7fa700 1 -- 172.21.15.26:0/1581472052 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8004230 con 0x5596691c39e0 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.960+0000 7f2cdffff700 1 -- 172.21.15.26:0/1365180314 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2cec0ce380 con 0x7f2cf8107f90 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.960+0000 7f2cdd7fa700 1 -- 172.21.15.26:0/1581472052 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce8004cf0 con 0x5596691c39e0 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.960+0000 7f2cdd7fa700 1 -- 172.21.15.26:0/1581472052 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8003680 con 0x5596691c39e0 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.960+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 0x7f2ce00a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1581472052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 msgr2=0x5596691ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691ce350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8011440 tx=0x7f2ce8011750 comp rx=0 tx=0).stop 2023-10-06T21:13:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1581472052 shutdown_connections 2023-10-06T21:13:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x5596691d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1581472052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691ce350 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1581472052 >> 172.21.15.26:0/1581472052 conn(0x559669138260 msgr2=0x5596691bf450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.961+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 0x7f2ce00a3bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009a50 tx=0x7f2cf0009130 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.962+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1581472052 shutdown_connections 2023-10-06T21:13:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.962+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1581472052 wait complete. 2023-10-06T21:13:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.962+0000 7f2d10fc1b80 1 -- start start 2023-10-06T21:13:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.963+0000 7f2d10fc1b80 1 Processor -- start 2023-10-06T21:13:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 0x5596691cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691ce890 0x5596691cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559669116e30 con 0x5596691c39e0 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596690b3710 con 0x5596691c30c0 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596690b33b0 con 0x5596691ce890 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2d10fc1b80 10 client.0 _ll_register_callbacks cb 0x559668f10800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.964+0000 7f2cf67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691ce890 0x5596691cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46630/0 (socket says 172.21.15.26:46630) 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1226280078 learned_addr learned my addr 172.21.15.26:0/1226280078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1226280078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691ce890 msgr2=0x5596691cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691ce890 0x5596691cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 msgr2=0x5596691cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 0x5596691cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 0x5596691cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 -- 172.21.15.26:0/1226280078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce8009c90 con 0x5596691c39e0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 0x5596691cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.965+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2cec09b1c0 tx=0x7f2cec0cbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bcdc917b799d29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2cdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec09ae20 con 0x5596691c39e0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2cec09bbc0 con 0x5596691c39e0 2023-10-06T21:13:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596691b6f10 con 0x5596691c39e0 2023-10-06T21:13:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cec09b4c0 con 0x5596691c39e0 2023-10-06T21:13:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2d10fc1b80 10 client.15768 fetch_fsmap learned FSMap version 31 2023-10-06T21:13:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559668f2ac60 con 0x5596691c39e0 2023-10-06T21:13:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.966+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596691b6f10 con 0x5596691c39e0 2023-10-06T21:13:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2cec09a9b0 con 0x5596691c39e0 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 20 client.15768 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5596691ef110 0x5596691f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 4 client.15768 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.967+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55966910fe10 con 0x5596691ef110 2023-10-06T21:13:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.969+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5596691ef110 0x5596691f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.969+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5596691ef110 0x5596691f1530 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6466798cf5d3f8fd server_cookie=e6c42e37185f3fbe in_seq=0 out_seq=0 2023-10-06T21:13:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.969+0000 7f2cdaff5700 10 client.15768 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.970+0000 7f2cdaff5700 1 -- 172.21.15.26:0/1226280078 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55966910fe10 con 0x5596691ef110 2023-10-06T21:13:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.970+0000 7f2cdaff5700 10 client.15768 handle_command_reply: tid=0 2023-10-06T21:13:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.970+0000 7f2d10fc1b80 1 client.15768 shutdown 2023-10-06T21:13:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.971+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 msgr2=0x5596691cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.971+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2cec09b1c0 tx=0x7f2cec0cbfe0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 shutdown_connections 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691ce890 0x5596691cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5596691ef110 0x5596691f1530 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c30c0 0x5596691cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/1226280078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cbcd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 >> 172.21.15.26:0/1226280078 conn(0x559669138260 msgr2=0x5596691befe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2cdaff5700 0 client.15768 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 shutdown_connections 2023-10-06T21:13:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.972+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/1226280078 wait complete. 2023-10-06T21:13:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:19.974+0000 7f2d10fc1b80 20 client.15768 trim_cache size 0 max 16384 2023-10-06T21:13:20.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.007+0000 7f2d10fc1b80 1 Processor -- start 2023-10-06T21:13:20.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.007+0000 7f2d10fc1b80 1 -- start start 2023-10-06T21:13:20.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559668f97920 con 0x5596691c39e0 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596691c0280 con 0x5596691ce890 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596691d5910 con 0x5596691c30c0 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2cf6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2cf77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:44100/0 (socket says 172.21.15.26:44100) 2023-10-06T21:13:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2cf77fe700 1 -- 172.21.15.26:0/4275421043 learned_addr learned my addr 172.21.15.26:0/4275421043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.008+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 -- 172.21.15.26:0/4275421043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 msgr2=0x5596691d53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 -- 172.21.15.26:0/4275421043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 msgr2=0x5596691cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 -- 172.21.15.26:0/4275421043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596691bb140 con 0x5596691ce890 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.009+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0004590 tx=0x7f2cf0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0f1a4636d2cc09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.010+0000 7f2cddffb700 1 -- 172.21.15.26:0/4275421043 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0004db0 con 0x5596691ce890 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.010+0000 7f2cddffb700 1 -- 172.21.15.26:0/4275421043 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf000f040 con 0x5596691ce890 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.010+0000 7f2cddffb700 1 -- 172.21.15.26:0/4275421043 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0016d10 con 0x5596691ce890 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/4275421043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 msgr2=0x5596691c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0004590 tx=0x7f2cf0016000 comp rx=0 tx=0).stop 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/4275421043 shutdown_connections 2023-10-06T21:13:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691ce890 0x5596691c6ad0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/4275421043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691c39e0 0x5596691cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/4275421043 >> 172.21.15.26:0/4275421043 conn(0x559669138260 msgr2=0x5596691f1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/4275421043 shutdown_connections 2023-10-06T21:13:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/4275421043 wait complete. 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.011+0000 7f2d10fc1b80 1 -- start start 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.012+0000 7f2d10fc1b80 1 Processor -- start 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.012+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.012+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.012+0000 7f2d10fc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x55966924cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966924d490 con 0x5596691ce890 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966924d600 con 0x5596691c39e0 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2d10fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966924d7c0 con 0x5596691c30c0 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2d10fc1b80 10 client.0 _ll_register_callbacks cb 0x559668f10800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2cf6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2cf6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47698/0 (socket says 172.21.15.26:47698) 2023-10-06T21:13:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.013+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/963412810 learned_addr learned my addr 172.21.15.26:0/963412810 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x55966924cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 msgr2=0x5596691d35f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691d35f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/963412810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 msgr2=0x55966924cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x55966924cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 -- 172.21.15.26:0/963412810 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf0004280 con 0x5596691c30c0 2023-10-06T21:13:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf77fe700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x55966924cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.014+0000 7f2cf6ffd700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80097c0 tx=0x7f2ce8004fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38d42512fc86a5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.015+0000 7f2cdaff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:13:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.015+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80054a0 con 0x5596691c30c0 2023-10-06T21:13:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.015+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2ce8005600 con 0x5596691c30c0 2023-10-06T21:13:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.015+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ce8004cf0 con 0x5596691c30c0 2023-10-06T21:13:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce8028aa0 con 0x5596691c30c0 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2d10fc1b80 10 client.25909 fetch_fsmap learned FSMap version 31 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559668f68250 con 0x5596691c30c0 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce8004cf0 con 0x5596691c30c0 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2ce8003860 con 0x5596691c30c0 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2d10fc1b80 10 client.25909 fetch_fsmap finished waiting for FSMap version 31 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2d10fc1b80 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.016+0000 7f2d10fc1b80 20 client.25909 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.017+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559669254ee0 0x559669257300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.017+0000 7f2d10fc1b80 4 client.25909 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.017+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55966910fb30 con 0x559669254ee0 2023-10-06T21:13:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.017+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559669254ee0 0x559669257300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.018+0000 7f2cf67fc700 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559669254ee0 0x559669257300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239b82ab5ed1ee40 server_cookie=933314b6305f8124 in_seq=0 out_seq=0 2023-10-06T21:13:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.018+0000 7f2cdaff5700 10 client.25909 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.018+0000 7f2cdaff5700 1 -- 172.21.15.26:0/963412810 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55966910fb30 con 0x559669254ee0 2023-10-06T21:13:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.018+0000 7f2cdaff5700 10 client.25909 handle_command_reply: tid=0 2023-10-06T21:13:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.018+0000 7f2d10fc1b80 1 client.25909 shutdown 2023-10-06T21:13:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 msgr2=0x5596691d30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80097c0 tx=0x7f2ce8004fe0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 shutdown_connections 2023-10-06T21:13:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5596691ce890 0x55966924cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596691c30c0 0x5596691d30b0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80097c0 tx=0x7f2ce8004fe0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.019+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559669254ee0 0x559669257300 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.020+0000 7f2d10fc1b80 1 --2- 172.21.15.26:0/963412810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596691c39e0 0x5596691d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.020+0000 7f2cdaff5700 0 client.25909 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.020+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 >> 172.21.15.26:0/963412810 conn(0x559669138260 msgr2=0x5596691c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.020+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 shutdown_connections 2023-10-06T21:13:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.023+0000 7f2d10fc1b80 1 -- 172.21.15.26:0/963412810 wait complete. 2023-10-06T21:13:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.023+0000 7f2d10fc1b80 20 client.25909 trim_cache size 0 max 16384 2023-10-06T21:13:20.027 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:13:20.027 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:13:20.027 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:13:20.027 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:13:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 msgr2=0x7f2ce00a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 0x7f2ce00a3bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009a50 tx=0x7f2cf0009130 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 msgr2=0x7f2cf819db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005910 tx=0x7f2cec005740 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 shutdown_connections 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2cf810d520 0x7f2cf81a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.024+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f2ce00a16f0 0x7f2ce00a3bb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.025+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf81075d0 0x7f2cf819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.025+0000 7f2cfd5ff700 1 --2- 172.21.15.26:0/1365180314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cf8107f90 0x7f2cf819db30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.025+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 >> 172.21.15.26:0/1365180314 conn(0x7f2cf8076d20 msgr2=0x7f2cf8077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.026+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 shutdown_connections 2023-10-06T21:13:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:20.027+0000 7f2cfd5ff700 1 -- 172.21.15.26:0/1365180314 wait complete. 2023-10-06T21:13:20.058 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:13:20.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:20 smithi026 ceph-mon[98490]: pgmap v94: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:20.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:20 smithi167 ceph-mon[134494]: pgmap v94: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:20 smithi078 ceph-mon[134896]: pgmap v94: 97 pgs: 97 active+clean; 3.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2023-10-06T21:13:21.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:21 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:21.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:21 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:21.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:21 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 160.0 s, 2973.6 tps, lat 33.742 ms stddev 34.081 2023-10-06T21:13:22.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:22 smithi167 ceph-mon[134494]: pgmap v95: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:22.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:22 smithi026 ceph-mon[98490]: pgmap v95: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:22 smithi078 ceph-mon[134896]: pgmap v95: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:24.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:23 smithi167 ceph-mon[134494]: pgmap v96: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:23 smithi026 ceph-mon[98490]: pgmap v96: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:24.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:23 smithi078 ceph-mon[134896]: pgmap v96: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 759 op/s 2023-10-06T21:13:26.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:26 smithi167 ceph-mon[134494]: pgmap v97: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-10-06T21:13:26.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:26 smithi026 ceph-mon[98490]: pgmap v97: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-10-06T21:13:26.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:26 smithi078 ceph-mon[134896]: pgmap v97: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 787 op/s 2023-10-06T21:13:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 165.0 s, 3034.1 tps, lat 32.837 ms stddev 36.182 2023-10-06T21:13:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:28 smithi167 ceph-mon[134494]: pgmap v98: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 702 op/s 2023-10-06T21:13:28.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:28 smithi026 ceph-mon[98490]: pgmap v98: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 702 op/s 2023-10-06T21:13:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:28 smithi078 ceph-mon[134896]: pgmap v98: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 702 op/s 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 -- 172.21.15.26:0/224715347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 msgr2=0x7f1468073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 --2- 172.21.15.26:0/224715347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 0x7f1468073660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1458009990 tx=0x7f1458009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 -- 172.21.15.26:0/224715347 shutdown_connections 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 --2- 172.21.15.26:0/224715347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 0x7f146810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 --2- 172.21.15.26:0/224715347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 0x7f1468073660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 --2- 172.21.15.26:0/224715347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f146810d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 -- 172.21.15.26:0/224715347 >> 172.21.15.26:0/224715347 conn(0x7f14680fc940 msgr2=0x7f14680fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 -- 172.21.15.26:0/224715347 shutdown_connections 2023-10-06T21:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.399+0000 7f146e9ff700 1 -- 172.21.15.26:0/224715347 wait complete. 2023-10-06T21:13:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.401+0000 7f146e9ff700 1 Processor -- start 2023-10-06T21:13:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.401+0000 7f146e9ff700 1 -- start start 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 0x7f146810ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 0x7f146810a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146810a7e0 con 0x7f1468073280 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146810a950 con 0x7f1468073ba0 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146e9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146810ab10 con 0x7f146810d6f0 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.402+0000 7f146cf9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 0x7f146810a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:46586/0 (socket says 172.21.15.26:46586) 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 -- 172.21.15.26:0/1419257017 learned_addr learned my addr 172.21.15.26:0/1419257017 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 -- 172.21.15.26:0/1419257017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 msgr2=0x7f146810a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 0x7f146810a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 -- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 msgr2=0x7f146810ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 0x7f146810ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.403+0000 7f14677fe700 1 -- 172.21.15.26:0/1419257017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580095f0 con 0x7f1468073ba0 2023-10-06T21:13:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.404+0000 7f14677fe700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9a0 tx=0x7f145c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.404+0000 7f14657fa700 1 -- 172.21.15.26:0/1419257017 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c009790 con 0x7f1468073ba0 2023-10-06T21:13:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.404+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14681aa090 con 0x7f1468073ba0 2023-10-06T21:13:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.405+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14681aa5f0 con 0x7f1468073ba0 2023-10-06T21:13:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.405+0000 7f14657fa700 1 -- 172.21.15.26:0/1419257017 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145c00cbd0 con 0x7f1468073ba0 2023-10-06T21:13:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.405+0000 7f14657fa700 1 -- 172.21.15.26:0/1419257017 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c018680 con 0x7f1468073ba0 2023-10-06T21:13:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.410+0000 7f14657fa700 1 -- 172.21.15.26:0/1419257017 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f145c0187e0 con 0x7f1468073ba0 2023-10-06T21:13:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.413+0000 7f14657fa700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 0x7f14500a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.413+0000 7f14657fa700 1 -- 172.21.15.26:0/1419257017 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f145c014070 con 0x7f1468073ba0 2023-10-06T21:13:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.415+0000 7f148229ab80 1 Processor -- start 2023-10-06T21:13:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.415+0000 7f148229ab80 1 -- start start 2023-10-06T21:13:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 0x55a7852dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f1467fff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 0x7f14500a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 0x55a7852eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a7852ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a785249400 con 0x55a7852eb3d0 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78523abc0 con 0x55a7852e0520 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f148229ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7852384e0 con 0x55a7852dfc00 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.416+0000 7f1467fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 0x55a7852dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f1467fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 0x55a7852dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:48274/0 (socket says 172.21.15.26:48274) 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f1467fff700 1 -- 172.21.15.26:0/3408075505 learned_addr learned my addr 172.21.15.26:0/3408075505 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 0x55a7852eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f146cf9c700 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a7852ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 -- 172.21.15.26:0/3408075505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 msgr2=0x55a7852dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 0x55a7852dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 -- 172.21.15.26:0/3408075505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 msgr2=0x55a7852ed7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a7852ed7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.417+0000 7f14677fe700 1 -- 172.21.15.26:0/3408075505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78505ed80 con 0x55a7852e0520 2023-10-06T21:13:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.418+0000 7f1467fff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 0x7f14500a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14580098e0 tx=0x7f1458005510 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.419+0000 7f14677fe700 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 0x55a7852eae90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ca8d0 tx=0x7f145c09a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ffc827e55c5ccb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.419+0000 7f144e7fc700 1 -- 172.21.15.26:0/3408075505 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c09e2b0 con 0x55a7852e0520 2023-10-06T21:13:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.419+0000 7f144e7fc700 1 -- 172.21.15.26:0/3408075505 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145c0a24f0 con 0x55a7852e0520 2023-10-06T21:13:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.419+0000 7f144e7fc700 1 -- 172.21.15.26:0/3408075505 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c099890 con 0x55a7852e0520 2023-10-06T21:13:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 -- 172.21.15.26:0/3408075505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 msgr2=0x55a7852eae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 0x55a7852eae90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ca8d0 tx=0x7f145c09a860 comp rx=0 tx=0).stop 2023-10-06T21:13:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 -- 172.21.15.26:0/3408075505 shutdown_connections 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a7852ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852dfc00 0x55a7852dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 --2- 172.21.15.26:0/3408075505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e0520 0x55a7852eae90 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ca8d0 tx=0x7f145c09a860 comp rx=0 tx=0).stop 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.420+0000 7f148229ab80 1 -- 172.21.15.26:0/3408075505 >> 172.21.15.26:0/3408075505 conn(0x55a785255200 msgr2=0x55a7852dc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.421+0000 7f148229ab80 1 -- 172.21.15.26:0/3408075505 shutdown_connections 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.421+0000 7f148229ab80 1 -- 172.21.15.26:0/3408075505 wait complete. 2023-10-06T21:13:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.422+0000 7f148229ab80 1 -- start start 2023-10-06T21:13:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.422+0000 7f148229ab80 1 Processor -- start 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 0x55a7852e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e8d00 0x55a7852f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a785233dd0 con 0x55a7852dfc00 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7851d06b0 con 0x55a7852e8d00 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7851d0350 con 0x55a7852eb3d0 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f148229ab80 10 client.0 _ll_register_callbacks cb 0x55a78502d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f14677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f14677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:48278/0 (socket says 172.21.15.26:48278) 2023-10-06T21:13:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.423+0000 7f14677fe700 1 -- 172.21.15.26:0/3193079794 learned_addr learned my addr 172.21.15.26:0/3193079794 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 0x55a7852e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f146cf9c700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e8d00 0x55a7852f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 -- 172.21.15.26:0/3193079794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 msgr2=0x55a7852e87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 -- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e8d00 msgr2=0x55a7852f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e8d00 0x55a7852f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.424+0000 7f1467fff700 1 -- 172.21.15.26:0/3193079794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14681aa5f0 con 0x55a7852dfc00 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.425+0000 7f1467fff700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 0x55a7852e8280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1458006010 tx=0x7f1458026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5308c6d67e4a9946 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.425+0000 7f14477fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.425+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458032670 con 0x55a7852dfc00 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f14580327d0 con 0x55a7852dfc00 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7852d3d50 con 0x55a7852dfc00 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145800fe60 con 0x55a7852dfc00 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f148229ab80 10 client.15780 fetch_fsmap learned FSMap version 32 2023-10-06T21:13:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a785047c00 con 0x55a7852dfc00 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.426+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7852d3d50 con 0x55a7852dfc00 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1458039070 con 0x55a7852dfc00 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 20 client.15780 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a7852fb580 0x55a7852fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 4 client.15780 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78522cdb0 con 0x55a7852fb580 2023-10-06T21:13:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.427+0000 7f14677fe700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a7852fb580 0x55a7852fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.428+0000 7f14677fe700 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a7852fb580 0x55a7852fd9a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb049dcbcc188287 server_cookie=a80d3c9259d4f1c5 in_seq=0 out_seq=0 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.428+0000 7f14477fe700 10 client.15780 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.429+0000 7f14477fe700 1 -- 172.21.15.26:0/3193079794 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a78522cdb0 con 0x55a7852fb580 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.429+0000 7f14477fe700 10 client.15780 handle_command_reply: tid=0 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.429+0000 7f148229ab80 1 client.15780 shutdown 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.431+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 msgr2=0x55a7852e8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.431+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 0x55a7852e8280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1458006010 tx=0x7f1458026000 comp rx=0 tx=0).stop 2023-10-06T21:13:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 shutdown_connections 2023-10-06T21:13:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a7852fb580 0x55a7852fd9a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852e8d00 0x55a7852f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852dfc00 0x55a7852e8280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 --2- 172.21.15.26:0/3193079794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f14477fe700 0 client.15780 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.432+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 >> 172.21.15.26:0/3193079794 conn(0x55a785255200 msgr2=0x55a7852dbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.433+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 shutdown_connections 2023-10-06T21:13:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.433+0000 7f148229ab80 1 -- 172.21.15.26:0/3193079794 wait complete. 2023-10-06T21:13:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.433+0000 7f148229ab80 20 client.15780 trim_cache size 0 max 16384 2023-10-06T21:13:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.461+0000 7f148229ab80 1 Processor -- start 2023-10-06T21:13:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.464+0000 7f148229ab80 1 -- start start 2023-10-06T21:13:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7852f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7850b48c0 con 0x55a7852e8d00 2023-10-06T21:13:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7852e16c0 con 0x55a7852dfc00 2023-10-06T21:13:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.465+0000 7f148229ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7852f2490 con 0x55a7852eb3d0 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.466+0000 7f14677fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.466+0000 7f14677fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38140/0 (socket says 172.21.15.26:38140) 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.466+0000 7f14677fe700 1 -- 172.21.15.26:0/2589308000 learned_addr learned my addr 172.21.15.26:0/2589308000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f146cf9c700 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f14677fe700 1 -- 172.21.15.26:0/2589308000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 msgr2=0x55a7852e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f14677fe700 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f14677fe700 1 -- 172.21.15.26:0/2589308000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 msgr2=0x55a7852f6f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f14677fe700 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7852f6f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.467+0000 7f14677fe700 1 -- 172.21.15.26:0/2589308000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7850851f0 con 0x55a7852e8d00 2023-10-06T21:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.468+0000 7f14677fe700 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f145c099e60 tx=0x7f145c09a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5addb411b98d8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.468+0000 7f144effd700 1 -- 172.21.15.26:0/2589308000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c098580 con 0x55a7852e8d00 2023-10-06T21:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.468+0000 7f144effd700 1 -- 172.21.15.26:0/2589308000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145c00cd40 con 0x55a7852e8d00 2023-10-06T21:13:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.468+0000 7f144effd700 1 -- 172.21.15.26:0/2589308000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c099700 con 0x55a7852e8d00 2023-10-06T21:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 -- 172.21.15.26:0/2589308000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 msgr2=0x55a7852f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f145c099e60 tx=0x7f145c09a9d0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 -- 172.21.15.26:0/2589308000 shutdown_connections 2023-10-06T21:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852eb3d0 0x55a7852e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7852f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 --2- 172.21.15.26:0/2589308000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852e8d00 0x55a7852f1f50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.469+0000 7f148229ab80 1 -- 172.21.15.26:0/2589308000 >> 172.21.15.26:0/2589308000 conn(0x55a785255200 msgr2=0x55a7852f7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.470+0000 7f148229ab80 1 -- 172.21.15.26:0/2589308000 shutdown_connections 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.470+0000 7f148229ab80 1 -- 172.21.15.26:0/2589308000 wait complete. 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.470+0000 7f148229ab80 1 -- start start 2023-10-06T21:13:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.470+0000 7f148229ab80 1 Processor -- start 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.471+0000 7f148229ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7853485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.471+0000 7f148229ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 0x55a785348b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.471+0000 7f148229ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a785343340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f148229ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7853438d0 con 0x55a7852eb3d0 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f148229ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a785343a40 con 0x55a7852dfc00 2023-10-06T21:13:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f148229ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a785343c00 con 0x55a7852e8d00 2023-10-06T21:13:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f148229ab80 10 client.0 _ll_register_callbacks cb 0x55a78502d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f146cf9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a785343340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f146cf9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a785343340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38152/0 (socket says 172.21.15.26:38152) 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f146cf9c700 1 -- 172.21.15.26:0/2381107220 learned_addr learned my addr 172.21.15.26:0/2381107220 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.472+0000 7f14677fe700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 0x55a785348b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.473+0000 7f14677fe700 1 -- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 msgr2=0x55a7853485f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.473+0000 7f14677fe700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7853485f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.473+0000 7f14677fe700 1 -- 172.21.15.26:0/2381107220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 msgr2=0x55a785343340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.473+0000 7f14677fe700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a785343340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.473+0000 7f14677fe700 1 -- 172.21.15.26:0/2381107220 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c0b1040 con 0x55a7852e8d00 2023-10-06T21:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.474+0000 7f14677fe700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 0x55a785348b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a7852e97c0 tx=0x7f145c0b8550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc09e170e93acbbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.474+0000 7f14477fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.474+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c098580 con 0x55a7852e8d00 2023-10-06T21:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.474+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f145c09a3c0 con 0x55a7852e8d00 2023-10-06T21:13:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.476+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f145c0a24f0 con 0x55a7852e8d00 2023-10-06T21:13:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.493+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145c09a580 con 0x55a7852e8d00 2023-10-06T21:13:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.493+0000 7f148229ab80 10 client.25927 fetch_fsmap learned FSMap version 32 2023-10-06T21:13:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.493+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a78505d250 con 0x55a7852e8d00 2023-10-06T21:13:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145c0a24f0 con 0x55a7852e8d00 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f145c0a2060 con 0x55a7852e8d00 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 20 client.25927 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a785360ea0 0x55a785363290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 4 client.25927 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:13:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.504+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a78522cad0 con 0x55a785360ea0 2023-10-06T21:13:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.516+0000 7f1467fff700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a785360ea0 0x55a785363290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.518+0000 7f1467fff700 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a785360ea0 0x55a785363290 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e472777c9c58a0d server_cookie=6622366f87c07bbd in_seq=0 out_seq=0 2023-10-06T21:13:29.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.518+0000 7f14477fe700 10 client.25927 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.544+0000 7f14477fe700 1 -- 172.21.15.26:0/2381107220 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a78522cad0 con 0x55a785360ea0 2023-10-06T21:13:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.544+0000 7f14477fe700 10 client.25927 handle_command_reply: tid=0 2023-10-06T21:13:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.544+0000 7f148229ab80 1 client.25927 shutdown 2023-10-06T21:13:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 msgr2=0x55a785348b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 0x55a785348b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55a7852e97c0 tx=0x7f145c0b8550 comp rx=0 tx=0).stop 2023-10-06T21:13:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 shutdown_connections 2023-10-06T21:13:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7852eb3d0 0x55a785343340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a785360ea0 0x55a785363290 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7852dfc00 0x55a7853485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 --2- 172.21.15.26:0/2381107220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a7852e8d00 0x55a785348b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 >> 172.21.15.26:0/2381107220 conn(0x55a785255200 msgr2=0x55a7852e9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.548+0000 7f14477fe700 0 client.25927 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.549+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 shutdown_connections 2023-10-06T21:13:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.549+0000 7f148229ab80 1 -- 172.21.15.26:0/2381107220 wait complete. 2023-10-06T21:13:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.549+0000 7f148229ab80 20 client.25927 trim_cache size 0 max 16384 2023-10-06T21:13:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:13:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:13:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:13:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:13:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 msgr2=0x7f14500a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 0x7f14500a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14580098e0 tx=0x7f1458005510 comp rx=0 tx=0).stop 2023-10-06T21:13:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 msgr2=0x7f1468109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9a0 tx=0x7f145c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 shutdown_connections 2023-10-06T21:13:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f146810d6f0 0x7f146810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f14500a1670 0x7f14500a3b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1468073280 0x7f146810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 --2- 172.21.15.26:0/1419257017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1468073ba0 0x7f1468109c50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.552+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 >> 172.21.15.26:0/1419257017 conn(0x7f14680fc940 msgr2=0x7f146810f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.553+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 shutdown_connections 2023-10-06T21:13:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:29.553+0000 7f146e9ff700 1 -- 172.21.15.26:0/1419257017 wait complete. 2023-10-06T21:13:30.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:30 smithi167 ceph-mon[134494]: pgmap v99: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-10-06T21:13:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:30 smithi026 ceph-mon[98490]: pgmap v99: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-10-06T21:13:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:30 smithi078 ceph-mon[134896]: pgmap v99: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-10-06T21:13:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:31 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:31.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:31 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:31.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:31 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:32.128 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 170.0 s, 2303.7 tps, lat 43.318 ms stddev 48.305 2023-10-06T21:13:32.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:32 smithi167 ceph-mon[134494]: pgmap v100: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 985 op/s 2023-10-06T21:13:32.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:32 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:32 smithi026 ceph-mon[98490]: pgmap v100: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 985 op/s 2023-10-06T21:13:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:32 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:32.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:32 smithi078 ceph-mon[134896]: pgmap v100: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 985 op/s 2023-10-06T21:13:32.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:32 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:13:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:34 smithi167 ceph-mon[134494]: pgmap v101: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 653 op/s 2023-10-06T21:13:34.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:34.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:34.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:13:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:34 smithi026 ceph-mon[98490]: pgmap v101: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 653 op/s 2023-10-06T21:13:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:13:34.376 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:34 smithi078 ceph-mon[134896]: pgmap v101: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 653 op/s 2023-10-06T21:13:34.377 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:13:34.377 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:13:34.377 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:13:36.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:36 smithi167 ceph-mon[134494]: pgmap v102: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 992 op/s 2023-10-06T21:13:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:36 smithi026 ceph-mon[98490]: pgmap v102: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 992 op/s 2023-10-06T21:13:36.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:36 smithi078 ceph-mon[134896]: pgmap v102: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 992 op/s 2023-10-06T21:13:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 175.0 s, 2105.2 tps, lat 47.572 ms stddev 51.289 2023-10-06T21:13:38.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:38 smithi167 ceph-mon[134494]: pgmap v103: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 796 op/s 2023-10-06T21:13:38.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:38 smithi026 ceph-mon[98490]: pgmap v103: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 796 op/s 2023-10-06T21:13:38.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:38 smithi078 ceph-mon[134896]: pgmap v103: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 796 op/s 2023-10-06T21:13:40.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:40 smithi167 ceph-mon[134494]: pgmap v104: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 745 op/s 2023-10-06T21:13:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:40 smithi026 ceph-mon[98490]: pgmap v104: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 745 op/s 2023-10-06T21:13:40.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:40 smithi078 ceph-mon[134896]: pgmap v104: 97 pgs: 97 active+clean; 4.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 745 op/s 2023-10-06T21:13:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 180.0 s, 2213.2 tps, lat 44.786 ms stddev 50.329 2023-10-06T21:13:42.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:42 smithi167 ceph-mon[134494]: pgmap v105: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2023-10-06T21:13:42.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:42 smithi026 ceph-mon[98490]: pgmap v105: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2023-10-06T21:13:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:42 smithi078 ceph-mon[134896]: pgmap v105: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2023-10-06T21:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.807+0000 7f04e7bff700 1 -- 172.21.15.26:0/953066438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e0103300 msgr2=0x7f04e01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.807+0000 7f04e7bff700 1 --2- 172.21.15.26:0/953066438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e0103300 0x7f04e01036e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009990 tx=0x7f04dc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 -- 172.21.15.26:0/953066438 shutdown_connections 2023-10-06T21:13:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 --2- 172.21.15.26:0/953066438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0108420 0x7f04e010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 --2- 172.21.15.26:0/953066438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e0103300 0x7f04e01036e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 --2- 172.21.15.26:0/953066438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0103cb0 0x7f04e0107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 -- 172.21.15.26:0/953066438 >> 172.21.15.26:0/953066438 conn(0x7f04e00feb60 msgr2=0x7f04e0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 -- 172.21.15.26:0/953066438 shutdown_connections 2023-10-06T21:13:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.808+0000 7f04e7bff700 1 -- 172.21.15.26:0/953066438 wait complete. 2023-10-06T21:13:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 Processor -- start 2023-10-06T21:13:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 -- start start 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0108420 0x7f04e00729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e010c9d0 0x7f04e010ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e01103a0 con 0x7f04e010c9d0 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e010d3c0 con 0x7f04e0108420 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.810+0000 7f04e7bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e010d550 con 0x7f04e0103300 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.811+0000 7f04e599b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.811+0000 7f04e599b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:50808/0 (socket says 172.21.15.26:50808) 2023-10-06T21:13:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.811+0000 7f04e599b700 1 -- 172.21.15.26:0/863738195 learned_addr learned my addr 172.21.15.26:0/863738195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.811+0000 7f04e619c700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e010c9d0 0x7f04e010ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e519a700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0108420 0x7f04e00729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e599b700 1 -- 172.21.15.26:0/863738195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0108420 msgr2=0x7f04e00729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e599b700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0108420 0x7f04e00729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e599b700 1 -- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e010c9d0 msgr2=0x7f04e010ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e599b700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e010c9d0 0x7f04e010ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.812+0000 7f04e599b700 1 -- 172.21.15.26:0/863738195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0095f0 con 0x7f04e0103300 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04e599b700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005770 tx=0x7f04dc005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04d6ffd700 1 -- 172.21.15.26:0/863738195 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc01d070 con 0x7f04e0103300 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e010d7d0 con 0x7f04e0103300 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04d6ffd700 1 -- 172.21.15.26:0/863738195 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04dc003fe0 con 0x7f04e0103300 2023-10-06T21:13:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04d6ffd700 1 -- 172.21.15.26:0/863738195 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc022d00 con 0x7f04e0103300 2023-10-06T21:13:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.813+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e01aa920 con 0x7f04e0103300 2023-10-06T21:13:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.821+0000 7f04fb568b80 1 Processor -- start 2023-10-06T21:13:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.821+0000 7f04d6ffd700 1 -- 172.21.15.26:0/863738195 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f04dc003b20 con 0x7f04e0103300 2023-10-06T21:13:42.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.821+0000 7f04fb568b80 1 -- start start 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f73ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f73d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f699400 con 0x557c1f72fc00 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f68abc0 con 0x557c1f73b3d0 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.822+0000 7f04fb568b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f6884e0 con 0x557c1f730520 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.823+0000 7f04e619c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.823+0000 7f04e619c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60376/0 (socket says 172.21.15.26:60376) 2023-10-06T21:13:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.823+0000 7f04e619c700 1 -- 172.21.15.26:0/2325776697 learned_addr learned my addr 172.21.15.26:0/2325776697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.824+0000 7f04d6ffd700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 0x7f04cc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.824+0000 7f04e619c700 1 -- 172.21.15.26:0/2325776697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 msgr2=0x557c1f73ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.825+0000 7f04e519a700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f73d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.825+0000 7f04e599b700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f73ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.825+0000 7f04e619c700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f73ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.825+0000 7f04e619c700 1 -- 172.21.15.26:0/2325776697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 msgr2=0x557c1f73d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.825+0000 7f04e619c700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f73d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04e619c700 1 -- 172.21.15.26:0/2325776697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1f4aed80 con 0x557c1f72fc00 2023-10-06T21:13:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04e519a700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f73d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04d6ffd700 1 -- 172.21.15.26:0/863738195 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04dc0ce4b0 con 0x7f04e0103300 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04e619c700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 0x7f04cc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04e619c700 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c4a0 tx=0x7f04d800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291d69e86fb3442e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04e619c700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 0x7f04cc0a3b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d801d000 tx=0x7f04d8018000 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.826+0000 7f04cbfff700 1 -- 172.21.15.26:0/2325776697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8018440 con 0x557c1f72fc00 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.827+0000 7f04cbfff700 1 -- 172.21.15.26:0/2325776697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d8017860 con 0x557c1f72fc00 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.827+0000 7f04cbfff700 1 -- 172.21.15.26:0/2325776697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8021610 con 0x557c1f72fc00 2023-10-06T21:13:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.827+0000 7f04fb568b80 1 -- 172.21.15.26:0/2325776697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 msgr2=0x557c1f72ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.827+0000 7f04fb568b80 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c4a0 tx=0x7f04d800c4d0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.828+0000 7f04fb568b80 1 -- 172.21.15.26:0/2325776697 shutdown_connections 2023-10-06T21:13:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.828+0000 7f04fb568b80 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f73d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.828+0000 7f04fb568b80 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f72ffe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.828+0000 7f04fb568b80 1 --2- 172.21.15.26:0/2325776697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f73ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.828+0000 7f04fb568b80 1 -- 172.21.15.26:0/2325776697 >> 172.21.15.26:0/2325776697 conn(0x557c1f6a5200 msgr2=0x557c1f72c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.830+0000 7f04fb568b80 1 -- 172.21.15.26:0/2325776697 shutdown_connections 2023-10-06T21:13:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.830+0000 7f04fb568b80 1 -- 172.21.15.26:0/2325776697 wait complete. 2023-10-06T21:13:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 -- start start 2023-10-06T21:13:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 Processor -- start 2023-10-06T21:13:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f730520 0x557c1f738050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f73b3d0 0x557c1f7386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f683dd0 con 0x557c1f730520 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f6206b0 con 0x557c1f72fc00 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f620350 con 0x557c1f73b3d0 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.831+0000 7f04fb568b80 10 client.0 _ll_register_callbacks cb 0x557c1f47d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:47734/0 (socket says 172.21.15.26:47734) 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 -- 172.21.15.26:0/335663155 learned_addr learned my addr 172.21.15.26:0/335663155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e519a700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f730520 0x557c1f738050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e619c700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f73b3d0 0x557c1f7386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 -- 172.21.15.26:0/335663155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f73b3d0 msgr2=0x557c1f7386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f73b3d0 0x557c1f7386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 -- 172.21.15.26:0/335663155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f730520 msgr2=0x557c1f738050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f730520 0x557c1f738050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 -- 172.21.15.26:0/335663155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d8009c20 con 0x557c1f72fc00 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04e599b700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04dc09b460 tx=0x7f04dc0cc110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676efae63079ead3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.832+0000 7f04c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc09abc0 con 0x557c1f72fc00 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04dc022470 con 0x557c1f72fc00 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1f723d50 con 0x557c1f72fc00 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04dc09bbd0 con 0x557c1f72fc00 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04fb568b80 10 client.25940 fetch_fsmap learned FSMap version 33 2023-10-06T21:13:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557c1f497c00 con 0x557c1f72fc00 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c1f723d50 con 0x557c1f72fc00 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.833+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f04dc0a3460 con 0x557c1f72fc00 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 20 client.25940 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f74aac0 0x557c1f74cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 4 client.25940 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.834+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c1f67cdb0 con 0x557c1f74aac0 2023-10-06T21:13:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.837+0000 7f04e519a700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f74aac0 0x557c1f74cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.837+0000 7f04e519a700 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f74aac0 0x557c1f74cee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a1cd0abab4567e server_cookie=483ef64a9a3ca1bd in_seq=0 out_seq=0 2023-10-06T21:13:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.837+0000 7f04c97fa700 10 client.25940 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.838+0000 7f04c97fa700 1 -- 172.21.15.26:0/335663155 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c1f67cdb0 con 0x557c1f74aac0 2023-10-06T21:13:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.838+0000 7f04c97fa700 10 client.25940 handle_command_reply: tid=0 2023-10-06T21:13:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.838+0000 7f04fb568b80 1 client.25940 shutdown 2023-10-06T21:13:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.838+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 msgr2=0x557c1f73add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.838+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04dc09b460 tx=0x7f04dc0cc110 comp rx=0 tx=0).stop 2023-10-06T21:13:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 shutdown_connections 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f73b3d0 0x557c1f7386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f74aac0 0x557c1f74cee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f72fc00 0x557c1f73add0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 --2- 172.21.15.26:0/335663155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f730520 0x557c1f738050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04c97fa700 0 client.25940 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 >> 172.21.15.26:0/335663155 conn(0x557c1f6a5200 msgr2=0x557c1f72bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.839+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 shutdown_connections 2023-10-06T21:13:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.841+0000 7f04fb568b80 1 -- 172.21.15.26:0/335663155 wait complete. 2023-10-06T21:13:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.842+0000 7f04fb568b80 20 client.25940 trim_cache size 0 max 16384 2023-10-06T21:13:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.865+0000 7f04fb568b80 1 Processor -- start 2023-10-06T21:13:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.865+0000 7f04fb568b80 1 -- start start 2023-10-06T21:13:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 0x557c1f737890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f73b3d0 0x557c1f737dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f5048c0 con 0x557c1f73b3d0 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f738fe0 con 0x557c1f730520 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04fb568b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f7313f0 con 0x557c1f72fc00 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04e599b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04e599b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:50830/0 (socket says 172.21.15.26:50830) 2023-10-06T21:13:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.866+0000 7f04e599b700 1 -- 172.21.15.26:0/4019156820 learned_addr learned my addr 172.21.15.26:0/4019156820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e519a700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 0x557c1f737890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e619c700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f73b3d0 0x557c1f737dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e599b700 1 -- 172.21.15.26:0/4019156820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 msgr2=0x557c1f737890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e599b700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 0x557c1f737890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e599b700 1 -- 172.21.15.26:0/4019156820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f73b3d0 msgr2=0x557c1f737dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e599b700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f73b3d0 0x557c1f737dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e599b700 1 -- 172.21.15.26:0/4019156820 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1f4d51f0 con 0x557c1f72fc00 2023-10-06T21:13:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.867+0000 7f04e519a700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 0x557c1f737890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:13:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.868+0000 7f04e599b700 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04dc09b1d0 tx=0x7f04dc099a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b676d3800c966b5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.868+0000 7f04d4bff700 1 -- 172.21.15.26:0/4019156820 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc09abc0 con 0x557c1f72fc00 2023-10-06T21:13:42.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.868+0000 7f04d4bff700 1 -- 172.21.15.26:0/4019156820 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04dc022470 con 0x557c1f72fc00 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.868+0000 7f04d4bff700 1 -- 172.21.15.26:0/4019156820 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0a3580 con 0x557c1f72fc00 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 -- 172.21.15.26:0/4019156820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 msgr2=0x557c1f73a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04dc09b1d0 tx=0x7f04dc099a80 comp rx=0 tx=0).stop 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 -- 172.21.15.26:0/4019156820 shutdown_connections 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f73b3d0 0x557c1f737dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f72fc00 0x557c1f73a970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 --2- 172.21.15.26:0/4019156820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f730520 0x557c1f737890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 -- 172.21.15.26:0/4019156820 >> 172.21.15.26:0/4019156820 conn(0x557c1f6a5200 msgr2=0x557c1f733430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.869+0000 7f04fb568b80 1 -- 172.21.15.26:0/4019156820 shutdown_connections 2023-10-06T21:13:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.870+0000 7f04fb568b80 1 -- 172.21.15.26:0/4019156820 wait complete. 2023-10-06T21:13:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.870+0000 7f04fb568b80 1 -- start start 2023-10-06T21:13:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.871+0000 7f04fb568b80 1 Processor -- start 2023-10-06T21:13:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.871+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f740470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f7409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f79da20 con 0x557c1f72fc00 2023-10-06T21:13:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f79db90 con 0x557c1f73b3d0 2023-10-06T21:13:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1f79dd50 con 0x557c1f730520 2023-10-06T21:13:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.872+0000 7f04fb568b80 10 client.0 _ll_register_callbacks cb 0x7f04dc09b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.873+0000 7f04e599b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60418/0 (socket says 172.21.15.26:60418) 2023-10-06T21:13:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 -- 172.21.15.26:0/1849415717 learned_addr learned my addr 172.21.15.26:0/1849415717 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 -- 172.21.15.26:0/1849415717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 msgr2=0x557c1f740470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f740470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 -- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 msgr2=0x557c1f7409b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f7409b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.874+0000 7f04e599b700 1 -- 172.21.15.26:0/1849415717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d000bd70 con 0x557c1f72fc00 2023-10-06T21:13:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04e599b700 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f04dc097f40 tx=0x7f04dc0a3620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff6ad61780afc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0d00d0 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f04dc0a37d0 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04d8017860 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04dc09b4d0 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.875+0000 7f04fb568b80 10 client.15804 fetch_fsmap learned FSMap version 33 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557c1f497c00 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d8017860 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f04dc084ab0 con 0x557c1f72fc00 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 33 2023-10-06T21:13:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 20 client.15804 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f79efc0 0x557c1f7a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 4 client.15804 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:13:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c1f67cad0 con 0x557c1f79efc0 2023-10-06T21:13:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.876+0000 7f04e519a700 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f79efc0 0x557c1f7a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.880+0000 7f04c97fa700 10 client.15804 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.880+0000 7f04e519a700 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f79efc0 0x557c1f7a13b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce898b4d160e0f9a server_cookie=cf2bc4d8c86556b9 in_seq=0 out_seq=0 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04c97fa700 1 -- 172.21.15.26:0/1849415717 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c1f67cad0 con 0x557c1f79efc0 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04c97fa700 10 client.15804 handle_command_reply: tid=0 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 client.15804 shutdown 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 msgr2=0x557c1f73ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f04dc097f40 tx=0x7f04dc0a3620 comp rx=0 tx=0).stop 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 shutdown_connections 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c1f79efc0 0x557c1f7a13b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1f73b3d0 0x557c1f7409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c1f72fc00 0x557c1f73ff30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04c97fa700 0 client.15804 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 --2- 172.21.15.26:0/1849415717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c1f730520 0x557c1f740470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.894+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 >> 172.21.15.26:0/1849415717 conn(0x557c1f6a5200 msgr2=0x557c1f72b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.895+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 shutdown_connections 2023-10-06T21:13:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.895+0000 7f04fb568b80 1 -- 172.21.15.26:0/1849415717 wait complete. 2023-10-06T21:13:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.895+0000 7f04fb568b80 20 client.15804 trim_cache size 0 max 16384 2023-10-06T21:13:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:13:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:13:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:13:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 msgr2=0x7f04cc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 0x7f04cc0a3b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d801d000 tx=0x7f04d8018000 comp rx=0 tx=0).stop 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 msgr2=0x7f04e010fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005770 tx=0x7f04dc005900 comp rx=0 tx=0).stop 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 shutdown_connections 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f04cc0a16b0 0x7f04cc0a3b70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e010c9d0 0x7f04e010ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04e0103300 0x7f04e010fbd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 --2- 172.21.15.26:0/863738195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04e0108420 0x7f04e00729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 >> 172.21.15.26:0/863738195 conn(0x7f04e00feb60 msgr2=0x7f04e0106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 shutdown_connections 2023-10-06T21:13:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:42.896+0000 7f04e7bff700 1 -- 172.21.15.26:0/863738195 wait complete. 2023-10-06T21:13:44.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:44 smithi167 ceph-mon[134494]: pgmap v106: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 742 op/s 2023-10-06T21:13:44.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:44 smithi026 ceph-mon[98490]: pgmap v106: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 742 op/s 2023-10-06T21:13:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:44 smithi078 ceph-mon[134896]: pgmap v106: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 742 op/s 2023-10-06T21:13:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:46 smithi167 ceph-mon[134494]: pgmap v107: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 1.92k op/s 2023-10-06T21:13:46.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:46 smithi026 ceph-mon[98490]: pgmap v107: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 1.92k op/s 2023-10-06T21:13:46.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:46 smithi078 ceph-mon[134896]: pgmap v107: 97 pgs: 97 active+clean; 4.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 1.92k op/s 2023-10-06T21:13:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 185.0 s, 1760.4 tps, lat 57.064 ms stddev 90.581 2023-10-06T21:13:48.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:48 smithi167 ceph-mon[134494]: pgmap v108: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2023-10-06T21:13:48.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:48 smithi026 ceph-mon[98490]: pgmap v108: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2023-10-06T21:13:48.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:48 smithi078 ceph-mon[134896]: pgmap v108: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2023-10-06T21:13:50.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.402+0000 7f1843fff700 1 -- 172.21.15.26:0/2135511406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103b10 msgr2=0x7f1844107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.402+0000 7f1843fff700 1 --2- 172.21.15.26:0/2135511406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103b10 0x7f1844107b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1838009990 tx=0x7f1838009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.403+0000 7f1843fff700 1 -- 172.21.15.26:0/2135511406 shutdown_connections 2023-10-06T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.403+0000 7f1843fff700 1 --2- 172.21.15.26:0/2135511406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 0x7f184410fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.403+0000 7f1843fff700 1 --2- 172.21.15.26:0/2135511406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1844103160 0x7f1844103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.403+0000 7f1843fff700 1 --2- 172.21.15.26:0/2135511406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103b10 0x7f1844107b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.403+0000 7f1843fff700 1 -- 172.21.15.26:0/2135511406 >> 172.21.15.26:0/2135511406 conn(0x7f18440fe9c0 msgr2=0x7f1844100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.404+0000 7f1843fff700 1 -- 172.21.15.26:0/2135511406 shutdown_connections 2023-10-06T21:13:50.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.404+0000 7f1843fff700 1 -- 172.21.15.26:0/2135511406 wait complete. 2023-10-06T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.405+0000 7f1843fff700 1 Processor -- start 2023-10-06T21:13:50.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 -- start start 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 0x7f1844198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 0x7f18441994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 0x7f184419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184419e3d0 con 0x7f184419d950 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184419e540 con 0x7f1844108280 2023-10-06T21:13:50.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.406+0000 7f1843fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184419e700 con 0x7f1844103160 2023-10-06T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.407+0000 7f18437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 0x7f184419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.407+0000 7f18437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 0x7f184419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60442/0 (socket says 172.21.15.26:60442) 2023-10-06T21:13:50.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.407+0000 7f18437fe700 1 -- 172.21.15.26:0/808221969 learned_addr learned my addr 172.21.15.26:0/808221969 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:50.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 0x7f1844198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f18427fc700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 0x7f18441994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 -- 172.21.15.26:0/808221969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 msgr2=0x7f18441994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 0x7f18441994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 -- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 msgr2=0x7f184419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 0x7f184419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.409+0000 7f1842ffd700 1 -- 172.21.15.26:0/808221969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380095f0 con 0x7f1844103160 2023-10-06T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.410+0000 7f1842ffd700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 0x7f1844198fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f183400e9f0 tx=0x7f183400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.410+0000 7f18489fd700 1 -- 172.21.15.26:0/808221969 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183400eee0 con 0x7f1844103160 2023-10-06T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.410+0000 7f18489fd700 1 -- 172.21.15.26:0/808221969 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183400cbd0 con 0x7f1844103160 2023-10-06T21:13:50.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.410+0000 7f18489fd700 1 -- 172.21.15.26:0/808221969 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834018680 con 0x7f1844103160 2023-10-06T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.411+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18441a5c60 con 0x7f1844103160 2023-10-06T21:13:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.411+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18441a60f0 con 0x7f1844103160 2023-10-06T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.419+0000 7f185d300b80 1 Processor -- start 2023-10-06T21:13:50.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.419+0000 7f185d300b80 1 -- start start 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f018680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f018bc0 0x55636f023530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef81460 con 0x55636f0182a0 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef72c20 con 0x55636f023a70 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.420+0000 7f185d300b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef70540 con 0x55636f018bc0 2023-10-06T21:13:50.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18489fd700 1 -- 172.21.15.26:0/808221969 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f183400cd40 con 0x7f1844103160 2023-10-06T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:47774/0 (socket says 172.21.15.26:47774) 2023-10-06T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 -- 172.21.15.26:0/3245561664 learned_addr learned my addr 172.21.15.26:0/3245561664 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:50.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f1842ffd700 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f018bc0 0x55636f023530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18437fe700 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f018680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 -- 172.21.15.26:0/3245561664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f018bc0 msgr2=0x55636f023530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f018bc0 0x55636f023530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 -- 172.21.15.26:0/3245561664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 msgr2=0x55636f018680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f018680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.421+0000 7f18427fc700 1 -- 172.21.15.26:0/3245561664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55636ed96de0 con 0x55636f023a70 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.422+0000 7f18427fc700 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1838005b40 tx=0x7f1838005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbca8e2a6f4fa59a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.422+0000 7f1829ffb700 1 -- 172.21.15.26:0/3245561664 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183801e070 con 0x55636f023a70 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.422+0000 7f1829ffb700 1 -- 172.21.15.26:0/3245561664 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183800bc90 con 0x55636f023a70 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.422+0000 7f1829ffb700 1 -- 172.21.15.26:0/3245561664 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838004920 con 0x55636f023a70 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.423+0000 7f18489fd700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 0x7f182c0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.423+0000 7f18489fd700 1 -- 172.21.15.26:0/808221969 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1834014070 con 0x7f1844103160 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.423+0000 7f185d300b80 1 -- 172.21.15.26:0/3245561664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 msgr2=0x55636f025e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.423+0000 7f185d300b80 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1838005b40 tx=0x7f1838005c00 comp rx=0 tx=0).stop 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.423+0000 7f18437fe700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 0x7f182c0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.424+0000 7f185d300b80 1 -- 172.21.15.26:0/3245561664 shutdown_connections 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.424+0000 7f185d300b80 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f023a70 0x55636f025e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.424+0000 7f185d300b80 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f018680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.424+0000 7f185d300b80 1 --2- 172.21.15.26:0/3245561664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f018bc0 0x55636f023530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.424+0000 7f185d300b80 1 -- 172.21.15.26:0/3245561664 >> 172.21.15.26:0/3245561664 conn(0x55636ef8d260 msgr2=0x55636f014630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.425+0000 7f18437fe700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 0x7f182c0a3d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f183c007560 tx=0x7f183c00b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.425+0000 7f185d300b80 1 -- 172.21.15.26:0/3245561664 shutdown_connections 2023-10-06T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.425+0000 7f185d300b80 1 -- 172.21.15.26:0/3245561664 wait complete. 2023-10-06T21:13:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.426+0000 7f185d300b80 1 -- start start 2023-10-06T21:13:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.428+0000 7f185d300b80 1 Processor -- start 2023-10-06T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.428+0000 7f185d300b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.428+0000 7f185d300b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 0x55636f020d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.429+0000 7f185d300b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f021260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.429+0000 7f185d300b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef6be30 con 0x55636f018bc0 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.429+0000 7f185d300b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef08710 con 0x55636f0182a0 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.429+0000 7f185d300b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef083b0 con 0x55636f023a70 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.429+0000 7f185d300b80 10 client.0 _ll_register_callbacks cb 0x55636ed65800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:47782/0 (socket says 172.21.15.26:47782) 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 -- 172.21.15.26:0/1545508300 learned_addr learned my addr 172.21.15.26:0/1545508300 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f18427fc700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 0x55636f020d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 -- 172.21.15.26:0/1545508300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 msgr2=0x55636f021260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f021260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 -- 172.21.15.26:0/1545508300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 msgr2=0x55636f020d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 0x55636f020d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 -- 172.21.15.26:0/1545508300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1844110670 con 0x55636f0182a0 2023-10-06T21:13:50.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f18427fc700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 0x55636f020d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1842ffd700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1834096c30 tx=0x7f1834010b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568dc763921285 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.430+0000 7f1822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183409a620 con 0x55636f0182a0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f183409a8c0 con 0x55636f0182a0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55636f00c160 con 0x55636f0182a0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f183409e120 con 0x55636f0182a0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f185d300b80 10 client.25949 fetch_fsmap learned FSMap version 34 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.431+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55636ed7fc60 con 0x55636f0182a0 2023-10-06T21:13:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18340b1030 con 0x55636f0182a0 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f18340996b0 con 0x55636f0182a0 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 20 client.25949 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f03bd80 0x55636f03e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 4 client.25949 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55636ef64e10 con 0x55636f03bd80 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.432+0000 7f18427fc700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f03bd80 0x55636f03e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.433+0000 7f1822ffd700 10 client.25949 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.433+0000 7f18427fc700 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f03bd80 0x55636f03e1a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275ca73b4a5c33ba server_cookie=92d2631299857cfe in_seq=0 out_seq=0 2023-10-06T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.434+0000 7f1822ffd700 1 -- 172.21.15.26:0/1545508300 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55636ef64e10 con 0x55636f03bd80 2023-10-06T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.434+0000 7f1822ffd700 10 client.25949 handle_command_reply: tid=0 2023-10-06T21:13:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.434+0000 7f185d300b80 1 client.25949 shutdown 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 msgr2=0x55636f0207e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1834096c30 tx=0x7f1834010b90 comp rx=0 tx=0).stop 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 shutdown_connections 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f021260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f03bd80 0x55636f03e1a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f0182a0 0x55636f0207e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 --2- 172.21.15.26:0/1545508300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f018bc0 0x55636f020d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 >> 172.21.15.26:0/1545508300 conn(0x55636ef8d260 msgr2=0x55636f014220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f1822ffd700 0 client.25949 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 shutdown_connections 2023-10-06T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.437+0000 7f185d300b80 1 -- 172.21.15.26:0/1545508300 wait complete. 2023-10-06T21:13:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.438+0000 7f185d300b80 20 client.25949 trim_cache size 0 max 16384 2023-10-06T21:13:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 Processor -- start 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 -- start start 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f032db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f01dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636edec920 con 0x55636f0182a0 2023-10-06T21:13:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ef64e10 con 0x55636f018bc0 2023-10-06T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.466+0000 7f185d300b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636f018130 con 0x55636f023a70 2023-10-06T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f1842ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f032db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:47794/0 (socket says 172.21.15.26:47794) 2023-10-06T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18437fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f01dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 -- 172.21.15.26:0/1379797412 learned_addr learned my addr 172.21.15.26:0/1379797412 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 -- 172.21.15.26:0/1379797412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 msgr2=0x55636f01dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f01dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 -- 172.21.15.26:0/1379797412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 msgr2=0x55636f032db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f032db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.467+0000 7f18427fc700 1 -- 172.21.15.26:0/1379797412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55636f03e550 con 0x55636f018bc0 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.468+0000 7f18427fc700 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1838009990 tx=0x7f183800be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad21f12e6a463b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.468+0000 7f182a7fc700 1 -- 172.21.15.26:0/1379797412 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183801e070 con 0x55636f018bc0 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.468+0000 7f182a7fc700 1 -- 172.21.15.26:0/1379797412 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183802c390 con 0x55636f018bc0 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.468+0000 7f182a7fc700 1 -- 172.21.15.26:0/1379797412 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838028040 con 0x55636f018bc0 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 -- 172.21.15.26:0/1379797412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 msgr2=0x55636f01da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1838009990 tx=0x7f183800be00 comp rx=0 tx=0).stop 2023-10-06T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 -- 172.21.15.26:0/1379797412 shutdown_connections 2023-10-06T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f01dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f032db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 --2- 172.21.15.26:0/1379797412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f01da70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.469+0000 7f185d300b80 1 -- 172.21.15.26:0/1379797412 >> 172.21.15.26:0/1379797412 conn(0x55636ef8d260 msgr2=0x55636f01bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.470+0000 7f185d300b80 1 -- 172.21.15.26:0/1379797412 shutdown_connections 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.470+0000 7f185d300b80 1 -- 172.21.15.26:0/1379797412 wait complete. 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.471+0000 7f185d300b80 1 -- start start 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.471+0000 7f185d300b80 1 Processor -- start 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f0884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f083b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636f084240 con 0x55636f0182a0 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636f0843b0 con 0x55636f018bc0 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636f084570 con 0x55636f023a70 2023-10-06T21:13:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.472+0000 7f185d300b80 10 client.0 _ll_register_callbacks cb 0x55636ed65800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.473+0000 7f18427fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f0884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.473+0000 7f1842ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f1842ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60484/0 (socket says 172.21.15.26:60484) 2023-10-06T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f1842ffd700 1 -- 172.21.15.26:0/3321990491 learned_addr learned my addr 172.21.15.26:0/3321990491 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:50.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f18427fc700 1 -- 172.21.15.26:0/3321990491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 msgr2=0x55636f083b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f18427fc700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f083b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f18427fc700 1 -- 172.21.15.26:0/3321990491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 msgr2=0x55636f087fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f18427fc700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f18427fc700 1 -- 172.21.15.26:0/3321990491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55636ed96de0 con 0x55636f018bc0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.474+0000 7f1842ffd700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.475+0000 7f18427fc700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f0884f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55636f03ea80 tx=0x7f183802c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16779175adffc6cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.475+0000 7f1822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.475+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183801e070 con 0x55636f018bc0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.475+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f183800b7d0 con 0x55636f018bc0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f183800bc90 con 0x55636f018bc0 2023-10-06T21:13:50.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1838028040 con 0x55636f018bc0 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f185d300b80 10 client.25955 fetch_fsmap learned FSMap version 34 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55636edbd250 con 0x55636f018bc0 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183800bc90 con 0x55636f018bc0 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.476+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1838004760 con 0x55636f018bc0 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 10 client.25955 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 20 client.25955 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f090930 0x55636f092d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:50.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 4 client.25955 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55636ef64b30 con 0x55636f090930 2023-10-06T21:13:50.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.477+0000 7f1842ffd700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f090930 0x55636f092d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.479+0000 7f1842ffd700 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f090930 0x55636f092d20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1a9a7b86c14d7c server_cookie=d2933efdb7cf97e9 in_seq=0 out_seq=0 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.479+0000 7f1822ffd700 10 client.25955 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f1822ffd700 1 -- 172.21.15.26:0/3321990491 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55636ef64b30 con 0x55636f090930 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f1822ffd700 10 client.25955 handle_command_reply: tid=0 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 client.25955 shutdown 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 msgr2=0x55636f0884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f0884f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55636f03ea80 tx=0x7f183802c500 comp rx=0 tx=0).stop 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 shutdown_connections 2023-10-06T21:13:50.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55636f023a70 0x55636f083b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55636f0182a0 0x55636f087fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55636f090930 0x55636f092d20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f1822ffd700 0 client.25955 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 --2- 172.21.15.26:0/3321990491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55636f018bc0 0x55636f0884f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 >> 172.21.15.26:0/3321990491 conn(0x55636ef8d260 msgr2=0x55636f01a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 shutdown_connections 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.480+0000 7f185d300b80 1 -- 172.21.15.26:0/3321990491 wait complete. 2023-10-06T21:13:50.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.481+0000 7f185d300b80 20 client.25955 trim_cache size 0 max 16384 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "4b2d891c-26c5-42e9-a3d6-305bd09e41fa": { 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "4b2d891c-26c5-42e9-a3d6-305bd09e41fa", 2023-10-06T21:13:50.485 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:13:50.486 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:13:50.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 msgr2=0x7f182c0a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 0x7f182c0a3d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f183c007560 tx=0x7f183c00b040 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 msgr2=0x7f1844198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 0x7f1844198fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f183400e9f0 tx=0x7f183400ed00 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 shutdown_connections 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f182c0a1870 0x7f182c0a3d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184419d950 0x7f184419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1844103160 0x7f1844198fb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 --2- 172.21.15.26:0/808221969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1844108280 0x7f18441994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 >> 172.21.15.26:0/808221969 conn(0x7f18440fe9c0 msgr2=0x7f18440ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.484+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 shutdown_connections 2023-10-06T21:13:50.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:13:50.485+0000 7f1843fff700 1 -- 172.21.15.26:0/808221969 wait complete. 2023-10-06T21:13:50.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:50 smithi026 ceph-mon[98490]: pgmap v109: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-10-06T21:13:50.525 INFO:tasks.cephfs.filesystem:scrub status for tag:4b2d891c-26c5-42e9-a3d6-305bd09e41fa - {'path': '/', 'tag': '4b2d891c-26c5-42e9-a3d6-305bd09e41fa', 'options': 'recursive,force'} 2023-10-06T21:13:50.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:50 smithi167 ceph-mon[134494]: pgmap v109: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-10-06T21:13:50.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:50 smithi078 ceph-mon[134896]: pgmap v109: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.76k op/s 2023-10-06T21:13:51.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:51 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:51.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:51 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:51.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:51 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 190.0 s, 2328.2 tps, lat 43.025 ms stddev 44.164 2023-10-06T21:13:52.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:52 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:13:52.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:52 smithi167 ceph-mon[134494]: pgmap v110: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 46 MiB/s wr, 2.06k op/s 2023-10-06T21:13:52.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:52 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:13:52.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:52 smithi026 ceph-mon[98490]: pgmap v110: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 46 MiB/s wr, 2.06k op/s 2023-10-06T21:13:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:52 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:13:52.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:52 smithi078 ceph-mon[134896]: pgmap v110: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 46 MiB/s wr, 2.06k op/s 2023-10-06T21:13:54.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:54 smithi167 ceph-mon[134494]: pgmap v111: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.72k op/s 2023-10-06T21:13:54.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:54 smithi026 ceph-mon[98490]: pgmap v111: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.72k op/s 2023-10-06T21:13:54.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:54 smithi078 ceph-mon[134896]: pgmap v111: 97 pgs: 97 active+clean; 4.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 1.72k op/s 2023-10-06T21:13:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 -- 172.21.15.26:0/862097965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2734100980 msgr2=0x7f27341049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 --2- 172.21.15.26:0/862097965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2734100980 0x7f27341049c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2724009990 tx=0x7f2724009ca0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 -- 172.21.15.26:0/862097965 shutdown_connections 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 --2- 172.21.15.26:0/862097965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f273410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 --2- 172.21.15.26:0/862097965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27340fffd0 0x7f27341003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 --2- 172.21.15.26:0/862097965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2734100980 0x7f27341049c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 -- 172.21.15.26:0/862097965 >> 172.21.15.26:0/862097965 conn(0x7f2734076ea0 msgr2=0x7f27340772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.136+0000 7f273bbff700 1 -- 172.21.15.26:0/862097965 shutdown_connections 2023-10-06T21:13:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.137+0000 7f273bbff700 1 -- 172.21.15.26:0/862097965 wait complete. 2023-10-06T21:13:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.138+0000 7f273bbff700 1 Processor -- start 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 -- start start 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340fffd0 0x7f2734109910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 0x7f273419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273419b5c0 con 0x7f273419ab70 2023-10-06T21:13:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273419b700 con 0x7f27341050f0 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273bbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273419b870 con 0x7f27340fffd0 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60462/0 (socket says 172.21.15.26:60462) 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273919a700 1 -- 172.21.15.26:0/2830963757 learned_addr learned my addr 172.21.15.26:0/2830963757 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.140+0000 7f273a19c700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 0x7f273419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273999b700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340fffd0 0x7f2734109910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 -- 172.21.15.26:0/2830963757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340fffd0 msgr2=0x7f2734109910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340fffd0 0x7f2734109910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 -- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 msgr2=0x7f273419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 0x7f273419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 -- 172.21.15.26:0/2830963757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27240095f0 con 0x7f27341050f0 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273a19c700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 0x7f273419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:13:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.141+0000 7f273919a700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2724000c00 tx=0x7f2724005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.142+0000 7f272affd700 1 -- 172.21.15.26:0/2830963757 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272401d070 con 0x7f27341050f0 2023-10-06T21:13:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.142+0000 7f272affd700 1 -- 172.21.15.26:0/2830963757 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2724021410 con 0x7f27341050f0 2023-10-06T21:13:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.142+0000 7f272affd700 1 -- 172.21.15.26:0/2830963757 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724017650 con 0x7f27341050f0 2023-10-06T21:13:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.142+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273419dc40 con 0x7f27341050f0 2023-10-06T21:13:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.142+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273419e0a0 con 0x7f27341050f0 2023-10-06T21:13:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.147+0000 7f272affd700 1 -- 172.21.15.26:0/2830963757 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f272400fa90 con 0x7f27341050f0 2023-10-06T21:13:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.148+0000 7f274f574b80 1 Processor -- start 2023-10-06T21:13:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.149+0000 7f274f574b80 1 -- start start 2023-10-06T21:13:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e6a520 0x55d270e74e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270e777c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270dd3400 con 0x55d270e753d0 2023-10-06T21:13:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270dc4bc0 con 0x55d270e69c00 2023-10-06T21:13:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f274f574b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270dc24e0 con 0x55d270e6a520 2023-10-06T21:13:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f273a19c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f273a19c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60464/0 (socket says 172.21.15.26:60464) 2023-10-06T21:13:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f273a19c700 1 -- 172.21.15.26:0/2171599914 learned_addr learned my addr 172.21.15.26:0/2171599914 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.150+0000 7f273919a700 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270e777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f272affd700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 0x7f27200a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f272affd700 1 -- 172.21.15.26:0/2830963757 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f27240ce230 con 0x7f27341050f0 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273999b700 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e6a520 0x55d270e74e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273a19c700 1 -- 172.21.15.26:0/2171599914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e6a520 msgr2=0x55d270e74e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273a19c700 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e6a520 0x55d270e74e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273a19c700 1 -- 172.21.15.26:0/2171599914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 msgr2=0x55d270e777c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273a19c700 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270e777c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.151+0000 7f273a19c700 1 -- 172.21.15.26:0/2171599914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d270be8d80 con 0x55d270e69c00 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.152+0000 7f273a19c700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 0x7f27200a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.152+0000 7f273a19c700 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f272c010480 tx=0x7f272c010790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0496854a59ebba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.152+0000 7f271ffff700 1 -- 172.21.15.26:0/2171599914 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011640 con 0x55d270e69c00 2023-10-06T21:13:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.153+0000 7f271ffff700 1 -- 172.21.15.26:0/2171599914 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f272c015070 con 0x55d270e69c00 2023-10-06T21:13:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.153+0000 7f271ffff700 1 -- 172.21.15.26:0/2171599914 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c01bcc0 con 0x55d270e69c00 2023-10-06T21:13:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.153+0000 7f274f574b80 1 -- 172.21.15.26:0/2171599914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 msgr2=0x55d270e69fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.153+0000 7f274f574b80 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f272c010480 tx=0x7f272c010790 comp rx=0 tx=0).stop 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 -- 172.21.15.26:0/2171599914 shutdown_connections 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270e777c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e69fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 --2- 172.21.15.26:0/2171599914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e6a520 0x55d270e74e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 -- 172.21.15.26:0/2171599914 >> 172.21.15.26:0/2171599914 conn(0x55d270ddf200 msgr2=0x55d270e66080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f273a19c700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 0x7f27200a3c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f272c01beb0 tx=0x7f272c01be20 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 -- 172.21.15.26:0/2171599914 shutdown_connections 2023-10-06T21:13:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.154+0000 7f274f574b80 1 -- 172.21.15.26:0/2171599914 wait complete. 2023-10-06T21:13:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.155+0000 7f274f574b80 1 -- start start 2023-10-06T21:13:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.155+0000 7f274f574b80 1 Processor -- start 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e69c00 0x55d270e74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e72030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270dbddd0 con 0x55d270e69c00 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270d5a6b0 con 0x55d270e6a520 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270d5a350 con 0x55d270e753d0 2023-10-06T21:13:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.156+0000 7f274f574b80 10 client.0 _ll_register_callbacks cb 0x55d270bb77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:37954/0 (socket says 172.21.15.26:37954) 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 -- 172.21.15.26:0/3022452655 learned_addr learned my addr 172.21.15.26:0/3022452655 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273919a700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e72030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273999b700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e69c00 0x55d270e74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 -- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 msgr2=0x55d270e72030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e72030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 -- 172.21.15.26:0/3022452655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e69c00 msgr2=0x55d270e74e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e69c00 0x55d270e74e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.157+0000 7f273a19c700 1 -- 172.21.15.26:0/3022452655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c00bc90 con 0x55d270e753d0 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f273a19c700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f272c01baa0 tx=0x7f272c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=226917d939959652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f271d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:13:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c00bb30 con 0x55d270e753d0 2023-10-06T21:13:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f272c017070 con 0x55d270e753d0 2023-10-06T21:13:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d270e5dd50 con 0x55d270e753d0 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f272c01b5e0 con 0x55d270e753d0 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f274f574b80 10 client.25957 fetch_fsmap learned FSMap version 35 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.158+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d270bd1c00 con 0x55d270e753d0 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f272c03e020 con 0x55d270e753d0 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f272c028450 con 0x55d270e753d0 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 10 client.25957 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 20 client.25957 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270e84f30 0x55d270e87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 4 client.25957 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.159+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d270db6db0 con 0x55d270e84f30 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.160+0000 7f273999b700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270e84f30 0x55d270e87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.161+0000 7f273999b700 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270e84f30 0x55d270e87350 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5495c11f951e4ef7 server_cookie=776807a9bffff7de in_seq=0 out_seq=0 2023-10-06T21:13:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.161+0000 7f271d7fa700 10 client.25957 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.162+0000 7f271d7fa700 1 -- 172.21.15.26:0/3022452655 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d270db6db0 con 0x55d270e84f30 2023-10-06T21:13:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.162+0000 7f271d7fa700 10 client.25957 handle_command_reply: tid=0 2023-10-06T21:13:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.162+0000 7f274f574b80 1 client.25957 shutdown 2023-10-06T21:13:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.163+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 msgr2=0x55d270e72770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.163+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f272c01baa0 tx=0x7f272c017000 comp rx=0 tx=0).stop 2023-10-06T21:13:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 shutdown_connections 2023-10-06T21:13:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e72770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e69c00 0x55d270e74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270e84f30 0x55d270e87350 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 --2- 172.21.15.26:0/3022452655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e72030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.164+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 >> 172.21.15.26:0/3022452655 conn(0x55d270ddf200 msgr2=0x55d270e65c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.165+0000 7f271d7fa700 0 client.25957 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.166+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 shutdown_connections 2023-10-06T21:13:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.166+0000 7f274f574b80 1 -- 172.21.15.26:0/3022452655 wait complete. 2023-10-06T21:13:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.167+0000 7f274f574b80 20 client.25957 trim_cache size 0 max 16384 2023-10-06T21:13:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.186+0000 7f274f574b80 1 Processor -- start 2023-10-06T21:13:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.186+0000 7f274f574b80 1 -- start start 2023-10-06T21:13:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e6a520 0x55d270e6f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270c3e8c0 con 0x55d270e6a520 2023-10-06T21:13:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270e81870 con 0x55d270e69c00 2023-10-06T21:13:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f274f574b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270e72d70 con 0x55d270e753d0 2023-10-06T21:13:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f273999b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f273a19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f273a19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:37966/0 (socket says 172.21.15.26:37966) 2023-10-06T21:13:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.187+0000 7f273a19c700 1 -- 172.21.15.26:0/3579590405 learned_addr learned my addr 172.21.15.26:0/3579590405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:56.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273919a700 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e6a520 0x55d270e6f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 -- 172.21.15.26:0/3579590405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 msgr2=0x55d270e74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 -- 172.21.15.26:0/3579590405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e6a520 msgr2=0x55d270e6f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e6a520 0x55d270e6f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 -- 172.21.15.26:0/3579590405 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d270c0f1f0 con 0x55d270e753d0 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.188+0000 7f273a19c700 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f272c005c40 tx=0x7f272c010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0f1b4115528213 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.189+0000 7f2728bff700 1 -- 172.21.15.26:0/3579590405 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c005ee0 con 0x55d270e753d0 2023-10-06T21:13:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.189+0000 7f2728bff700 1 -- 172.21.15.26:0/3579590405 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f272c03e030 con 0x55d270e753d0 2023-10-06T21:13:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.189+0000 7f2728bff700 1 -- 172.21.15.26:0/3579590405 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c004910 con 0x55d270e753d0 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 -- 172.21.15.26:0/3579590405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 msgr2=0x55d270e6f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f272c005c40 tx=0x7f272c010790 comp rx=0 tx=0).stop 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 -- 172.21.15.26:0/3579590405 shutdown_connections 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e753d0 0x55d270e6f800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e69c00 0x55d270e74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 --2- 172.21.15.26:0/3579590405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e6a520 0x55d270e6f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 -- 172.21.15.26:0/3579590405 >> 172.21.15.26:0/3579590405 conn(0x55d270ddf200 msgr2=0x55d270e647b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 -- 172.21.15.26:0/3579590405 shutdown_connections 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.190+0000 7f274f574b80 1 -- 172.21.15.26:0/3579590405 wait complete. 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.191+0000 7f274f574b80 1 -- start start 2023-10-06T21:13:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.191+0000 7f274f574b80 1 Processor -- start 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e69c00 0x55d270e78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270ed7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270ed8290 con 0x55d270e753d0 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270ed83d0 con 0x55d270e6a520 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d270ed8510 con 0x55d270e69c00 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.192+0000 7f274f574b80 10 client.0 _ll_register_callbacks cb 0x7f27240d1b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.193+0000 7f273919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.193+0000 7f273919a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60478/0 (socket says 172.21.15.26:60478) 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.193+0000 7f273919a700 1 -- 172.21.15.26:0/3889706205 learned_addr learned my addr 172.21.15.26:0/3889706205 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:13:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.193+0000 7f273999b700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e69c00 0x55d270e78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 -- 172.21.15.26:0/3889706205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e69c00 msgr2=0x55d270e78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e69c00 0x55d270e78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 -- 172.21.15.26:0/3889706205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 msgr2=0x55d270ed7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270ed7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 -- 172.21.15.26:0/3889706205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d270bd1c00 con 0x55d270e6a520 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f273919a700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f272400c010 tx=0x7f27240cbe90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7574ae4aaa07f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:13:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f271d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.194+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272409f230 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f27240d1e30 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f272c015070 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2724097620 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f274f574b80 10 client.25973 fetch_fsmap learned FSMap version 35 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d270be7250 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f272c015070 con 0x55d270e6a520 2023-10-06T21:13:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.195+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f272409a7d0 con 0x55d270e6a520 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 20 client.25973 populate_metadata read hostname 'smithi026' 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270ed99d0 0x55d270edbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 4 client.25973 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d270db6ad0 con 0x55d270ed99d0 2023-10-06T21:13:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.196+0000 7f273999b700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270ed99d0 0x55d270edbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:13:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.197+0000 7f271d7fa700 10 client.25973 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.197+0000 7f273999b700 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270ed99d0 0x55d270edbdf0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec3b302892ca914 server_cookie=c9f25a296434ac22 in_seq=0 out_seq=0 2023-10-06T21:13:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.244+0000 7f271d7fa700 1 -- 172.21.15.26:0/3889706205 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d270db6ad0 con 0x55d270ed99d0 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.244+0000 7f271d7fa700 10 client.25973 handle_command_reply: tid=0 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.244+0000 7f274f574b80 1 client.25973 shutdown 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 msgr2=0x55d270e78680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f272400c010 tx=0x7f27240cbe90 comp rx=0 tx=0).stop 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 shutdown_connections 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d270e753d0 0x55d270ed7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d270e69c00 0x55d270e78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d270ed99d0 0x55d270edbdf0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 --2- 172.21.15.26:0/3889706205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d270e6a520 0x55d270e78680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f271d7fa700 0 client.25973 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 >> 172.21.15.26:0/3889706205 conn(0x55d270ddf200 msgr2=0x55d270e6ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 shutdown_connections 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.245+0000 7f274f574b80 1 -- 172.21.15.26:0/3889706205 wait complete. 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.246+0000 7f274f574b80 20 client.25973 trim_cache size 0 max 16384 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:13:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:13:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.247+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 msgr2=0x7f27200a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.247+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 0x7f27200a3c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f272c01beb0 tx=0x7f272c01be20 comp rx=0 tx=0).stop 2023-10-06T21:13:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.247+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 msgr2=0x7f2734109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:13:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.247+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2724000c00 tx=0x7f2724005740 comp rx=0 tx=0).stop 2023-10-06T21:13:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 shutdown_connections 2023-10-06T21:13:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27200a1780 0x7f27200a3c40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273419ab70 0x7f273419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27340fffd0 0x7f2734109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 --2- 172.21.15.26:0/2830963757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27341050f0 0x7f2734109e50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:13:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 >> 172.21.15.26:0/2830963757 conn(0x7f2734076ea0 msgr2=0x7f273410b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:13:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 shutdown_connections 2023-10-06T21:13:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:13:56.248+0000 7f273bbff700 1 -- 172.21.15.26:0/2830963757 wait complete. 2023-10-06T21:13:56.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:56 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:56.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:56 smithi026 ceph-mon[98490]: pgmap v112: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-06T21:13:56.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:56 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:56.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:56 smithi078 ceph-mon[134896]: pgmap v112: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-06T21:13:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:56 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:13:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:56 smithi167 ceph-mon[134494]: pgmap v112: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 43 MiB/s wr, 1.97k op/s 2023-10-06T21:13:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 195.0 s, 2527.3 tps, lat 39.747 ms stddev 40.988 2023-10-06T21:13:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:13:58 smithi167 ceph-mon[134494]: pgmap v113: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 20 MiB/s wr, 840 op/s 2023-10-06T21:13:58.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:13:58 smithi026 ceph-mon[98490]: pgmap v113: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 20 MiB/s wr, 840 op/s 2023-10-06T21:13:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:13:58 smithi078 ceph-mon[134896]: pgmap v113: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 20 MiB/s wr, 840 op/s 2023-10-06T21:14:00.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:00 smithi167 ceph-mon[134494]: pgmap v114: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 598 op/s 2023-10-06T21:14:00.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:00 smithi026 ceph-mon[98490]: pgmap v114: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 598 op/s 2023-10-06T21:14:00.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:00 smithi078 ceph-mon[134896]: pgmap v114: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 15 MiB/s wr, 598 op/s 2023-10-06T21:14:01.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:01 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:14:01.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:01 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:14:01.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:01 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:14:01.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:01 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:14:01.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:01 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:14:01.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:01 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:14:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:01 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:14:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:01 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:14:01.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:01 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:14:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 200.0 s, 1891.4 tps, lat 51.827 ms stddev 66.894 2023-10-06T21:14:02.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:02 smithi167 ceph-mon[134494]: pgmap v115: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 21 MiB/s wr, 893 op/s 2023-10-06T21:14:02.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:02 smithi026 ceph-mon[98490]: pgmap v115: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 21 MiB/s wr, 893 op/s 2023-10-06T21:14:02.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:02 smithi078 ceph-mon[134896]: pgmap v115: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 21 MiB/s wr, 893 op/s 2023-10-06T21:14:04.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:04 smithi078 ceph-mon[134896]: pgmap v116: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 597 op/s 2023-10-06T21:14:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:04 smithi167 ceph-mon[134494]: pgmap v116: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 597 op/s 2023-10-06T21:14:04.578 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:04 smithi026 ceph-mon[98490]: pgmap v116: 97 pgs: 97 active+clean; 4.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 597 op/s 2023-10-06T21:14:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:06 smithi167 ceph-mon[134494]: pgmap v117: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 1.76k op/s 2023-10-06T21:14:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:06 smithi026 ceph-mon[98490]: pgmap v117: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 1.76k op/s 2023-10-06T21:14:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:06 smithi078 ceph-mon[134896]: pgmap v117: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 1.76k op/s 2023-10-06T21:14:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 205.0 s, 1174.3 tps, lat 85.200 ms stddev 136.117 2023-10-06T21:14:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:08 smithi167 ceph-mon[134494]: pgmap v118: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-06T21:14:08.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:08 smithi026 ceph-mon[98490]: pgmap v118: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-06T21:14:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:08 smithi078 ceph-mon[134896]: pgmap v118: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 MiB/s wr, 1.74k op/s 2023-10-06T21:14:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3419517327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7810b670 msgr2=0x7fed78107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3419517327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7810b670 0x7fed78107eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed6800b0a0 tx=0x7fed6800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3419517327 shutdown_connections 2023-10-06T21:14:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3419517327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed78115fe0 0x7fed781183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3419517327 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7810b670 0x7fed78107eb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3419517327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed78108480 0x7fed78108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3419517327 >> 172.21.15.26:0/3419517327 conn(0x7fed7806d3e0 msgr2=0x7fed7806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3419517327 shutdown_connections 2023-10-06T21:14:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.496+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3419517327 wait complete. 2023-10-06T21:14:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.497+0000 7fed7e9ff700 1 Processor -- start 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 -- start start 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 0x7fed7810ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed78115fe0 0x7fed7810f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 0x7fed781a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a8450 con 0x7fed78115fe0 2023-10-06T21:14:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a85c0 con 0x7fed781a79a0 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.498+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a8780 con 0x7fed78108480 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.499+0000 7fed77fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 0x7fed7810ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.499+0000 7fed77fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 0x7fed7810ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38840/0 (socket says 172.21.15.26:38840) 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.499+0000 7fed77fff700 1 -- 172.21.15.26:0/3430233041 learned_addr learned my addr 172.21.15.26:0/3430233041 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.499+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 0x7fed781a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.500+0000 7fed7cf9c700 1 -- 172.21.15.26:0/3430233041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 msgr2=0x7fed7810ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.500+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 0x7fed7810ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.500+0000 7fed7cf9c700 1 -- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed78115fe0 msgr2=0x7fed7810f370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:14:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.500+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed78115fe0 0x7fed7810f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.500+0000 7fed7cf9c700 1 -- 172.21.15.26:0/3430233041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68009d00 con 0x7fed781a79a0 2023-10-06T21:14:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.501+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 0x7fed781a7e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e940 tx=0x7fed7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.501+0000 7fed75ffb700 1 -- 172.21.15.26:0/3430233041 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000eee0 con 0x7fed781a79a0 2023-10-06T21:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.502+0000 7fed75ffb700 1 -- 172.21.15.26:0/3430233041 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed700044e0 con 0x7fed781a79a0 2023-10-06T21:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.502+0000 7fed75ffb700 1 -- 172.21.15.26:0/3430233041 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70005020 con 0x7fed781a79a0 2023-10-06T21:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.502+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed78114e00 con 0x7fed781a79a0 2023-10-06T21:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.502+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781152c0 con 0x7fed781a79a0 2023-10-06T21:14:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.508+0000 7fed75ffb700 1 -- 172.21.15.26:0/3430233041 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fed70005180 con 0x7fed781a79a0 2023-10-06T21:14:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.510+0000 7fed75ffb700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 0x7fed5c0a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.510+0000 7fed75ffb700 1 -- 172.21.15.26:0/3430233041 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fed70014070 con 0x7fed781a79a0 2023-10-06T21:14:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.510+0000 7fed923a4b80 1 Processor -- start 2023-10-06T21:14:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.510+0000 7fed77fff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 0x7fed5c0a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.511+0000 7fed77fff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 0x7fed5c0a3bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed680062a0 tx=0x7fed68006210 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 -- start start 2023-10-06T21:14:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50b73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b7f3d0 0x55ea50b817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50add400 con 0x55ea50b73c00 2023-10-06T21:14:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50acebc0 con 0x55ea50b7f3d0 2023-10-06T21:14:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.513+0000 7fed923a4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50acc4e0 con 0x55ea50b74520 2023-10-06T21:14:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.516+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.516+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38848/0 (socket says 172.21.15.26:38848) 2023-10-06T21:14:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.516+0000 7fed6ffff700 1 -- 172.21.15.26:0/172897303 learned_addr learned my addr 172.21.15.26:0/172897303 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.518+0000 7fed6ffff700 1 -- 172.21.15.26:0/172897303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b7f3d0 msgr2=0x55ea50b817c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:14:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.518+0000 7fed6ffff700 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b7f3d0 0x55ea50b817c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.518+0000 7fed6ffff700 1 -- 172.21.15.26:0/172897303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 msgr2=0x55ea50b73fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.518+0000 7fed6ffff700 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50b73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.518+0000 7fed6ffff700 1 -- 172.21.15.26:0/172897303 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea508f2d80 con 0x55ea50b74520 2023-10-06T21:14:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.520+0000 7fed6ffff700 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed64009c20 tx=0x7fed64009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab2deb71fad591e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.520+0000 7fed6e7fc700 1 -- 172.21.15.26:0/172897303 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400ebb0 con 0x55ea50b74520 2023-10-06T21:14:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.522+0000 7fed6e7fc700 1 -- 172.21.15.26:0/172897303 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed64004cf0 con 0x55ea50b74520 2023-10-06T21:14:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.522+0000 7fed6e7fc700 1 -- 172.21.15.26:0/172897303 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64005630 con 0x55ea50b74520 2023-10-06T21:14:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.523+0000 7fed923a4b80 1 -- 172.21.15.26:0/172897303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 msgr2=0x55ea50b7ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.523+0000 7fed923a4b80 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed64009c20 tx=0x7fed64009f30 comp rx=0 tx=0).stop 2023-10-06T21:14:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.524+0000 7fed923a4b80 1 -- 172.21.15.26:0/172897303 shutdown_connections 2023-10-06T21:14:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.524+0000 7fed923a4b80 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b7f3d0 0x55ea50b817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.524+0000 7fed923a4b80 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50b73fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.524+0000 7fed923a4b80 1 --2- 172.21.15.26:0/172897303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7ee90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.524+0000 7fed923a4b80 1 -- 172.21.15.26:0/172897303 >> 172.21.15.26:0/172897303 conn(0x55ea50ae9200 msgr2=0x55ea50b70080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.543+0000 7fed923a4b80 1 -- 172.21.15.26:0/172897303 shutdown_connections 2023-10-06T21:14:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.551+0000 7fed923a4b80 1 -- 172.21.15.26:0/172897303 wait complete. 2023-10-06T21:14:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.552+0000 7fed923a4b80 1 -- start start 2023-10-06T21:14:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.553+0000 7fed923a4b80 1 Processor -- start 2023-10-06T21:14:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b7c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50ac7dd0 con 0x55ea50b7f3d0 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50a646b0 con 0x55ea50b73c00 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50a64350 con 0x55ea50b74520 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.558+0000 7fed923a4b80 10 client.0 _ll_register_callbacks cb 0x55ea508c17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.559+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.560+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38856/0 (socket says 172.21.15.26:38856) 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.560+0000 7fed6ffff700 1 -- 172.21.15.26:0/1300365984 learned_addr learned my addr 172.21.15.26:0/1300365984 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.560+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 -- 172.21.15.26:0/1300365984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 msgr2=0x55ea50b7c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 -- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 msgr2=0x55ea50b7c250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b7c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 -- 172.21.15.26:0/1300365984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64009900 con 0x55ea50b7f3d0 2023-10-06T21:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed6ffff700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed7cf9c700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b7ccd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed700ca860 tx=0x7fed700cadf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee678387623e2259 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.561+0000 7fed5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.571+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70099810 con 0x55ea50b7f3d0 2023-10-06T21:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.571+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fed7009e270 con 0x55ea50b7f3d0 2023-10-06T21:14:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.571+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea50b67d50 con 0x55ea50b7f3d0 2023-10-06T21:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.571+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed700a24b0 con 0x55ea50b7f3d0 2023-10-06T21:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.571+0000 7fed923a4b80 10 client.15840 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.572+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ea508dbc00 con 0x55ea50b7f3d0 2023-10-06T21:14:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea50b67d50 con 0x55ea50b7f3d0 2023-10-06T21:14:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fed700a2020 con 0x55ea50b7f3d0 2023-10-06T21:14:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 20 client.15840 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50b8efe0 0x55ea50b91400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 4 client.15840 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.573+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea50ac0db0 con 0x55ea50b8efe0 2023-10-06T21:14:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.574+0000 7fed77fff700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50b8efe0 0x55ea50b91400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.575+0000 7fed77fff700 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50b8efe0 0x55ea50b91400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862f2134e7a00d81 server_cookie=c2dabc39418eebd6 in_seq=0 out_seq=0 2023-10-06T21:14:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.575+0000 7fed5b7fe700 10 client.15840 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.576+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1300365984 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea50ac0db0 con 0x55ea50b8efe0 2023-10-06T21:14:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.576+0000 7fed5b7fe700 10 client.15840 handle_command_reply: tid=0 2023-10-06T21:14:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.576+0000 7fed923a4b80 1 client.15840 shutdown 2023-10-06T21:14:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.583+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 msgr2=0x55ea50b7ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.583+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b7ccd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed700ca860 tx=0x7fed700cadf0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 shutdown_connections 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b7ccd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50b8efe0 0x55ea50b91400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b7c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1300365984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b7c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed5b7fe700 0 client.15840 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 >> 172.21.15.26:0/1300365984 conn(0x55ea50ae9200 msgr2=0x55ea50b6fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 shutdown_connections 2023-10-06T21:14:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.585+0000 7fed923a4b80 1 -- 172.21.15.26:0/1300365984 wait complete. 2023-10-06T21:14:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.586+0000 7fed923a4b80 20 client.15840 trim_cache size 0 max 16384 2023-10-06T21:14:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 Processor -- start 2023-10-06T21:14:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 -- start start 2023-10-06T21:14:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea509488c0 con 0x55ea50b7f3d0 2023-10-06T21:14:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50b74e90 con 0x55ea50b73c00 2023-10-06T21:14:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.615+0000 7fed923a4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50b864e0 con 0x55ea50b74520 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed7cf9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37762/0 (socket says 172.21.15.26:37762) 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 -- 172.21.15.26:0/1622913954 learned_addr learned my addr 172.21.15.26:0/1622913954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 -- 172.21.15.26:0/1622913954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 msgr2=0x55ea50b85f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 -- 172.21.15.26:0/1622913954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 msgr2=0x55ea50b77570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b77570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 -- 172.21.15.26:0/1622913954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea509191f0 con 0x55ea50b73c00 2023-10-06T21:14:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.616+0000 7fed77fff700 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed68009fd0 tx=0x7fed6800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989037a7e62d7699 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.617+0000 7fed6effd700 1 -- 172.21.15.26:0/1622913954 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6802aa40 con 0x55ea50b73c00 2023-10-06T21:14:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.617+0000 7fed6effd700 1 -- 172.21.15.26:0/1622913954 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6800f040 con 0x55ea50b73c00 2023-10-06T21:14:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.617+0000 7fed6effd700 1 -- 172.21.15.26:0/1622913954 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6802a3c0 con 0x55ea50b73c00 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- 172.21.15.26:0/1622913954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 msgr2=0x55ea50b76d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed68009fd0 tx=0x7fed6800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- 172.21.15.26:0/1622913954 shutdown_connections 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b7f3d0 0x55ea50b77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b73c00 0x55ea50b76d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1622913954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b74520 0x55ea50b85f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- 172.21.15.26:0/1622913954 >> 172.21.15.26:0/1622913954 conn(0x55ea50ae9200 msgr2=0x55ea50b8b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- 172.21.15.26:0/1622913954 shutdown_connections 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- 172.21.15.26:0/1622913954 wait complete. 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.618+0000 7fed923a4b80 1 -- start start 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 Processor -- start 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50bd6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b7f3d0 0x55ea50bd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50be1f90 con 0x55ea50b73c00 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50be2100 con 0x55ea50b74520 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea50be22c0 con 0x55ea50b7f3d0 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.619+0000 7fed923a4b80 10 client.0 _ll_register_callbacks cb 0x7fed70099a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37766/0 (socket says 172.21.15.26:37766) 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 -- 172.21.15.26:0/1642193221 learned_addr learned my addr 172.21.15.26:0/1642193221 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed77fff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50bd6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 -- 172.21.15.26:0/1642193221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b7f3d0 msgr2=0x55ea50bd7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b7f3d0 0x55ea50bd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 -- 172.21.15.26:0/1642193221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 msgr2=0x55ea50bd6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50bd6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 -- 172.21.15.26:0/1642193221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed78118fc0 con 0x55ea50b74520 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed6ffff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed640058e0 tx=0x7fed64005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e0465c18d6b5ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.620+0000 7fed5b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400edc0 con 0x55ea50b74520 2023-10-06T21:14:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed64004cf0 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fed64014070 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed64005b00 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed923a4b80 10 client.25985 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ea508f1250 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.621+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed64004cf0 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fed64018430 con 0x55ea50b74520 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 20 client.25985 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50be3a70 0x55ea50be5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 4 client.25985 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:14:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea50ac0ad0 con 0x55ea50be3a70 2023-10-06T21:14:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.622+0000 7fed77fff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50be3a70 0x55ea50be5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.623+0000 7fed77fff700 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50be3a70 0x55ea50be5e90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718040acf02cf5ad server_cookie=4570e194d6118040 in_seq=0 out_seq=0 2023-10-06T21:14:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.623+0000 7fed5b7fe700 10 client.25985 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.649+0000 7fed5b7fe700 1 -- 172.21.15.26:0/1642193221 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea50ac0ad0 con 0x55ea50be3a70 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.649+0000 7fed5b7fe700 10 client.25985 handle_command_reply: tid=0 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.649+0000 7fed923a4b80 1 client.25985 shutdown 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 msgr2=0x55ea50bd72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed640058e0 tx=0x7fed64005710 comp rx=0 tx=0).stop 2023-10-06T21:14:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 shutdown_connections 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea50b7f3d0 0x55ea50bd7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ea50b73c00 0x55ea50bd6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ea50be3a70 0x55ea50be5e90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 --2- 172.21.15.26:0/1642193221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea50b74520 0x55ea50bd72a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 >> 172.21.15.26:0/1642193221 conn(0x55ea50ae9200 msgr2=0x55ea50b77570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed5b7fe700 0 client.25985 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 shutdown_connections 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 1 -- 172.21.15.26:0/1642193221 wait complete. 2023-10-06T21:14:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.650+0000 7fed923a4b80 20 client.25985 trim_cache size 0 max 16384 2023-10-06T21:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:14:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 msgr2=0x7fed5c0a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 0x7fed5c0a3bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed680062a0 tx=0x7fed68006210 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 msgr2=0x7fed781a7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 0x7fed781a7e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e940 tx=0x7fed7000ed00 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 shutdown_connections 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed781a79a0 0x7fed781a7e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed78108480 0x7fed7810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fed5c0a16f0 0x7fed5c0a3bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 --2- 172.21.15.26:0/3430233041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed78115fe0 0x7fed7810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 >> 172.21.15.26:0/3430233041 conn(0x7fed7806d3e0 msgr2=0x7fed78071890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 shutdown_connections 2023-10-06T21:14:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:09.651+0000 7fed7e9ff700 1 -- 172.21.15.26:0/3430233041 wait complete. 2023-10-06T21:14:10.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:10 smithi167 ceph-mon[134494]: pgmap v119: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2023-10-06T21:14:10.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:10 smithi026 ceph-mon[98490]: pgmap v119: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2023-10-06T21:14:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:10 smithi078 ceph-mon[134896]: pgmap v119: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2023-10-06T21:14:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 210.0 s, 1482.1 tps, lat 68.552 ms stddev 79.913 2023-10-06T21:14:12.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:12 smithi167 ceph-mon[134494]: pgmap v120: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 2.30k op/s 2023-10-06T21:14:12.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:12 smithi026 ceph-mon[98490]: pgmap v120: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 2.30k op/s 2023-10-06T21:14:12.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:12 smithi078 ceph-mon[134896]: pgmap v120: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 38 MiB/s wr, 2.30k op/s 2023-10-06T21:14:14.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:13 smithi167 ceph-mon[134494]: pgmap v121: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-06T21:14:14.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:13 smithi026 ceph-mon[98490]: pgmap v121: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-06T21:14:14.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:13 smithi078 ceph-mon[134896]: pgmap v121: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2023-10-06T21:14:16.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:16 smithi167 ceph-mon[134494]: pgmap v122: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 40 MiB/s wr, 2.35k op/s 2023-10-06T21:14:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:16 smithi026 ceph-mon[98490]: pgmap v122: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 40 MiB/s wr, 2.35k op/s 2023-10-06T21:14:16.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:16 smithi078 ceph-mon[134896]: pgmap v122: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 40 MiB/s wr, 2.35k op/s 2023-10-06T21:14:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 215.0 s, 1867.8 tps, lat 53.135 ms stddev 55.750 2023-10-06T21:14:18.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:18 smithi167 ceph-mon[134494]: pgmap v123: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.25k op/s 2023-10-06T21:14:18.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:18 smithi026 ceph-mon[98490]: pgmap v123: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.25k op/s 2023-10-06T21:14:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:18 smithi078 ceph-mon[134896]: pgmap v123: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 1.25k op/s 2023-10-06T21:14:20.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:14:20.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:20 smithi167 ceph-mon[134494]: pgmap v124: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-06T21:14:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:20 smithi026 ceph-mon[98490]: pgmap v124: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-06T21:14:20.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:20 smithi078 ceph-mon[134896]: pgmap v124: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-06T21:14:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.916+0000 7f287b3ff700 1 -- 172.21.15.26:0/713978575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f287410d890 msgr2=0x7f287410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.916+0000 7f287b3ff700 1 --2- 172.21.15.26:0/713978575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f287410d890 0x7f287410dc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28700074e0 tx=0x7f287000af90 comp rx=0 tx=0).stop 2023-10-06T21:14:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.917+0000 7f287b3ff700 1 -- 172.21.15.26:0/713978575 shutdown_connections 2023-10-06T21:14:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.917+0000 7f287b3ff700 1 --2- 172.21.15.26:0/713978575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874108870 0x7f2874114750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.917+0000 7f287b3ff700 1 --2- 172.21.15.26:0/713978575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f287410d890 0x7f287410dc70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.917+0000 7f287b3ff700 1 --2- 172.21.15.26:0/713978575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2874107eb0 0x7f2874108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.917+0000 7f287b3ff700 1 -- 172.21.15.26:0/713978575 >> 172.21.15.26:0/713978575 conn(0x7f287406d400 msgr2=0x7f287406d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.918+0000 7f287b3ff700 1 -- 172.21.15.26:0/713978575 shutdown_connections 2023-10-06T21:14:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.918+0000 7f287b3ff700 1 -- 172.21.15.26:0/713978575 wait complete. 2023-10-06T21:14:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.920+0000 7f287b3ff700 1 Processor -- start 2023-10-06T21:14:20.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.920+0000 7f287b3ff700 1 -- start start 2023-10-06T21:14:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 0x7f28741abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 0x7f28741ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287410d890 0x7f28741135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2874113d70 con 0x7f287410d890 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2874113ee0 con 0x7f2874107eb0 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.921+0000 7f287b3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28741140a0 con 0x7f2874108870 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.922+0000 7f287919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 0x7f28741abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.922+0000 7f287919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 0x7f28741abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38742/0 (socket says 172.21.15.26:38742) 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.922+0000 7f287919b700 1 -- 172.21.15.26:0/1341642956 learned_addr learned my addr 172.21.15.26:0/1341642956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.922+0000 7f287899a700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 0x7f28741ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.922+0000 7f287999c700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287410d890 0x7f28741135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 -- 172.21.15.26:0/1341642956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 msgr2=0x7f28741abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 0x7f28741abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 -- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287410d890 msgr2=0x7f28741135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287410d890 0x7f28741135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 -- 172.21.15.26:0/1341642956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700071c0 con 0x7f2874108870 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287899a700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 0x7f28741ac240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2864006fd0 tx=0x7f286400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f286a7fc700 1 -- 172.21.15.26:0/1341642956 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864011830 con 0x7f2874108870 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f286a7fc700 1 -- 172.21.15.26:0/1341642956 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2864011e50 con 0x7f2874108870 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f286a7fc700 1 -- 172.21.15.26:0/1341642956 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864010e40 con 0x7f2874108870 2023-10-06T21:14:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2874114380 con 0x7f2874108870 2023-10-06T21:14:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.924+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28741b2ca0 con 0x7f2874108870 2023-10-06T21:14:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.930+0000 7f286a7fc700 1 -- 172.21.15.26:0/1341642956 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f2864011990 con 0x7f2874108870 2023-10-06T21:14:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.932+0000 7f286a7fc700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 0x7f28600a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.932+0000 7f286a7fc700 1 -- 172.21.15.26:0/1341642956 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f28640cbd40 con 0x7f2874108870 2023-10-06T21:14:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.933+0000 7f287919b700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 0x7f28600a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.935+0000 7f287919b700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 0x7f28600a3b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2870005b40 tx=0x7f2870005ab0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:20.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 Processor -- start 2023-10-06T21:14:20.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 -- start start 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c2038864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c203891890 0x55c203893c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2037ef460 con 0x55c2038860c0 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2037e0c20 con 0x55c2038869e0 2023-10-06T21:14:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.936+0000 7f288ed67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2037de540 con 0x55c203891890 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38752/0 (socket says 172.21.15.26:38752) 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 -- 172.21.15.26:0/600254175 learned_addr learned my addr 172.21.15.26:0/600254175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287999c700 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c203891890 0x55c203893c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 -- 172.21.15.26:0/600254175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c203891890 msgr2=0x55c203893c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c203891890 0x55c203893c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 -- 172.21.15.26:0/600254175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 msgr2=0x55c2038864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c2038864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 -- 172.21.15.26:0/600254175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c203604de0 con 0x55c2038869e0 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.937+0000 7f287899a700 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2864095340 tx=0x7f2864098a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605e45b798eeb8a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.938+0000 7f285f7fe700 1 -- 172.21.15.26:0/600254175 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864098e40 con 0x55c2038869e0 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.939+0000 7f285f7fe700 1 -- 172.21.15.26:0/600254175 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286409cf90 con 0x55c2038869e0 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.939+0000 7f285f7fe700 1 -- 172.21.15.26:0/600254175 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864098330 con 0x55c2038869e0 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 -- 172.21.15.26:0/600254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 msgr2=0x55c203891350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2864095340 tx=0x7f2864098a00 comp rx=0 tx=0).stop 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 -- 172.21.15.26:0/600254175 shutdown_connections 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c203891890 0x55c203893c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c2038864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 --2- 172.21.15.26:0/600254175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2038869e0 0x55c203891350 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2864095340 tx=0x7f2864098a00 comp rx=0 tx=0).stop 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 -- 172.21.15.26:0/600254175 >> 172.21.15.26:0/600254175 conn(0x55c2037fb260 msgr2=0x55c203882450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.940+0000 7f288ed67b80 1 -- 172.21.15.26:0/600254175 shutdown_connections 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.941+0000 7f288ed67b80 1 -- 172.21.15.26:0/600254175 wait complete. 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.941+0000 7f288ed67b80 1 -- start start 2023-10-06T21:14:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.942+0000 7f288ed67b80 1 Processor -- start 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c20388f360 0x55c20389d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2037d9e30 con 0x55c2038860c0 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c203776710 con 0x55c203891890 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2037763b0 con 0x55c20388f360 2023-10-06T21:14:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.943+0000 7f288ed67b80 10 client.0 _ll_register_callbacks cb 0x55c2035d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.944+0000 7f287919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.944+0000 7f287919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58768/0 (socket says 172.21.15.26:58768) 2023-10-06T21:14:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.944+0000 7f287919b700 1 -- 172.21.15.26:0/1597160120 learned_addr learned my addr 172.21.15.26:0/1597160120 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.944+0000 7f287999c700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c20388f360 0x55c20389d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287899a700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 -- 172.21.15.26:0/1597160120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c20388f360 msgr2=0x55c20389d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c20388f360 0x55c20389d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 -- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 msgr2=0x55c20388ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 -- 172.21.15.26:0/1597160120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28741b2ca0 con 0x55c2038860c0 2023-10-06T21:14:20.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287899a700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:20.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.945+0000 7f287919b700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28700044a0 tx=0x7f2870007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c7a4d84d12d129 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:20.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.946+0000 7f285cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.946+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870023070 con 0x55c2038860c0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.946+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f28700038e0 con 0x55c2038860c0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.947+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c203879f10 con 0x55c2038860c0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.947+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f287000f040 con 0x55c2038860c0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.948+0000 7f288ed67b80 10 client.15852 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.948+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c2035edc60 con 0x55c2038860c0 2023-10-06T21:14:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c203879f10 con 0x55c2038860c0 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f287000a4e0 con 0x55c2038860c0 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 20 client.15852 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2038b2a30 0x55c2038b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 4 client.15852 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:20.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.949+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2037d2e10 con 0x55c2038b2a30 2023-10-06T21:14:20.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.950+0000 7f287899a700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2038b2a30 0x55c2038b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.950+0000 7f287899a700 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2038b2a30 0x55c2038b4e50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27890fde90be1dab server_cookie=c162948358814db8 in_seq=0 out_seq=0 2023-10-06T21:14:20.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.950+0000 7f285cff9700 10 client.15852 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.951+0000 7f285cff9700 1 -- 172.21.15.26:0/1597160120 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2037d2e10 con 0x55c2038b2a30 2023-10-06T21:14:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.951+0000 7f285cff9700 10 client.15852 handle_command_reply: tid=0 2023-10-06T21:14:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.951+0000 7f288ed67b80 1 client.15852 shutdown 2023-10-06T21:14:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 msgr2=0x55c20388e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28700044a0 tx=0x7f2870007ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 shutdown_connections 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c20388f360 0x55c20389d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c2038860c0 0x55c20388e8e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2038b2a30 0x55c2038b4e50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1597160120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.952+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 >> 172.21.15.26:0/1597160120 conn(0x55c2037fb260 msgr2=0x55c203881fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.953+0000 7f285cff9700 0 client.15852 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.953+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 shutdown_connections 2023-10-06T21:14:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.953+0000 7f288ed67b80 1 -- 172.21.15.26:0/1597160120 wait complete. 2023-10-06T21:14:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.955+0000 7f288ed67b80 20 client.15852 trim_cache size 0 max 16384 2023-10-06T21:14:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.990+0000 7f288ed67b80 1 Processor -- start 2023-10-06T21:14:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.990+0000 7f288ed67b80 1 -- start start 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c2038983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c20389d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20365a920 con 0x55c20388f360 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20388f1b0 con 0x55c203891890 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f288ed67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c203879ae0 con 0x55c2038860c0 2023-10-06T21:14:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f287999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f287999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38768/0 (socket says 172.21.15.26:38768) 2023-10-06T21:14:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f287999c700 1 -- 172.21.15.26:0/1199256429 learned_addr learned my addr 172.21.15.26:0/1199256429 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.991+0000 7f287899a700 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c20389d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 -- 172.21.15.26:0/1199256429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 msgr2=0x55c2038983d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c2038983d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 -- 172.21.15.26:0/1199256429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 msgr2=0x55c20389d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c20389d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 -- 172.21.15.26:0/1199256429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20362b250 con 0x55c203891890 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f287999c700 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f286c009df0 tx=0x7f286c009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2f22b4e2636b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f285ffff700 1 -- 172.21.15.26:0/1199256429 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c009970 con 0x55c203891890 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f285ffff700 1 -- 172.21.15.26:0/1199256429 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c00f040 con 0x55c203891890 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f285ffff700 1 -- 172.21.15.26:0/1199256429 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c01f8a0 con 0x55c203891890 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 -- 172.21.15.26:0/1199256429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 msgr2=0x55c20388bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:20.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f286c009df0 tx=0x7f286c009e20 comp rx=0 tx=0).stop 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 -- 172.21.15.26:0/1199256429 shutdown_connections 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c20388bbd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c2038983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 --2- 172.21.15.26:0/1199256429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c20389d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.993+0000 7f288ed67b80 1 -- 172.21.15.26:0/1199256429 >> 172.21.15.26:0/1199256429 conn(0x55c2037fb260 msgr2=0x55c20389cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.994+0000 7f288ed67b80 1 -- 172.21.15.26:0/1199256429 shutdown_connections 2023-10-06T21:14:20.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.994+0000 7f288ed67b80 1 -- 172.21.15.26:0/1199256429 wait complete. 2023-10-06T21:14:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.994+0000 7f288ed67b80 1 -- start start 2023-10-06T21:14:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.994+0000 7f288ed67b80 1 Processor -- start 2023-10-06T21:14:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c2038965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c2038ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2038ffd90 con 0x55c20388f360 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2038fff00 con 0x55c203891890 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2039000c0 con 0x55c2038860c0 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f288ed67b80 10 client.0 _ll_register_callbacks cb 0x7f2870004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.995+0000 7f287919b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.996+0000 7f287919b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33778/0 (socket says 172.21.15.26:33778) 2023-10-06T21:14:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.996+0000 7f287919b700 1 -- 172.21.15.26:0/4022144042 learned_addr learned my addr 172.21.15.26:0/4022144042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.996+0000 7f287999c700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c2038ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 -- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 msgr2=0x55c2038ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c2038ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 -- 172.21.15.26:0/4022144042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 msgr2=0x55c2038965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c2038965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 -- 172.21.15.26:0/4022144042 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c009ad0 con 0x55c2038860c0 2023-10-06T21:14:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f287919b700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f28700081f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336f9e767c5b29d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f285cff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:14:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287000f070 con 0x55c2038860c0 2023-10-06T21:14:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.997+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f287000cab0 con 0x55c2038860c0 2023-10-06T21:14:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286409cf90 con 0x55c2038860c0 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2870023070 con 0x55c2038860c0 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 10 client.25990 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c2036032b0 con 0x55c2038860c0 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286409cf90 con 0x55c2038860c0 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2870008490 con 0x55c2038860c0 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 20 client.25990 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2039075f0 0x55c203909a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 4 client.25990 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:14:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:20.999+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2037d2b30 con 0x55c2039075f0 2023-10-06T21:14:21.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.000+0000 7f287899a700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2039075f0 0x55c203909a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.004+0000 7f287899a700 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2039075f0 0x55c203909a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207ca5b22cd2a013 server_cookie=e7b2b6bc6d2f07ca in_seq=0 out_seq=0 2023-10-06T21:14:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.004+0000 7f285cff9700 10 client.25990 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.004+0000 7f285cff9700 1 -- 172.21.15.26:0/4022144042 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c2037d2b30 con 0x55c2039075f0 2023-10-06T21:14:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.004+0000 7f285cff9700 10 client.25990 handle_command_reply: tid=0 2023-10-06T21:14:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.006+0000 7f288ed67b80 1 client.25990 shutdown 2023-10-06T21:14:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.006+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 msgr2=0x55c203896090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.006+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f28700081f0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 shutdown_connections 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c2039075f0 0x55c203909a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c203891890 0x55c2038ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c2038860c0 0x55c203896090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 --2- 172.21.15.26:0/4022144042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c20388f360 0x55c2038965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 >> 172.21.15.26:0/4022144042 conn(0x55c2037fb260 msgr2=0x55c203881660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f285cff9700 0 client.25990 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 shutdown_connections 2023-10-06T21:14:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.008+0000 7f288ed67b80 1 -- 172.21.15.26:0/4022144042 wait complete. 2023-10-06T21:14:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.010+0000 7f288ed67b80 20 client.25990 trim_cache size 0 max 16384 2023-10-06T21:14:21.012 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:14:21.012 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:14:21.013 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:14:21.013 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:14:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 msgr2=0x7f28600a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 0x7f28600a3b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2870005b40 tx=0x7f2870005ab0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 msgr2=0x7f28741ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 0x7f28741ac240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2864006fd0 tx=0x7f286400b730 comp rx=0 tx=0).stop 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 shutdown_connections 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f28600a1660 0x7f28600a3b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f287410d890 0x7f28741135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2874107eb0 0x7f28741abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 --2- 172.21.15.26:0/1341642956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2874108870 0x7f28741ac240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 >> 172.21.15.26:0/1341642956 conn(0x7f287406d400 msgr2=0x7f287406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.012+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 shutdown_connections 2023-10-06T21:14:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.014+0000 7f287b3ff700 1 -- 172.21.15.26:0/1341642956 wait complete. 2023-10-06T21:14:21.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:14:21.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:14:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 -- 172.21.15.26:0/2660363233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40103cb0 msgr2=0x7fec40107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 --2- 172.21.15.26:0/2660363233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40103cb0 0x7fec40107cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0098f0 tx=0x7fec2c009c00 comp rx=0 tx=0).stop 2023-10-06T21:14:21.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 -- 172.21.15.26:0/2660363233 shutdown_connections 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 --2- 172.21.15.26:0/2660363233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec40108420 0x7fec4010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 --2- 172.21.15.26:0/2660363233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec40103300 0x7fec401036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 --2- 172.21.15.26:0/2660363233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40103cb0 0x7fec40107cf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.445+0000 7fec46bff700 1 -- 172.21.15.26:0/2660363233 >> 172.21.15.26:0/2660363233 conn(0x7fec400feb60 msgr2=0x7fec40100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.446+0000 7fec46bff700 1 -- 172.21.15.26:0/2660363233 shutdown_connections 2023-10-06T21:14:21.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.446+0000 7fec46bff700 1 -- 172.21.15.26:0/2660363233 wait complete. 2023-10-06T21:14:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.448+0000 7fec46bff700 1 Processor -- start 2023-10-06T21:14:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.448+0000 7fec46bff700 1 -- start start 2023-10-06T21:14:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec40103300 0x7fec4010ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401afbb0 0x7fec401b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b24e0 con 0x7fec401afbb0 2023-10-06T21:14:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b2620 con 0x7fec40103300 2023-10-06T21:14:21.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec46bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401b2760 con 0x7fec40108420 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec3ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec3ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33796/0 (socket says 172.21.15.26:33796) 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.449+0000 7fec3ffff700 1 -- 172.21.15.26:0/3826692761 learned_addr learned my addr 172.21.15.26:0/3826692761 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.450+0000 7fec3ffff700 1 -- 172.21.15.26:0/3826692761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec40103300 msgr2=0x7fec4010ee90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.450+0000 7fec3ffff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec40103300 0x7fec4010ee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.450+0000 7fec3ffff700 1 -- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401afbb0 msgr2=0x7fec401b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.450+0000 7fec3ffff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401afbb0 0x7fec401b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.450+0000 7fec3ffff700 1 -- 172.21.15.26:0/3826692761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009520 con 0x7fec40108420 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec3ffff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009810 tx=0x7fec2c00bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c005520 con 0x7fec40108420 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec2c005680 con 0x7fec40108420 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c017a70 con 0x7fec40108420 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec401b29b0 con 0x7fec40108420 2023-10-06T21:14:21.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.451+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec401b2d80 con 0x7fec40108420 2023-10-06T21:14:21.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.454+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec24005320 con 0x7fec40108420 2023-10-06T21:14:21.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.458+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fec2c017bd0 con 0x7fec40108420 2023-10-06T21:14:21.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.461+0000 7fec3dffb700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 0x7fec300a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:21.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.461+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fec2c0ce660 con 0x7fec40108420 2023-10-06T21:14:21.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.462+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec2c097a40 con 0x7fec40108420 2023-10-06T21:14:21.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.463+0000 7fec4499b700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 0x7fec300a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:21.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.465+0000 7fec4499b700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 0x7fec300a3d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec34005b40 tx=0x7fec34005ca0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.954+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec24005190 con 0x7fec40108420 2023-10-06T21:14:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.959+0000 7fec3dffb700 1 -- 172.21.15.26:0/3826692761 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11317 (secure 0 0 0) 0x7fec2c026090 con 0x7fec40108420 2023-10-06T21:14:21.961 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:14:21.961 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:13:54.422161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:14:21.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.967+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 msgr2=0x7fec300a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.967+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 0x7fec300a3d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec34005b40 tx=0x7fec34005ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.967+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 msgr2=0x7fec4010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:21.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.967+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009810 tx=0x7fec2c00bc00 comp rx=0 tx=0).stop 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 shutdown_connections 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fec300a1850 0x7fec300a3d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401afbb0 0x7fec401b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec40103300 0x7fec4010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 --2- 172.21.15.26:0/3826692761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec40108420 0x7fec4010f3d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 >> 172.21.15.26:0/3826692761 conn(0x7fec400feb60 msgr2=0x7fec40100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:21.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 shutdown_connections 2023-10-06T21:14:21.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:21.968+0000 7fec46bff700 1 -- 172.21.15.26:0/3826692761 wait complete. 2023-10-06T21:14:21.977 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 35 2023-10-06T21:14:22.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:14:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 220.0 s, 1580.9 tps, lat 63.559 ms stddev 84.221 2023-10-06T21:14:22.332 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:22 smithi026 ceph-mon[98490]: pgmap v125: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 42 MiB/s wr, 2.31k op/s 2023-10-06T21:14:22.332 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:22 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3826692761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.370+0000 7f00b27ff700 1 -- 172.21.15.26:0/2632393264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 msgr2=0x7f00ac10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.370+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2632393264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10dd60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009990 tx=0x7f00a4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 -- 172.21.15.26:0/2632393264 shutdown_connections 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2632393264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10dd60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2632393264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac1010c0 0x7f00ac1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2632393264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac101a70 0x7f00ac105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 -- 172.21.15.26:0/2632393264 >> 172.21.15.26:0/2632393264 conn(0x7f00ac0fc960 msgr2=0x7f00ac0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 -- 172.21.15.26:0/2632393264 shutdown_connections 2023-10-06T21:14:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.371+0000 7f00b27ff700 1 -- 172.21.15.26:0/2632393264 wait complete. 2023-10-06T21:14:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.373+0000 7f00b27ff700 1 Processor -- start 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.375+0000 7f00b27ff700 1 -- start start 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 0x7f00ac10a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 0x7f00ac1b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac072dd0 con 0x7f00ac070910 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac072f10 con 0x7f00ac1010c0 2023-10-06T21:14:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.376+0000 7f00b27ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac113d40 con 0x7f00ac1061e0 2023-10-06T21:14:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00b0d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 0x7f00ac1b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00b0d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 0x7f00ac1b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58834/0 (socket says 172.21.15.26:58834) 2023-10-06T21:14:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00b0d9c700 1 -- 172.21.15.26:0/2361196726 learned_addr learned my addr 172.21.15.26:0/2361196726 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00abfff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 0x7f00ac10a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 -- 172.21.15.26:0/2361196726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 msgr2=0x7f00ac10a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 0x7f00ac10a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 -- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 msgr2=0x7f00ac1b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 0x7f00ac1b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00a3fff700 1 -- 172.21.15.26:0/2361196726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40095f0 con 0x7f00ac1061e0 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.377+0000 7f00abfff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 0x7f00ac10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.378+0000 7f00a3fff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10ab70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f009400d950 tx=0x7f009400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.378+0000 7f00a9ffb700 1 -- 172.21.15.26:0/2361196726 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094009930 con 0x7f00ac1061e0 2023-10-06T21:14:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.378+0000 7f00a9ffb700 1 -- 172.21.15.26:0/2361196726 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0094010460 con 0x7f00ac1061e0 2023-10-06T21:14:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.378+0000 7f00a9ffb700 1 -- 172.21.15.26:0/2361196726 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400b5d0 con 0x7f00ac1061e0 2023-10-06T21:14:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.378+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac114020 con 0x7f00ac1061e0 2023-10-06T21:14:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.379+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac1144e0 con 0x7f00ac1061e0 2023-10-06T21:14:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.385+0000 7f00a9ffb700 1 -- 172.21.15.26:0/2361196726 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f00940105d0 con 0x7f00ac1061e0 2023-10-06T21:14:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.388+0000 7f00c604db80 1 Processor -- start 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 -- start start 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 0x55d60d3ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d347460 con 0x55d60d3de0c0 2023-10-06T21:14:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d338c20 con 0x55d60d3e9890 2023-10-06T21:14:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.389+0000 7f00c604db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d336540 con 0x55d60d3de9e0 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.390+0000 7f00abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.390+0000 7f00abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33822/0 (socket says 172.21.15.26:33822) 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.390+0000 7f00abfff700 1 -- 172.21.15.26:0/3769852229 learned_addr learned my addr 172.21.15.26:0/3769852229 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.390+0000 7f00abfff700 1 -- 172.21.15.26:0/3769852229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 msgr2=0x55d60d3ebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.390+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00a3fff700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 0x55d60d3ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00abfff700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 0x55d60d3ebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00abfff700 1 -- 172.21.15.26:0/3769852229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 msgr2=0x55d60d3de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00abfff700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3de4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00abfff700 1 -- 172.21.15.26:0/3769852229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d60d15cde0 con 0x55d60d3de9e0 2023-10-06T21:14:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00a3fff700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 0x55d60d3ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00abfff700 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c4a0 tx=0x7f009c00c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ad00c8736c433a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.391+0000 7f00a9ffb700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 0x7f00980a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.392+0000 7f00a9ffb700 1 -- 172.21.15.26:0/2361196726 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f00940ccc00 con 0x7f00ac1061e0 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.392+0000 7f00a27fc700 1 -- 172.21.15.26:0/3769852229 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c011070 con 0x55d60d3de9e0 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.392+0000 7f00a27fc700 1 -- 172.21.15.26:0/3769852229 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f009c00cbf0 con 0x55d60d3de9e0 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.392+0000 7f00a27fc700 1 -- 172.21.15.26:0/3769852229 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00a660 con 0x55d60d3de9e0 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.393+0000 7f00c604db80 1 -- 172.21.15.26:0/3769852229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 msgr2=0x55d60d3e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.393+0000 7f00c604db80 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c4a0 tx=0x7f009c00c580 comp rx=0 tx=0).stop 2023-10-06T21:14:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.393+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 0x7f00980a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.394+0000 7f00c604db80 1 -- 172.21.15.26:0/3769852229 shutdown_connections 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.394+0000 7f00c604db80 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3e9890 0x55d60d3ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.394+0000 7f00c604db80 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.394+0000 7f00c604db80 1 --2- 172.21.15.26:0/3769852229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3de9e0 0x55d60d3e9350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.394+0000 7f00c604db80 1 -- 172.21.15.26:0/3769852229 >> 172.21.15.26:0/3769852229 conn(0x55d60d353260 msgr2=0x55d60d3da450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.395+0000 7f00c604db80 1 -- 172.21.15.26:0/3769852229 shutdown_connections 2023-10-06T21:14:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.395+0000 7f00c604db80 1 -- 172.21.15.26:0/3769852229 wait complete. 2023-10-06T21:14:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.395+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 0x7f00980a3b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f00a40036f0 tx=0x7f00a4003680 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.395+0000 7f00c604db80 1 -- start start 2023-10-06T21:14:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 Processor -- start 2023-10-06T21:14:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de9e0 0x55d60d3e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d331e30 con 0x55d60d3de0c0 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d2ce710 con 0x55d60d3de9e0 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.396+0000 7f00c604db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d2ce3b0 con 0x55d60d3e9890 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00c604db80 10 client.0 _ll_register_callbacks cb 0x55d60d12b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de9e0 0x55d60d3e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3e65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58842/0 (socket says 172.21.15.26:58842) 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00b0d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.397+0000 7f00abfff700 1 -- 172.21.15.26:0/2311479689 learned_addr learned my addr 172.21.15.26:0/2311479689 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.398+0000 7f00b0d9c700 1 -- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de9e0 msgr2=0x55d60d3e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.398+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de9e0 0x55d60d3e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.398+0000 7f00b0d9c700 1 -- 172.21.15.26:0/2311479689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 msgr2=0x55d60d3e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.398+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.398+0000 7f00b0d9c700 1 -- 172.21.15.26:0/2311479689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c009d20 con 0x55d60d3e9890 2023-10-06T21:14:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.401+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e7020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f00a40092e0 tx=0x7f00a4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e600aa4f535b65e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.401+0000 7f008b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:14:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.401+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a402c070 con 0x55d60d3e9890 2023-10-06T21:14:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.401+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f00a4004710 con 0x55d60d3e9890 2023-10-06T21:14:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.401+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d60d3d1f10 con 0x55d60d3e9890 2023-10-06T21:14:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a40048d0 con 0x55d60d3e9890 2023-10-06T21:14:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f00c604db80 10 client.26008 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d60d145c60 con 0x55d60d3e9890 2023-10-06T21:14:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d60d3d1f10 con 0x55d60d3e9890 2023-10-06T21:14:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f00a4038070 con 0x55d60d3e9890 2023-10-06T21:14:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f00c604db80 10 client.26008 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.403+0000 7f00c604db80 10 client.26008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:14:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.404+0000 7f00c604db80 20 client.26008 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.404+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d3f9250 0x55d60d3fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.404+0000 7f00c604db80 4 client.26008 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.404+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d60d32ae10 con 0x55d60d3f9250 2023-10-06T21:14:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.405+0000 7f00abfff700 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d3f9250 0x55d60d3fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.405+0000 7f00abfff700 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d3f9250 0x55d60d3fb670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82a83785590b6d02 server_cookie=8b580418518bd169 in_seq=0 out_seq=0 2023-10-06T21:14:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.405+0000 7f008b7fe700 10 client.26008 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.406+0000 7f008b7fe700 1 -- 172.21.15.26:0/2311479689 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d60d32ae10 con 0x55d60d3f9250 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.406+0000 7f008b7fe700 10 client.26008 handle_command_reply: tid=0 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.406+0000 7f00c604db80 1 client.26008 shutdown 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 msgr2=0x55d60d3e7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e7020 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f00a40092e0 tx=0x7f00a4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 shutdown_connections 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d3f9250 0x55d60d3fb670 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e7020 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de0c0 0x55d60d3e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 --2- 172.21.15.26:0/2311479689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de9e0 0x55d60d3e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 >> 172.21.15.26:0/2311479689 conn(0x55d60d353260 msgr2=0x55d60d3da040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f008b7fe700 0 client.26008 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 shutdown_connections 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 1 -- 172.21.15.26:0/2311479689 wait complete. 2023-10-06T21:14:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.407+0000 7f00c604db80 20 client.26008 trim_cache size 0 max 16384 2023-10-06T21:14:22.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.431+0000 7f00c604db80 1 Processor -- start 2023-10-06T21:14:22.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.431+0000 7f00c604db80 1 -- start start 2023-10-06T21:14:22.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d1b2920 con 0x55d60d3de9e0 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d3e2430 con 0x55d60d3de0c0 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00c604db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d3e9650 con 0x55d60d3e9890 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00abfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38848/0 (socket says 172.21.15.26:38848) 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00abfff700 1 -- 172.21.15.26:0/3531683668 learned_addr learned my addr 172.21.15.26:0/3531683668 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.432+0000 7f00a3fff700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 -- 172.21.15.26:0/3531683668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 msgr2=0x55d60d3e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 -- 172.21.15.26:0/3531683668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 msgr2=0x55d60d3f03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3f03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 -- 172.21.15.26:0/3531683668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d60d3d6140 con 0x55d60d3de0c0 2023-10-06T21:14:22.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00a3fff700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00abfff700 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f009c0097d0 tx=0x7f009c004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a846afe0d079655 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00a2ffd700 1 -- 172.21.15.26:0/3531683668 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c028030 con 0x55d60d3de0c0 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00a2ffd700 1 -- 172.21.15.26:0/3531683668 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f009c024cd0 con 0x55d60d3de0c0 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.433+0000 7f00a2ffd700 1 -- 172.21.15.26:0/3531683668 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c011040 con 0x55d60d3de0c0 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.434+0000 7f00c604db80 1 -- 172.21.15.26:0/3531683668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 msgr2=0x55d60d3e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.434+0000 7f00c604db80 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f009c0097d0 tx=0x7f009c004be0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.435+0000 7f00c604db80 1 -- 172.21.15.26:0/3531683668 shutdown_connections 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.435+0000 7f00c604db80 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d3e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.435+0000 7f00c604db80 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3e1080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.435+0000 7f00c604db80 1 --2- 172.21.15.26:0/3531683668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.435+0000 7f00c604db80 1 -- 172.21.15.26:0/3531683668 >> 172.21.15.26:0/3531683668 conn(0x55d60d353260 msgr2=0x55d60d3e18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.436+0000 7f00c604db80 1 -- 172.21.15.26:0/3531683668 shutdown_connections 2023-10-06T21:14:22.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.437+0000 7f00c604db80 1 -- 172.21.15.26:0/3531683668 wait complete. 2023-10-06T21:14:22.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.437+0000 7f00c604db80 1 -- start start 2023-10-06T21:14:22.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.437+0000 7f00c604db80 1 Processor -- start 2023-10-06T21:14:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d44c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d3eea40 con 0x55d60d3de9e0 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d44c5e0 con 0x55d60d3de0c0 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d44c720 con 0x55d60d3e9890 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00c604db80 10 client.0 _ll_register_callbacks cb 0x7f00a40053f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58860/0 (socket says 172.21.15.26:58860) 2023-10-06T21:14:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00a3fff700 1 -- 172.21.15.26:0/1252543025 learned_addr learned my addr 172.21.15.26:0/1252543025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00abfff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.439+0000 7f00b0d9c700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d44c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 -- 172.21.15.26:0/1252543025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 msgr2=0x55d60d44c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d44c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 -- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 msgr2=0x55d60d3edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 -- 172.21.15.26:0/1252543025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d60d15cde0 con 0x55d60d3de9e0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f00a3fff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0094099b80 tx=0x7f009409a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cab21405b80dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.440+0000 7f008b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.441+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940a1f80 con 0x55d60d3de9e0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.441+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f00940108b0 con 0x55d60d3de9e0 2023-10-06T21:14:22.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.441+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f009c00cbf0 con 0x55d60d3de9e0 2023-10-06T21:14:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.444+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0094096300 con 0x55d60d3de9e0 2023-10-06T21:14:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 10 client.15867 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d60d183250 con 0x55d60d3de9e0 2023-10-06T21:14:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f009c00cbf0 con 0x55d60d3de9e0 2023-10-06T21:14:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f00940990e0 con 0x55d60d3de9e0 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 10 client.15867 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 20 client.15867 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d44dc90 0x55d60d4500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 4 client.15867 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.445+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d60d32ab30 con 0x55d60d44dc90 2023-10-06T21:14:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.446+0000 7f00abfff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d44dc90 0x55d60d4500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.448+0000 7f00abfff700 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d44dc90 0x55d60d4500b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69d9fb7da4b59f99 server_cookie=69506c6218b72e99 in_seq=0 out_seq=0 2023-10-06T21:14:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.449+0000 7f008b7fe700 10 client.15867 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:14:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.449+0000 7f008b7fe700 1 -- 172.21.15.26:0/1252543025 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d60d32ab30 con 0x55d60d44dc90 2023-10-06T21:14:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.449+0000 7f008b7fe700 10 client.15867 handle_command_reply: tid=0 2023-10-06T21:14:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 client.15867 shutdown 2023-10-06T21:14:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 msgr2=0x55d60d3ee130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0094099b80 tx=0x7f009409a370 comp rx=0 tx=0).stop 2023-10-06T21:14:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 shutdown_connections 2023-10-06T21:14:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x55d60d44dc90 0x55d60d4500b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d60d3e9890 0x55d60d44c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d60d3de0c0 0x55d60d3edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 --2- 172.21.15.26:0/1252543025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d3de9e0 0x55d60d3ee130 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 >> 172.21.15.26:0/1252543025 conn(0x55d60d353260 msgr2=0x55d60d3e6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f008b7fe700 0 client.15867 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 shutdown_connections 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 1 -- 172.21.15.26:0/1252543025 wait complete. 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.450+0000 7f00c604db80 20 client.15867 trim_cache size 0 max 16384 2023-10-06T21:14:22.453 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:14:22.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 msgr2=0x7f00980a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 0x7f00980a3b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f00a40036f0 tx=0x7f00a4003680 comp rx=0 tx=0).stop 2023-10-06T21:14:22.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 msgr2=0x7f00ac10ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10ab70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f009400d950 tx=0x7f009400dc60 comp rx=0 tx=0).stop 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 shutdown_connections 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00980a16a0 0x7f00980a3b60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ac070910 0x7f00ac1b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00ac1010c0 0x7f00ac10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 --2- 172.21.15.26:0/2361196726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00ac1061e0 0x7f00ac10ab70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.451+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 >> 172.21.15.26:0/2361196726 conn(0x7f00ac0fc960 msgr2=0x7f00ac104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.452+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 shutdown_connections 2023-10-06T21:14:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.452+0000 7f00b27ff700 1 -- 172.21.15.26:0/2361196726 wait complete. 2023-10-06T21:14:22.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:14:22.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:22 smithi167 ceph-mon[134494]: pgmap v125: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 42 MiB/s wr, 2.31k op/s 2023-10-06T21:14:22.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:22 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3826692761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:14:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:22 smithi078 ceph-mon[134896]: pgmap v125: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 42 MiB/s wr, 2.31k op/s 2023-10-06T21:14:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:22 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3826692761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:14:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 -- 172.21.15.26:0/1376505992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24201083f0 msgr2=0x7f2420108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 --2- 172.21.15.26:0/1376505992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24201083f0 0x7f2420108870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2414009990 tx=0x7f2414009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 -- 172.21.15.26:0/1376505992 shutdown_connections 2023-10-06T21:14:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 --2- 172.21.15.26:0/1376505992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242011a690 0x7f242011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 --2- 172.21.15.26:0/1376505992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f2420107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 --2- 172.21.15.26:0/1376505992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24201083f0 0x7f2420108870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.852+0000 7f2424dff700 1 -- 172.21.15.26:0/1376505992 >> 172.21.15.26:0/1376505992 conn(0x7f242006d1a0 msgr2=0x7f242006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.856+0000 7f2424dff700 1 -- 172.21.15.26:0/1376505992 shutdown_connections 2023-10-06T21:14:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.856+0000 7f2424dff700 1 -- 172.21.15.26:0/1376505992 wait complete. 2023-10-06T21:14:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.857+0000 7f2424dff700 1 Processor -- start 2023-10-06T21:14:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.858+0000 7f2424dff700 1 -- start start 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.858+0000 7f2424dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 0x7f24201a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.858+0000 7f2424dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f24201a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f2424dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242011a690 0x7f24201aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f2424dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201aae40 con 0x7f242011a690 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f2424dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201aafb0 con 0x7f242010f640 2023-10-06T21:14:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f2424dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201ab170 con 0x7f24201083f0 2023-10-06T21:14:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f24201a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f24201a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38886/0 (socket says 172.21.15.26:38886) 2023-10-06T21:14:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f241dd9b700 1 -- 172.21.15.26:0/2790261102 learned_addr learned my addr 172.21.15.26:0/2790261102 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.859+0000 7f241e59c700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 0x7f24201a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.861+0000 7f241e59c700 1 -- 172.21.15.26:0/2790261102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 msgr2=0x7f24201a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.861+0000 7f241e59c700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f24201a63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.861+0000 7f241e59c700 1 -- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242011a690 msgr2=0x7f24201aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.861+0000 7f241e59c700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242011a690 0x7f24201aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.861+0000 7f241e59c700 1 -- 172.21.15.26:0/2790261102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140095f0 con 0x7f24201083f0 2023-10-06T21:14:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.862+0000 7f241e59c700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 0x7f24201a5e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f241000e9f0 tx=0x7f241000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.863+0000 7f240f7fe700 1 -- 172.21.15.26:0/2790261102 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241000eee0 con 0x7f24201083f0 2023-10-06T21:14:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.863+0000 7f240f7fe700 1 -- 172.21.15.26:0/2790261102 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241000cbd0 con 0x7f24201083f0 2023-10-06T21:14:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.863+0000 7f240f7fe700 1 -- 172.21.15.26:0/2790261102 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410018680 con 0x7f24201083f0 2023-10-06T21:14:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.863+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24201ab450 con 0x7f24201083f0 2023-10-06T21:14:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.863+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24201b2c90 con 0x7f24201083f0 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.870+0000 7f24387f0b80 1 Processor -- start 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 -- start start 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 0x55ef695c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c69e0 0x55ef695d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6952f460 con 0x55ef695c69e0 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef69520c20 con 0x55ef695d1890 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.871+0000 7f24387f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6951e540 con 0x55ef695c60c0 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 0x55ef695c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38888/0 (socket says 172.21.15.26:38888) 2023-10-06T21:14:22.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 -- 172.21.15.26:0/1798601454 learned_addr learned my addr 172.21.15.26:0/1798601454 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f240f7fe700 1 -- 172.21.15.26:0/2790261102 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f241000cd40 con 0x7f24201083f0 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 -- 172.21.15.26:0/1798601454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 msgr2=0x55ef695c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 0x55ef695c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 -- 172.21.15.26:0/1798601454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c69e0 msgr2=0x55ef695d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c69e0 0x55ef695d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.872+0000 7f241dd9b700 1 -- 172.21.15.26:0/1798601454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef69344de0 con 0x55ef695d1890 2023-10-06T21:14:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.873+0000 7f241ed9d700 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 0x55ef695c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.873+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f241400b640 tx=0x7f241400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c112e3c5fa073e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.875+0000 7f240f7fe700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 0x7f24080a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.875+0000 7f240f7fe700 1 -- 172.21.15.26:0/2790261102 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2410014070 con 0x7f24201083f0 2023-10-06T21:14:22.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.876+0000 7f240cff9700 1 -- 172.21.15.26:0/1798601454 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241401d070 con 0x55ef695d1890 2023-10-06T21:14:22.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.876+0000 7f240cff9700 1 -- 172.21.15.26:0/1798601454 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241401e040 con 0x55ef695d1890 2023-10-06T21:14:22.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.876+0000 7f240cff9700 1 -- 172.21.15.26:0/1798601454 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24140049a0 con 0x55ef695d1890 2023-10-06T21:14:22.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f241ed9d700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 0x7f24080a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 -- 172.21.15.26:0/1798601454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 msgr2=0x55ef695d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f241400b640 tx=0x7f241400b670 comp rx=0 tx=0).stop 2023-10-06T21:14:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 -- 172.21.15.26:0/1798601454 shutdown_connections 2023-10-06T21:14:22.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695d1890 0x55ef695d3c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c60c0 0x55ef695c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.877+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1798601454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c69e0 0x55ef695d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.878+0000 7f24387f0b80 1 -- 172.21.15.26:0/1798601454 >> 172.21.15.26:0/1798601454 conn(0x55ef6953b260 msgr2=0x55ef695c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.878+0000 7f24387f0b80 1 -- 172.21.15.26:0/1798601454 shutdown_connections 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.878+0000 7f241ed9d700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 0x7f24080a3b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f242006a4d0 tx=0x7f241800d040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.878+0000 7f24387f0b80 1 -- 172.21.15.26:0/1798601454 wait complete. 2023-10-06T21:14:22.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.878+0000 7f24387f0b80 1 -- start start 2023-10-06T21:14:22.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.879+0000 7f24387f0b80 1 Processor -- start 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef69519e30 con 0x55ef695c60c0 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef694b6710 con 0x55ef695c69e0 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef694b63b0 con 0x55ef695d1890 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f24387f0b80 10 client.0 _ll_register_callbacks cb 0x55ef69313800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f241ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f241ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33888/0 (socket says 172.21.15.26:33888) 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f241ed9d700 1 -- 172.21.15.26:0/1275759937 learned_addr learned my addr 172.21.15.26:0/1275759937 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.880+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 -- 172.21.15.26:0/1275759937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 msgr2=0x55ef695cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 -- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 msgr2=0x55ef695ce5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695ce5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 -- 172.21.15.26:0/1275759937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241800e0c0 con 0x55ef695c69e0 2023-10-06T21:14:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f241dd9b700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695ceb30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f241400b610 tx=0x7f2414023690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f6cd51626a6618 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.881+0000 7f2405ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.882+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241401d070 con 0x55ef695c69e0 2023-10-06T21:14:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.882+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f241400f660 con 0x55ef695c69e0 2023-10-06T21:14:22.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.882+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef695b9f10 con 0x55ef695c69e0 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.883+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2414005bc0 con 0x55ef695c69e0 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.883+0000 7f24387f0b80 10 client.26027 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.883+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f242011d430 con 0x55ef695c69e0 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.883+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef695b9f10 con 0x55ef695c69e0 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.883+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f24140173f0 con 0x55ef695c69e0 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 10 client.26027 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:14:22.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 20 client.26027 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef695e0e70 0x55ef695e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 4 client.26027 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.884+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef69512e10 con 0x55ef695e0e70 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.885+0000 7f241e59c700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef695e0e70 0x55ef695e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.885+0000 7f241e59c700 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef695e0e70 0x55ef695e3260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5414c022ec8bbdf server_cookie=6e07b97ac2270604 in_seq=0 out_seq=0 2023-10-06T21:14:22.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.886+0000 7f2405ffb700 10 client.26027 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:14:22.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.888+0000 7f2405ffb700 1 -- 172.21.15.26:0/1275759937 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ef69512e10 con 0x55ef695e0e70 2023-10-06T21:14:22.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.888+0000 7f2405ffb700 10 client.26027 handle_command_reply: tid=0 2023-10-06T21:14:22.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 client.26027 shutdown 2023-10-06T21:14:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 msgr2=0x55ef695ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695ceb30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f241400b610 tx=0x7f2414023690 comp rx=0 tx=0).stop 2023-10-06T21:14:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 shutdown_connections 2023-10-06T21:14:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef695e0e70 0x55ef695e3260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 --2- 172.21.15.26:0/1275759937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695ceb30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 >> 172.21.15.26:0/1275759937 conn(0x55ef6953b260 msgr2=0x55ef695c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.889+0000 7f2405ffb700 0 client.26027 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.890+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 shutdown_connections 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.890+0000 7f24387f0b80 1 -- 172.21.15.26:0/1275759937 wait complete. 2023-10-06T21:14:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.890+0000 7f24387f0b80 20 client.26027 trim_cache size 0 max 16384 2023-10-06T21:14:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.900+0000 7f43171ff700 1 -- 172.21.15.26:0/2249402983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4310105e20 msgr2=0x7f43100682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.900+0000 7f43171ff700 1 --2- 172.21.15.26:0/2249402983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4310105e20 0x7f43100682d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4300009990 tx=0x7f4300009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.901+0000 7f43171ff700 1 -- 172.21.15.26:0/2249402983 shutdown_connections 2023-10-06T21:14:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.901+0000 7f43171ff700 1 --2- 172.21.15.26:0/2249402983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43101038b0 0x7f4310103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.901+0000 7f43171ff700 1 --2- 172.21.15.26:0/2249402983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4310105e20 0x7f43100682d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.901+0000 7f43171ff700 1 --2- 172.21.15.26:0/2249402983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310068810 0x7f4310068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.901+0000 7f43171ff700 1 -- 172.21.15.26:0/2249402983 >> 172.21.15.26:0/2249402983 conn(0x7f4310076b70 msgr2=0x7f4310076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.905+0000 7f43171ff700 1 -- 172.21.15.26:0/2249402983 shutdown_connections 2023-10-06T21:14:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.905+0000 7f43171ff700 1 -- 172.21.15.26:0/2249402983 wait complete. 2023-10-06T21:14:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 Processor -- start 2023-10-06T21:14:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 -- start start 2023-10-06T21:14:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 0x7f431019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 0x7f43101a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43101a15b0 con 0x7f4310105e20 2023-10-06T21:14:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43101a1720 con 0x7f4310068810 2023-10-06T21:14:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.907+0000 7f43171ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43101a18e0 con 0x7f43101038b0 2023-10-06T21:14:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4307fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4307fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33904/0 (socket says 172.21.15.26:33904) 2023-10-06T21:14:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4307fff700 1 -- 172.21.15.26:0/2968045780 learned_addr learned my addr 172.21.15.26:0/2968045780 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 0x7f431019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f431579c700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 0x7f43101a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 -- 172.21.15.26:0/2968045780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 msgr2=0x7f431019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 -- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 msgr2=0x7f43101a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 0x7f43101a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4314f9b700 1 -- 172.21.15.26:0/2968045780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000095f0 con 0x7f4310068810 2023-10-06T21:14:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f431579c700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 0x7f43101a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.908+0000 7f4307fff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.929+0000 7f4314f9b700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 0x7f431019d4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43000098e0 tx=0x7f430000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.923+0000 7f24387f0b80 1 Processor -- start 2023-10-06T21:14:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.923+0000 7f24387f0b80 1 -- start start 2023-10-06T21:14:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 0x55ef695cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 0x55ef695c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695d1890 0x55ef695cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6939a920 con 0x55ef695d1890 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef695c3880 con 0x55ef695c60c0 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.924+0000 7f24387f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef695b9de0 con 0x55ef695c69e0 2023-10-06T21:14:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.925+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 0x55ef695c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.925+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 0x55ef695c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33916/0 (socket says 172.21.15.26:33916) 2023-10-06T21:14:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.925+0000 7f241dd9b700 1 -- 172.21.15.26:0/3665257150 learned_addr learned my addr 172.21.15.26:0/3665257150 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.926+0000 7f241e59c700 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 0x55ef695cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.928+0000 7f241e59c700 1 -- 172.21.15.26:0/3665257150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 msgr2=0x55ef695c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.928+0000 7f241e59c700 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 0x55ef695c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.928+0000 7f241e59c700 1 -- 172.21.15.26:0/3665257150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695d1890 msgr2=0x55ef695cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.928+0000 7f241e59c700 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695d1890 0x55ef695cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.928+0000 7f241e59c700 1 -- 172.21.15.26:0/3665257150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef6932dc60 con 0x55ef695c60c0 2023-10-06T21:14:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.935+0000 7f4305ffb700 1 -- 172.21.15.26:0/2968045780 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430001d070 con 0x7f4310068810 2023-10-06T21:14:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.935+0000 7f4305ffb700 1 -- 172.21.15.26:0/2968045780 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430000b610 con 0x7f4310068810 2023-10-06T21:14:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.935+0000 7f4305ffb700 1 -- 172.21.15.26:0/2968045780 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43000173f0 con 0x7f4310068810 2023-10-06T21:14:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.936+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43101a1bc0 con 0x7f4310068810 2023-10-06T21:14:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.936+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43101aa520 con 0x7f4310068810 2023-10-06T21:14:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.939+0000 7f241e59c700 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 0x55ef695cd880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24100ca9c0 tx=0x7f2410009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82215862ade044ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.943+0000 7f432ab7cb80 1 Processor -- start 2023-10-06T21:14:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.943+0000 7f432ab7cb80 1 -- start start 2023-10-06T21:14:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bacc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479ba28400 con 0x56479babf520 2023-10-06T21:14:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479ba19bc0 con 0x56479baca3d0 2023-10-06T21:14:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479ba174e0 con 0x56479babec00 2023-10-06T21:14:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f240d7fa700 1 -- 172.21.15.26:0/3665257150 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241009a250 con 0x55ef695c60c0 2023-10-06T21:14:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f240d7fa700 1 -- 172.21.15.26:0/3665257150 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241009e3a0 con 0x55ef695c60c0 2023-10-06T21:14:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.944+0000 7f240d7fa700 1 -- 172.21.15.26:0/3665257150 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100cf2f0 con 0x55ef695c60c0 2023-10-06T21:14:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 -- 172.21.15.26:0/3665257150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 msgr2=0x55ef695cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 0x55ef695cd880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24100ca9c0 tx=0x7f2410009ef0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 -- 172.21.15.26:0/3665257150 shutdown_connections 2023-10-06T21:14:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695d1890 0x55ef695cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c60c0 0x55ef695cd880 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 --2- 172.21.15.26:0/3665257150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695c69e0 0x55ef695c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f24387f0b80 1 -- 172.21.15.26:0/3665257150 >> 172.21.15.26:0/3665257150 conn(0x55ef6953b260 msgr2=0x55ef695c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f431579c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f431579c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33930/0 (socket says 172.21.15.26:33930) 2023-10-06T21:14:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.945+0000 7f431579c700 1 -- 172.21.15.26:0/4244779712 learned_addr learned my addr 172.21.15.26:0/4244779712 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f4307fff700 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bacc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f431579c700 1 -- 172.21.15.26:0/4244779712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 msgr2=0x56479bacc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f431579c700 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bacc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f431579c700 1 -- 172.21.15.26:0/4244779712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 msgr2=0x56479bac9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f431579c700 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f431579c700 1 -- 172.21.15.26:0/4244779712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56479b83dd80 con 0x56479babec00 2023-10-06T21:14:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.946+0000 7f4307fff700 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bacc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.948+0000 7f24387f0b80 1 -- 172.21.15.26:0/3665257150 shutdown_connections 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.949+0000 7f24387f0b80 1 -- 172.21.15.26:0/3665257150 wait complete. 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.949+0000 7f24387f0b80 1 -- start start 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 Processor -- start 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695d8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef695d6f80 con 0x55ef695c60c0 2023-10-06T21:14:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef695d70c0 con 0x55ef695c69e0 2023-10-06T21:14:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef695d7280 con 0x55ef695d1890 2023-10-06T21:14:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.950+0000 7f24387f0b80 10 client.0 _ll_register_callbacks cb 0x7f242006a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.952+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.952+0000 7f241dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695d6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38954/0 (socket says 172.21.15.26:38954) 2023-10-06T21:14:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.952+0000 7f241dd9b700 1 -- 172.21.15.26:0/948630909 learned_addr learned my addr 172.21.15.26:0/948630909 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.953+0000 7f241e59c700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695d8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.953+0000 7f241e59c700 1 -- 172.21.15.26:0/948630909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 msgr2=0x55ef695d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f241e59c700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f241e59c700 1 -- 172.21.15.26:0/948630909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 msgr2=0x55ef695d6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f241e59c700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695d6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f241e59c700 1 -- 172.21.15.26:0/948630909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24201b2c90 con 0x55ef695c60c0 2023-10-06T21:14:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f241e59c700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695d8210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f241009a1e0 tx=0x7f2410005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c68c62833ecd4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f2405ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100cf0b0 con 0x55ef695c60c0 2023-10-06T21:14:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.954+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f24100cf210 con 0x55ef695c60c0 2023-10-06T21:14:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f431579c700 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d50 tx=0x7f43080116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0fe7816010e06e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f241401e040 con 0x55ef695c60c0 2023-10-06T21:14:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f42f6ffd700 1 -- 172.21.15.26:0/4244779712 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011cd0 con 0x56479babec00 2023-10-06T21:14:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f42f6ffd700 1 -- 172.21.15.26:0/4244779712 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4308011e30 con 0x56479babec00 2023-10-06T21:14:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f42f6ffd700 1 -- 172.21.15.26:0/4244779712 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430800f870 con 0x56479babec00 2023-10-06T21:14:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.956+0000 7f432ab7cb80 1 -- 172.21.15.26:0/4244779712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 msgr2=0x56479babefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.956+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d50 tx=0x7f43080116c0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.955+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241009a4b0 con 0x55ef695c60c0 2023-10-06T21:14:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.957+0000 7f24387f0b80 10 client.15879 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.957+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ef6936b250 con 0x55ef695c60c0 2023-10-06T21:14:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f432ab7cb80 1 -- 172.21.15.26:0/4244779712 shutdown_connections 2023-10-06T21:14:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bacc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479babefe0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/4244779712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f432ab7cb80 1 -- 172.21.15.26:0/4244779712 >> 172.21.15.26:0/4244779712 conn(0x56479ba34200 msgr2=0x56479babb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.958+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241401e040 con 0x55ef695c60c0 2023-10-06T21:14:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f24100a2150 con 0x55ef695c60c0 2023-10-06T21:14:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 20 client.15879 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef69635c50 0x55ef69638070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 4 client.15879 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.959+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ef69512b30 con 0x55ef69635c50 2023-10-06T21:14:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.961+0000 7f241dd9b700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef69635c50 0x55ef69638070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.960+0000 7f4305ffb700 1 -- 172.21.15.26:0/2968045780 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4300021410 con 0x7f4310068810 2023-10-06T21:14:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.960+0000 7f432ab7cb80 1 -- 172.21.15.26:0/4244779712 shutdown_connections 2023-10-06T21:14:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.961+0000 7f432ab7cb80 1 -- 172.21.15.26:0/4244779712 wait complete. 2023-10-06T21:14:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.962+0000 7f241dd9b700 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef69635c50 0x55ef69638070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5a61ef9e3e157b9 server_cookie=aae5b324f14114 in_seq=0 out_seq=0 2023-10-06T21:14:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.962+0000 7f2405ffb700 10 client.15879 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:14:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.962+0000 7f432ab7cb80 1 -- start start 2023-10-06T21:14:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.963+0000 7f432ab7cb80 1 Processor -- start 2023-10-06T21:14:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.963+0000 7f4305ffb700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 0x7f42f80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.963+0000 7f4305ffb700 1 -- 172.21.15.26:0/2968045780 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f43000cde90 con 0x7f4310068810 2023-10-06T21:14:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f2405ffb700 1 -- 172.21.15.26:0/948630909 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ef69512b30 con 0x55ef69635c50 2023-10-06T21:14:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f2405ffb700 10 client.15879 handle_command_reply: tid=0 2023-10-06T21:14:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f24387f0b80 1 client.15879 shutdown 2023-10-06T21:14:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bac71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f431579c700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 0x7f42f80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479ba12dd0 con 0x56479babec00 2023-10-06T21:14:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479b9af6b0 con 0x56479baca3d0 2023-10-06T21:14:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479b9af350 con 0x56479babf520 2023-10-06T21:14:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f432ab7cb80 10 client.0 _ll_register_callbacks cb 0x56479b80c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 msgr2=0x55ef695d8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.966+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695d8210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f241009a1e0 tx=0x7f2410005b40 comp rx=0 tx=0).stop 2023-10-06T21:14:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 shutdown_connections 2023-10-06T21:14:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55ef69635c50 0x55ef69638070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ef695d1890 0x55ef695d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef695c60c0 0x55ef695d8210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 --2- 172.21.15.26:0/948630909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef695c69e0 0x55ef695d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 >> 172.21.15.26:0/948630909 conn(0x55ef6953b260 msgr2=0x55ef695cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f2405ffb700 0 client.15879 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:14:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f4314f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bac71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f4307fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f4307fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33944/0 (socket says 172.21.15.26:33944) 2023-10-06T21:14:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f4307fff700 1 -- 172.21.15.26:0/1906660057 learned_addr learned my addr 172.21.15.26:0/1906660057 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f431579c700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 shutdown_connections 2023-10-06T21:14:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.967+0000 7f24387f0b80 1 -- 172.21.15.26:0/948630909 wait complete. 2023-10-06T21:14:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f24387f0b80 20 client.15879 trim_cache size 0 max 16384 2023-10-06T21:14:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 -- 172.21.15.26:0/1906660057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 msgr2=0x56479bac7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 -- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 msgr2=0x56479bac7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.978 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:14:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 -- 172.21.15.26:0/1906660057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4308009980 con 0x56479babec00 2023-10-06T21:14:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4307fff700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f431579c700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 0x7f42f80a3bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4308009950 tx=0x7f4308009820 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f4314f9b700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bac71f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f43000cb560 tx=0x7f43000cbaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baa5e740d95fdbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.968+0000 7f42effff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.969+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430009af30 con 0x56479babec00 2023-10-06T21:14:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.969+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f430009f080 con 0x56479babec00 2023-10-06T21:14:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.969+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56479bab2d50 con 0x56479babec00 2023-10-06T21:14:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43000a3230 con 0x56479babec00 2023-10-06T21:14:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f432ab7cb80 10 client.15882 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56479b826c00 con 0x56479babec00 2023-10-06T21:14:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.971+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43000ba020 con 0x56479babec00 2023-10-06T21:14:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.971+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f430009a540 con 0x56479babec00 2023-10-06T21:14:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 msgr2=0x7f24080a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 0x7f24080a3b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f242006a4d0 tx=0x7f241800d040 comp rx=0 tx=0).stop 2023-10-06T21:14:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 msgr2=0x7f24201a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.970+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 0x7f24201a5e60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f241000e9f0 tx=0x7f241000ed00 comp rx=0 tx=0).stop 2023-10-06T21:14:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.971+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 shutdown_connections 2023-10-06T21:14:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f24080a16b0 0x7f24080a3b70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f242011a690 0x7f24201aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24201083f0 0x7f24201a5e60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 --2- 172.21.15.26:0/2790261102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f242010f640 0x7f24201a63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 >> 172.21.15.26:0/2790261102 conn(0x7f242006d1a0 msgr2=0x7f2420118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 20 client.15882 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bad9c60 0x56479badc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 4 client.15882 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56479ba0bdb0 con 0x56479bad9c60 2023-10-06T21:14:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 shutdown_connections 2023-10-06T21:14:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:22.972+0000 7f2424dff700 1 -- 172.21.15.26:0/2790261102 wait complete. 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.973+0000 7f4307fff700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bad9c60 0x56479badc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.974+0000 7f4307fff700 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bad9c60 0x56479badc080 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a0e88263800e23 server_cookie=8d07c5e2be84772d in_seq=0 out_seq=0 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.974+0000 7f42effff700 10 client.15882 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.974+0000 7f42effff700 1 -- 172.21.15.26:0/1906660057 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56479ba0bdb0 con 0x56479bad9c60 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.974+0000 7f42effff700 10 client.15882 handle_command_reply: tid=0 2023-10-06T21:14:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.974+0000 7f432ab7cb80 1 client.15882 shutdown 2023-10-06T21:14:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 msgr2=0x56479bac71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bac71f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f43000cb560 tx=0x7f43000cbaf0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 shutdown_connections 2023-10-06T21:14:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bad9c60 0x56479badc080 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bac71f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1906660057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babf520 0x56479bac7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 >> 172.21.15.26:0/1906660057 conn(0x56479ba34200 msgr2=0x56479babac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.975+0000 7f42effff700 0 client.15882 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.976+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 shutdown_connections 2023-10-06T21:14:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.976+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1906660057 wait complete. 2023-10-06T21:14:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:22.977+0000 7f432ab7cb80 20 client.15882 trim_cache size 0 max 16384 2023-10-06T21:14:23.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.003+0000 7f432ab7cb80 1 Processor -- start 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 -- start start 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479bad0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479b8938c0 con 0x56479babf520 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479bad1510 con 0x56479baca3d0 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479bac5810 con 0x56479babec00 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.004+0000 7f4307fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f4307fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58940/0 (socket says 172.21.15.26:58940) 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f4307fff700 1 -- 172.21.15.26:0/278812523 learned_addr learned my addr 172.21.15.26:0/278812523 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f4314f9b700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479bad0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 -- 172.21.15.26:0/278812523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 msgr2=0x56479bad0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479bad0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 -- 172.21.15.26:0/278812523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 msgr2=0x56479bac1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 -- 172.21.15.26:0/278812523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56479bac6d60 con 0x56479baca3d0 2023-10-06T21:14:23.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f4307fff700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.005+0000 7f431579c700 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac2530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f430801c040 tx=0x7f430802b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697872a69f7e0c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.006+0000 7f42f77fe700 1 -- 172.21.15.26:0/278812523 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011cd0 con 0x56479baca3d0 2023-10-06T21:14:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.006+0000 7f42f77fe700 1 -- 172.21.15.26:0/278812523 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4308004870 con 0x56479baca3d0 2023-10-06T21:14:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.006+0000 7f42f77fe700 1 -- 172.21.15.26:0/278812523 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080297d0 con 0x56479baca3d0 2023-10-06T21:14:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.007+0000 7f432ab7cb80 1 -- 172.21.15.26:0/278812523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 msgr2=0x56479bac2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.007+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac2530 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f430801c040 tx=0x7f430802b000 comp rx=0 tx=0).stop 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.008+0000 7f432ab7cb80 1 -- 172.21.15.26:0/278812523 shutdown_connections 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.008+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479baca3d0 0x56479bac2530 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.008+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479babec00 0x56479bad0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.008+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/278812523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babf520 0x56479bac1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.008+0000 7f432ab7cb80 1 -- 172.21.15.26:0/278812523 >> 172.21.15.26:0/278812523 conn(0x56479ba34200 msgr2=0x56479bad5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.009+0000 7f432ab7cb80 1 -- 172.21.15.26:0/278812523 shutdown_connections 2023-10-06T21:14:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.010+0000 7f432ab7cb80 1 -- 172.21.15.26:0/278812523 wait complete. 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.010+0000 7f432ab7cb80 1 -- start start 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.010+0000 7f432ab7cb80 1 Processor -- start 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bacef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479baca3d0 0x56479bacf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479bad1270 con 0x56479babec00 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479bb3da60 con 0x56479babf520 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56479bb3dba0 con 0x56479baca3d0 2023-10-06T21:14:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.011+0000 7f432ab7cb80 10 client.0 _ll_register_callbacks cb 0x7f430009ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f431579c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479baca3d0 0x56479bacf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38996/0 (socket says 172.21.15.26:38996) 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 -- 172.21.15.26:0/1063860190 learned_addr learned my addr 172.21.15.26:0/1063860190 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 -- 172.21.15.26:0/1063860190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479baca3d0 msgr2=0x56479bacf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479baca3d0 0x56479bacf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 -- 172.21.15.26:0/1063860190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 msgr2=0x56479bacef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.012+0000 7f4307fff700 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bacef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.013+0000 7f4307fff700 1 -- 172.21.15.26:0/1063860190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c007e30 con 0x56479babf520 2023-10-06T21:14:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.013+0000 7f4307fff700 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56479bac88a0 tx=0x7f430c009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0d28d5c1b5cce9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.013+0000 7f42effff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c01d040 con 0x56479babf520 2023-10-06T21:14:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f430c01d380 con 0x56479babf520 2023-10-06T21:14:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4308011e30 con 0x56479babf520 2023-10-06T21:14:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f430c01d540 con 0x56479babf520 2023-10-06T21:14:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f432ab7cb80 10 client.26048 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.016+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56479b83dd80 con 0x56479babf520 2023-10-06T21:14:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4308011e30 con 0x56479babf520 2023-10-06T21:14:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f430c026480 con 0x56479babf520 2023-10-06T21:14:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 10 client.26048 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 20 client.26048 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bb3f440 0x56479bb41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 4 client.26048 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:14:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.017+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56479ba0bad0 con 0x56479bb3f440 2023-10-06T21:14:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.018+0000 7f4314f9b700 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bb3f440 0x56479bb41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.018+0000 7f4314f9b700 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bb3f440 0x56479bb41860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50099a218ca70ae1 server_cookie=e6877bea77968fe3 in_seq=0 out_seq=0 2023-10-06T21:14:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.018+0000 7f42effff700 10 client.26048 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.032+0000 7f42effff700 1 -- 172.21.15.26:0/1063860190 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56479ba0bad0 con 0x56479bb3f440 2023-10-06T21:14:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.032+0000 7f42effff700 10 client.26048 handle_command_reply: tid=0 2023-10-06T21:14:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.032+0000 7f432ab7cb80 1 client.26048 shutdown 2023-10-06T21:14:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.033+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 msgr2=0x56479bacf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.033+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56479bac88a0 tx=0x7f430c009040 comp rx=0 tx=0).stop 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 shutdown_connections 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56479baca3d0 0x56479bacf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56479babec00 0x56479bacef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56479bb3f440 0x56479bb41860 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 --2- 172.21.15.26:0/1063860190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56479babf520 0x56479bacf440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 >> 172.21.15.26:0/1063860190 conn(0x56479ba34200 msgr2=0x56479bac2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f42effff700 0 client.26048 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 shutdown_connections 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.034+0000 7f432ab7cb80 1 -- 172.21.15.26:0/1063860190 wait complete. 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.035+0000 7f432ab7cb80 20 client.26048 trim_cache size 0 max 16384 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:14:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:14:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.036+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 msgr2=0x7f42f80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.036+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 0x7f42f80a3bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4308009950 tx=0x7f4308009820 comp rx=0 tx=0).stop 2023-10-06T21:14:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.036+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 msgr2=0x7f431019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.036+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 0x7f431019d4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43000098e0 tx=0x7f430000f690 comp rx=0 tx=0).stop 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 shutdown_connections 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f42f80a1710 0x7f42f80a3bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4310105e20 0x7f43101a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4310068810 0x7f431019d4b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 --2- 172.21.15.26:0/2968045780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43101038b0 0x7f431019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.037+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 >> 172.21.15.26:0/2968045780 conn(0x7f4310076b70 msgr2=0x7f43100fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.038+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 shutdown_connections 2023-10-06T21:14:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:23.038+0000 7f43171ff700 1 -- 172.21.15.26:0/2968045780 wait complete. 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 -- 172.21.15.26:0/3120432123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff288108400 msgr2=0x7ff28810ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 --2- 172.21.15.26:0/3120432123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff288108400 0x7ff28810ff80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff280009990 tx=0x7ff280009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 -- 172.21.15.26:0/3120432123 shutdown_connections 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 --2- 172.21.15.26:0/3120432123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff288108400 0x7ff28810ff80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 --2- 172.21.15.26:0/3120432123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 --2- 172.21.15.26:0/3120432123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288103c90 0x7ff288107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 -- 172.21.15.26:0/3120432123 >> 172.21.15.26:0/3120432123 conn(0x7ff2880feb60 msgr2=0x7ff288100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 -- 172.21.15.26:0/3120432123 shutdown_connections 2023-10-06T21:14:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.363+0000 7ff28dbff700 1 -- 172.21.15.26:0/3120432123 wait complete. 2023-10-06T21:14:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.364+0000 7ff28dbff700 1 Processor -- start 2023-10-06T21:14:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 -- start start 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288108400 0x7ff288199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 0x7ff28819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819e600 con 0x7ff28819db80 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819e770 con 0x7ff2881032e0 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff28dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28819e930 con 0x7ff288108400 2023-10-06T21:14:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 0x7ff28819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39028/0 (socket says 172.21.15.26:39028) 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.365+0000 7ff2877fe700 1 -- 172.21.15.26:0/281520786 learned_addr learned my addr 172.21.15.26:0/281520786 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 -- 172.21.15.26:0/281520786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288108400 msgr2=0x7ff288199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288108400 0x7ff288199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 -- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 msgr2=0x7ff28819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 0x7ff28819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 -- 172.21.15.26:0/281520786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800095f0 con 0x7ff2881032e0 2023-10-06T21:14:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff287fff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 0x7ff28819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.366+0000 7ff2877fe700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d950 tx=0x7ff27800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.367+0000 7ff2857fa700 1 -- 172.21.15.26:0/281520786 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278009940 con 0x7ff2881032e0 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.367+0000 7ff2857fa700 1 -- 172.21.15.26:0/281520786 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff278010460 con 0x7ff2881032e0 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.367+0000 7ff2857fa700 1 -- 172.21.15.26:0/281520786 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800f620 con 0x7ff2881032e0 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.367+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2881a5e90 con 0x7ff2881032e0 2023-10-06T21:14:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.367+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a62c0 con 0x7ff2881032e0 2023-10-06T21:14:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.372+0000 7ff2857fa700 1 -- 172.21.15.26:0/281520786 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff2780108d0 con 0x7ff2881032e0 2023-10-06T21:14:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.374+0000 7ff2a150fb80 1 Processor -- start 2023-10-06T21:14:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.374+0000 7ff2a150fb80 1 -- start start 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.374+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a7387acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a73885b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73888490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a737db460 con 0x563a7387a8d0 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a737ccc20 con 0x563a7387b1f0 2023-10-06T21:14:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a737ca540 con 0x563a738860a0 2023-10-06T21:14:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a7387acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.375+0000 7ff287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a7387acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58958/0 (socket says 172.21.15.26:58958) 2023-10-06T21:14:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a73885b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff287fff700 1 -- 172.21.15.26:0/3765781340 learned_addr learned my addr 172.21.15.26:0/3765781340 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff27effd700 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73888490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 -- 172.21.15.26:0/3765781340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 msgr2=0x563a73888490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73888490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 -- 172.21.15.26:0/3765781340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 msgr2=0x563a7387acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a7387acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.376+0000 7ff2877fe700 1 -- 172.21.15.26:0/3765781340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a735f0de0 con 0x563a7387b1f0 2023-10-06T21:14:23.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.378+0000 7ff2857fa700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 0x7ff2740a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.378+0000 7ff2857fa700 1 -- 172.21.15.26:0/281520786 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff2780ccf30 con 0x7ff2881032e0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.379+0000 7ff287fff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 0x7ff2740a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.379+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a73885b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ca630 tx=0x7ff2780d53c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6bc8f772fee159e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.380+0000 7ff287fff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 0x7ff2740a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff28000bc60 tx=0x7ff28000b540 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.380+0000 7ff27dffb700 1 -- 172.21.15.26:0/3765781340 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278096200 con 0x563a7387b1f0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.380+0000 7ff27dffb700 1 -- 172.21.15.26:0/3765781340 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff278099c90 con 0x563a7387b1f0 2023-10-06T21:14:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.380+0000 7ff27dffb700 1 -- 172.21.15.26:0/3765781340 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278099ec0 con 0x563a7387b1f0 2023-10-06T21:14:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3765781340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 msgr2=0x563a73885b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a73885b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ca630 tx=0x7ff2780d53c0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3765781340 shutdown_connections 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73888490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a7387acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3765781340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a73885b60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.381+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3765781340 >> 172.21.15.26:0/3765781340 conn(0x563a737e7260 msgr2=0x563a7386e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.382+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3765781340 shutdown_connections 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.382+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3765781340 wait complete. 2023-10-06T21:14:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.382+0000 7ff2a150fb80 1 -- start start 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.382+0000 7ff2a150fb80 1 Processor -- start 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a73885b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a738860a0 0x563a738834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a737c5e30 con 0x563a7387a8d0 2023-10-06T21:14:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a73762710 con 0x563a738860a0 2023-10-06T21:14:23.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a737623b0 con 0x563a7387b1f0 2023-10-06T21:14:23.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff2a150fb80 10 client.0 _ll_register_callbacks cb 0x563a735bf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:23.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff27effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff27effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34016/0 (socket says 172.21.15.26:34016) 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.383+0000 7ff27effd700 1 -- 172.21.15.26:0/3135171522 learned_addr learned my addr 172.21.15.26:0/3135171522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.384+0000 7ff27effd700 1 -- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a738860a0 msgr2=0x563a738834c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.384+0000 7ff27effd700 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a738860a0 0x563a738834c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.384+0000 7ff27effd700 1 -- 172.21.15.26:0/3135171522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 msgr2=0x563a73885b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.384+0000 7ff27effd700 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a73885b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.384+0000 7ff27effd700 1 -- 172.21.15.26:0/3135171522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a62c0 con 0x563a7387b1f0 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.385+0000 7ff27effd700 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff27000aad0 tx=0x7ff27000ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f37d7d3b99d7f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.385+0000 7ff266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:14:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.385+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270010040 con 0x563a7387b1f0 2023-10-06T21:14:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.385+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff27000ea80 con 0x563a7387b1f0 2023-10-06T21:14:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.385+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a73865f10 con 0x563a7387b1f0 2023-10-06T21:14:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.386+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27000ec40 con 0x563a7387b1f0 2023-10-06T21:14:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.387+0000 7ff2a150fb80 10 client.26053 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.387+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563a735d9c60 con 0x563a7387b1f0 2023-10-06T21:14:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.387+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a73865f10 con 0x563a7387b1f0 2023-10-06T21:14:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.388+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff270008030 con 0x563a7387b1f0 2023-10-06T21:14:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 10 client.26053 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 20 client.26053 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a73895bb0 0x563a73897fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 4 client.26053 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.391+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a737bee10 con 0x563a73895bb0 2023-10-06T21:14:23.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.392+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a73895bb0 0x563a73897fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.392+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a73895bb0 0x563a73897fd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f525f195c79b64f5 server_cookie=f6ddb85dd1733488 in_seq=0 out_seq=0 2023-10-06T21:14:23.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.392+0000 7ff266ffd700 10 client.26053 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.393+0000 7ff266ffd700 1 -- 172.21.15.26:0/3135171522 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a737bee10 con 0x563a73895bb0 2023-10-06T21:14:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.393+0000 7ff266ffd700 10 client.26053 handle_command_reply: tid=0 2023-10-06T21:14:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.393+0000 7ff2a150fb80 1 client.26053 shutdown 2023-10-06T21:14:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 msgr2=0x563a73882d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff27000aad0 tx=0x7ff27000ae90 comp rx=0 tx=0).stop 2023-10-06T21:14:23.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 shutdown_connections 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a73895bb0 0x563a73897fd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a738860a0 0x563a738834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387a8d0 0x563a73885b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3135171522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387b1f0 0x563a73882d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 >> 172.21.15.26:0/3135171522 conn(0x563a737e7260 msgr2=0x563a7386e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff266ffd700 0 client.26053 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 shutdown_connections 2023-10-06T21:14:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.395+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3135171522 wait complete. 2023-10-06T21:14:23.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.396+0000 7ff2a150fb80 20 client.26053 trim_cache size 0 max 16384 2023-10-06T21:14:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.418+0000 7ff2a150fb80 1 Processor -- start 2023-10-06T21:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.418+0000 7ff2a150fb80 1 -- start start 2023-10-06T21:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387a8d0 0x563a7387e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387b1f0 0x563a7387e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a73646920 con 0x563a7387b1f0 2023-10-06T21:14:23.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a738931e0 con 0x563a7387a8d0 2023-10-06T21:14:23.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.419+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a73881300 con 0x563a738860a0 2023-10-06T21:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34030/0 (socket says 172.21.15.26:34030) 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 -- 172.21.15.26:0/3090289518 learned_addr learned my addr 172.21.15.26:0/3090289518 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff27effd700 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387b1f0 0x563a7387e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff2877fe700 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387a8d0 0x563a7387e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 -- 172.21.15.26:0/3090289518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387a8d0 msgr2=0x563a7387e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387a8d0 0x563a7387e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 -- 172.21.15.26:0/3090289518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387b1f0 msgr2=0x563a7387e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387b1f0 0x563a7387e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.420+0000 7ff287fff700 1 -- 172.21.15.26:0/3090289518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a73617250 con 0x563a738860a0 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.421+0000 7ff287fff700 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff28000bb50 tx=0x7ff280023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f20b77bbd5bec9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.421+0000 7ff27e7fc700 1 -- 172.21.15.26:0/3090289518 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280004780 con 0x563a738860a0 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.421+0000 7ff27e7fc700 1 -- 172.21.15.26:0/3090289518 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff28001e040 con 0x563a738860a0 2023-10-06T21:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.421+0000 7ff27e7fc700 1 -- 172.21.15.26:0/3090289518 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800040a0 con 0x563a738860a0 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3090289518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 msgr2=0x563a73882560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff28000bb50 tx=0x7ff280023000 comp rx=0 tx=0).stop 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3090289518 shutdown_connections 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a738860a0 0x563a73882560 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387a8d0 0x563a7387e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/3090289518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a7387b1f0 0x563a7387e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3090289518 >> 172.21.15.26:0/3090289518 conn(0x563a737e7260 msgr2=0x563a73885730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3090289518 shutdown_connections 2023-10-06T21:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.422+0000 7ff2a150fb80 1 -- 172.21.15.26:0/3090289518 wait complete. 2023-10-06T21:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.423+0000 7ff2a150fb80 1 -- start start 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.423+0000 7ff2a150fb80 1 Processor -- start 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.424+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387a8d0 0x563a7388adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.424+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff2a150fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a738860a0 0x563a738e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a738e2f80 con 0x563a738860a0 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a738e30f0 con 0x563a7387b1f0 2023-10-06T21:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff2a150fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a738e32b0 con 0x563a7387a8d0 2023-10-06T21:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff2a150fb80 10 client.0 _ll_register_callbacks cb 0x7ff27000a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.425+0000 7ff27effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39072/0 (socket says 172.21.15.26:39072) 2023-10-06T21:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 -- 172.21.15.26:0/2986488332 learned_addr learned my addr 172.21.15.26:0/2986488332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff2877fe700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387a8d0 0x563a7388adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff287fff700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a738860a0 0x563a738e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 -- 172.21.15.26:0/2986488332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387a8d0 msgr2=0x563a7388adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387a8d0 0x563a7388adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 -- 172.21.15.26:0/2986488332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a738860a0 msgr2=0x563a738e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a738860a0 0x563a738e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.426+0000 7ff27effd700 1 -- 172.21.15.26:0/2986488332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881107f0 con 0x563a7387b1f0 2023-10-06T21:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.427+0000 7ff27effd700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff27001d490 tx=0x7ff270009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4a9aba86bfef18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.427+0000 7ff266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.427+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270010040 con 0x563a7387b1f0 2023-10-06T21:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.427+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff270005e40 con 0x563a7387b1f0 2023-10-06T21:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.427+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff278099c90 con 0x563a7387b1f0 2023-10-06T21:14:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27001eb60 con 0x563a7387b1f0 2023-10-06T21:14:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff2a150fb80 10 client.26063 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563a735ef2b0 con 0x563a7387b1f0 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff278099c90 con 0x563a7387b1f0 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff27000abb0 con 0x563a7387b1f0 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff2a150fb80 10 client.26063 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.430+0000 7ff2a150fb80 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2a150fb80 20 client.26063 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a738eab10 0x563a738ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2a150fb80 4 client.26063 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a737beb30 con 0x563a738eab10 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2877fe700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a738eab10 0x563a738ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff266ffd700 10 client.26063 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.431+0000 7ff2877fe700 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a738eab10 0x563a738ecf30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae303495050aeb3 server_cookie=937d2ef9313b61ce in_seq=0 out_seq=0 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff266ffd700 1 -- 172.21.15.26:0/2986488332 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a737beb30 con 0x563a738eab10 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff266ffd700 10 client.26063 handle_command_reply: tid=0 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 client.26063 shutdown 2023-10-06T21:14:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 msgr2=0x563a7388b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff27001d490 tx=0x7ff270009900 comp rx=0 tx=0).stop 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 shutdown_connections 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563a738860a0 0x563a738e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563a7387a8d0 0x563a7388adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563a738eab10 0x563a738ecf30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 --2- 172.21.15.26:0/2986488332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a7387b1f0 0x563a7388b300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 >> 172.21.15.26:0/2986488332 conn(0x563a737e7260 msgr2=0x563a7387c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff266ffd700 0 client.26063 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:23.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 shutdown_connections 2023-10-06T21:14:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.432+0000 7ff2a150fb80 1 -- 172.21.15.26:0/2986488332 wait complete. 2023-10-06T21:14:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.434+0000 7ff2a150fb80 20 client.26063 trim_cache size 0 max 16384 2023-10-06T21:14:23.439 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:14:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.435+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 msgr2=0x7ff2740a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.435+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 0x7ff2740a3b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff28000bc60 tx=0x7ff28000b540 comp rx=0 tx=0).stop 2023-10-06T21:14:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.435+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 msgr2=0x7ff2881991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.435+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d950 tx=0x7ff27800dc60 comp rx=0 tx=0).stop 2023-10-06T21:14:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 shutdown_connections 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff2740a1670 0x7ff2740a3b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff28819db80 0x7ff28819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2881032e0 0x7ff2881991e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 --2- 172.21.15.26:0/281520786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288108400 0x7ff288199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 >> 172.21.15.26:0/281520786 conn(0x7ff2880feb60 msgr2=0x7ff2880ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 shutdown_connections 2023-10-06T21:14:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:23.436+0000 7ff28dbff700 1 -- 172.21.15.26:0/281520786 wait complete. 2023-10-06T21:14:24.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:24 smithi167 ceph-mon[134494]: pgmap v126: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-10-06T21:14:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:24 smithi026 ceph-mon[98490]: pgmap v126: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-10-06T21:14:24.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:24 smithi078 ceph-mon[134896]: pgmap v126: 97 pgs: 97 active+clean; 5.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2023-10-06T21:14:24.467 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:14:24.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:14:24.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 -- 172.21.15.26:0/788886386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f442410d7a0 msgr2=0x7f442410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 --2- 172.21.15.26:0/788886386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f442410d7a0 0x7f442410fb90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f441c0098f0 tx=0x7f441c009c00 comp rx=0 tx=0).stop 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 -- 172.21.15.26:0/788886386 shutdown_connections 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 --2- 172.21.15.26:0/788886386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f442410d7a0 0x7f442410fb90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 --2- 172.21.15.26:0/788886386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44240ffbf0 0x7f44240fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 --2- 172.21.15.26:0/788886386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44241005a0 0x7f442410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.842+0000 7f44289ff700 1 -- 172.21.15.26:0/788886386 >> 172.21.15.26:0/788886386 conn(0x7f44240fb830 msgr2=0x7f44240fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.843+0000 7f44289ff700 1 -- 172.21.15.26:0/788886386 shutdown_connections 2023-10-06T21:14:24.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.843+0000 7f44289ff700 1 -- 172.21.15.26:0/788886386 wait complete. 2023-10-06T21:14:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.844+0000 7f44289ff700 1 Processor -- start 2023-10-06T21:14:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.845+0000 7f44289ff700 1 -- start start 2023-10-06T21:14:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 0x7f44241997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f442410d7a0 0x7f442419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e270 con 0x7f442410d7a0 2023-10-06T21:14:24.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e3e0 con 0x7f44241005a0 2023-10-06T21:14:24.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.846+0000 7f44289ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442419e5a0 con 0x7f44240ffbf0 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.847+0000 7f442259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.847+0000 7f442259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34058/0 (socket says 172.21.15.26:34058) 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.847+0000 7f442259c700 1 -- 172.21.15.26:0/767928784 learned_addr learned my addr 172.21.15.26:0/767928784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.847+0000 7f4422d9d700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f442410d7a0 0x7f442419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 0x7f44241997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 -- 172.21.15.26:0/767928784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 msgr2=0x7f4424199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 -- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f442410d7a0 msgr2=0x7f442419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f442410d7a0 0x7f442419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f4419d9b700 1 -- 172.21.15.26:0/767928784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009d00 con 0x7f44241005a0 2023-10-06T21:14:24.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.848+0000 7f442259c700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:24.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f4419d9b700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 0x7f44241997d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f440c006fd0 tx=0x7f440c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f441bfff700 1 -- 172.21.15.26:0/767928784 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c011830 con 0x7f44241005a0 2023-10-06T21:14:24.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c009520 con 0x7f44241005a0 2023-10-06T21:14:24.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f441bfff700 1 -- 172.21.15.26:0/767928784 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f440c011e50 con 0x7f44241005a0 2023-10-06T21:14:24.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f441bfff700 1 -- 172.21.15.26:0/767928784 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c010e80 con 0x7f44241005a0 2023-10-06T21:14:24.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.849+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44241a5ee0 con 0x7f44241005a0 2023-10-06T21:14:24.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.856+0000 7f441bfff700 1 -- 172.21.15.26:0/767928784 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f440c010380 con 0x7f44241005a0 2023-10-06T21:14:24.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.857+0000 7f443c27db80 1 Processor -- start 2023-10-06T21:14:24.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.860+0000 7f443c27db80 1 -- start start 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78cf120 0x564ed78d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed78244e0 con 0x564ed78cf120 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed7815ca0 con 0x564ed78c3950 2023-10-06T21:14:24.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f443c27db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed78135c0 con 0x564ed78c4270 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f4419d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78cf120 0x564ed78d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.861+0000 7f442259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34070/0 (socket says 172.21.15.26:34070) 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 -- 172.21.15.26:0/566262625 learned_addr learned my addr 172.21.15.26:0/566262625 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f4422d9d700 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f441bfff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 0x7f44100a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 -- 172.21.15.26:0/566262625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 msgr2=0x564ed78c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 -- 172.21.15.26:0/566262625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78cf120 msgr2=0x564ed78d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f441bfff700 1 -- 172.21.15.26:0/767928784 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f440c0ccae0 con 0x7f44241005a0 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78cf120 0x564ed78d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f442259c700 1 -- 172.21.15.26:0/566262625 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed7639e60 con 0x564ed78c4270 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f4422d9d700 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:24.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.862+0000 7f4422d9d700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 0x7f44100a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.863+0000 7f442259c700 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4414007690 tx=0x7f44140099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542864d1ff45f7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.864+0000 7f4422d9d700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 0x7f44100a3b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f441c0098c0 tx=0x7f441c019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.864+0000 7f4418d99700 1 -- 172.21.15.26:0/566262625 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e030 con 0x564ed78c4270 2023-10-06T21:14:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.864+0000 7f4418d99700 1 -- 172.21.15.26:0/566262625 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441400f040 con 0x564ed78c4270 2023-10-06T21:14:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.864+0000 7f4418d99700 1 -- 172.21.15.26:0/566262625 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414015ce0 con 0x564ed78c4270 2023-10-06T21:14:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 -- 172.21.15.26:0/566262625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 msgr2=0x564ed78cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4414007690 tx=0x7f44140099e0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 -- 172.21.15.26:0/566262625 shutdown_connections 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78cf120 0x564ed78d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 --2- 172.21.15.26:0/566262625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cebe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 -- 172.21.15.26:0/566262625 >> 172.21.15.26:0/566262625 conn(0x564ed78302e0 msgr2=0x564ed78b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 -- 172.21.15.26:0/566262625 shutdown_connections 2023-10-06T21:14:24.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.866+0000 7f443c27db80 1 -- 172.21.15.26:0/566262625 wait complete. 2023-10-06T21:14:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.867+0000 7f443c27db80 1 -- start start 2023-10-06T21:14:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.867+0000 7f443c27db80 1 Processor -- start 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed780eeb0 con 0x564ed78c3950 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed77ab790 con 0x564ed78cf120 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed77ab430 con 0x564ed78c4270 2023-10-06T21:14:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f443c27db80 10 client.0 _ll_register_callbacks cb 0x564ed7608880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f442259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39114/0 (socket says 172.21.15.26:39114) 2023-10-06T21:14:24.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.868+0000 7f4422d9d700 1 -- 172.21.15.26:0/2979120505 learned_addr learned my addr 172.21.15.26:0/2979120505 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.869+0000 7f4419d9b700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.870+0000 7f4422d9d700 1 -- 172.21.15.26:0/2979120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 msgr2=0x564ed78cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.870+0000 7f4422d9d700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.870+0000 7f4422d9d700 1 -- 172.21.15.26:0/2979120505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 msgr2=0x564ed78cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.870+0000 7f4422d9d700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.870+0000 7f4422d9d700 1 -- 172.21.15.26:0/2979120505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4414009300 con 0x564ed78cf120 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.871+0000 7f4422d9d700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f441c016000 tx=0x7f441c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b63fed3764f3791 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.871+0000 7f4409ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.871+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c01c040 con 0x564ed78cf120 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.871+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f441c005240 con 0x564ed78cf120 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.872+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ed78aef90 con 0x564ed78cf120 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.872+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f441c005400 con 0x564ed78cf120 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.872+0000 7f443c27db80 10 client.26072 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:24.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.872+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564ed7622ce0 con 0x564ed78cf120 2023-10-06T21:14:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ed78aef90 con 0x564ed78cf120 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f441c035070 con 0x564ed78cf120 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 10 client.26072 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 10 client.26072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 20 client.26072 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed78deab0 0x564ed78e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 4 client.26072 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.873+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ed7807e90 con 0x564ed78deab0 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.874+0000 7f442259c700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed78deab0 0x564ed78e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.874+0000 7f442259c700 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed78deab0 0x564ed78e0ed0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e600e54f719db58b server_cookie=569f81672954bddf in_seq=0 out_seq=0 2023-10-06T21:14:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.874+0000 7f4409ffb700 10 client.26072 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:24.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.875+0000 7f4409ffb700 1 -- 172.21.15.26:0/2979120505 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ed7807e90 con 0x564ed78deab0 2023-10-06T21:14:24.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.875+0000 7f4409ffb700 10 client.26072 handle_command_reply: tid=0 2023-10-06T21:14:24.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.875+0000 7f443c27db80 1 client.26072 shutdown 2023-10-06T21:14:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.875+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 msgr2=0x564ed78cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.875+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f441c016000 tx=0x7f441c0058e0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 shutdown_connections 2023-10-06T21:14:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed78deab0 0x564ed78e0ed0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78cf120 0x564ed78cc900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 --2- 172.21.15.26:0/2979120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78c4270 0x564ed78cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 >> 172.21.15.26:0/2979120505 conn(0x564ed78302e0 msgr2=0x564ed78b7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f4409ffb700 0 client.26072 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:24.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.877+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 shutdown_connections 2023-10-06T21:14:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.878+0000 7f443c27db80 1 -- 172.21.15.26:0/2979120505 wait complete. 2023-10-06T21:14:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.878+0000 7f443c27db80 20 client.26072 trim_cache size 0 max 16384 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 Processor -- start 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 -- start start 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c4270 0x564ed78cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed768f9a0 con 0x564ed78c4270 2023-10-06T21:14:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed78aee60 con 0x564ed78c3950 2023-10-06T21:14:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.906+0000 7f443c27db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed78b80a0 con 0x564ed78cf120 2023-10-06T21:14:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.907+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.907+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34100/0 (socket says 172.21.15.26:34100) 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.907+0000 7f442259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.907+0000 7f4422d9d700 1 -- 172.21.15.26:0/1656867674 learned_addr learned my addr 172.21.15.26:0/1656867674 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.907+0000 7f4419d9b700 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c4270 0x564ed78cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 -- 172.21.15.26:0/1656867674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 msgr2=0x564ed78d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 -- 172.21.15.26:0/1656867674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c4270 msgr2=0x564ed78cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c4270 0x564ed78cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 -- 172.21.15.26:0/1656867674 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed76602d0 con 0x564ed78cf120 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f442259c700 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.908+0000 7f4422d9d700 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f441c005980 tx=0x7f441c00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a3cf0f625291f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.909+0000 7f441959a700 1 -- 172.21.15.26:0/1656867674 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c01c040 con 0x564ed78cf120 2023-10-06T21:14:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.909+0000 7f441959a700 1 -- 172.21.15.26:0/1656867674 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441c029040 con 0x564ed78cf120 2023-10-06T21:14:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.909+0000 7f441959a700 1 -- 172.21.15.26:0/1656867674 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c02d650 con 0x564ed78cf120 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 -- 172.21.15.26:0/1656867674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 msgr2=0x564ed78c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f441c005980 tx=0x7f441c00f670 comp rx=0 tx=0).stop 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 -- 172.21.15.26:0/1656867674 shutdown_connections 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed78c5ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c3950 0x564ed78d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 --2- 172.21.15.26:0/1656867674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c4270 0x564ed78cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 -- 172.21.15.26:0/1656867674 >> 172.21.15.26:0/1656867674 conn(0x564ed78302e0 msgr2=0x564ed78c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 -- 172.21.15.26:0/1656867674 shutdown_connections 2023-10-06T21:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.910+0000 7f443c27db80 1 -- 172.21.15.26:0/1656867674 wait complete. 2023-10-06T21:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.911+0000 7f443c27db80 1 -- start start 2023-10-06T21:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.911+0000 7f443c27db80 1 Processor -- start 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 0x564ed78d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed79328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed78d5660 con 0x564ed78c3950 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed7932e60 con 0x564ed78c4270 2023-10-06T21:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed7933020 con 0x564ed78cf120 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f443c27db80 10 client.0 _ll_register_callbacks cb 0x7f441c00fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f442259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f442259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59046/0 (socket says 172.21.15.26:59046) 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f442259c700 1 -- 172.21.15.26:0/2376908402 learned_addr learned my addr 172.21.15.26:0/2376908402 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f4422d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed79328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.912+0000 7f4419d9b700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 0x564ed78d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f442259c700 1 -- 172.21.15.26:0/2376908402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 msgr2=0x564ed79328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f442259c700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed79328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f442259c700 1 -- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 msgr2=0x564ed78d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f442259c700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 0x564ed78d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f442259c700 1 -- 172.21.15.26:0/2376908402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ed7639e60 con 0x564ed78c3950 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.913+0000 7f4419d9b700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 0x564ed78d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.914+0000 7f442259c700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4414008fd0 tx=0x7f4414009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364334f41386e57d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.914+0000 7f4409ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.914+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414025040 con 0x564ed78c3950 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.914+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4414004b50 con 0x564ed78c3950 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.915+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f441400f040 con 0x564ed78c3950 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.915+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f441400e040 con 0x564ed78c3950 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.915+0000 7f443c27db80 10 client.15915 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.915+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564ed7622ce0 con 0x564ed78c3950 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f441400f040 con 0x564ed78c3950 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4414028d10 con 0x564ed78c3950 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 10 client.15915 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 20 client.15915 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed79346b0 0x564ed7936ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 4 client.15915 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:14:24.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.916+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ed7807bb0 con 0x564ed79346b0 2023-10-06T21:14:24.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.917+0000 7f4419d9b700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed79346b0 0x564ed7936ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.919+0000 7f4419d9b700 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed79346b0 0x564ed7936ad0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1022713e49e809c5 server_cookie=2c0602804d04e974 in_seq=0 out_seq=0 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.919+0000 7f4409ffb700 10 client.15915 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.921+0000 7f4409ffb700 1 -- 172.21.15.26:0/2376908402 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ed7807bb0 con 0x564ed79346b0 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.921+0000 7f4409ffb700 10 client.15915 handle_command_reply: tid=0 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.921+0000 7f443c27db80 1 client.15915 shutdown 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.921+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 msgr2=0x564ed78d4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.921+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4414008fd0 tx=0x7f4414009be0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 shutdown_connections 2023-10-06T21:14:24.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ed78cf120 0x564ed79328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ed78c3950 0x564ed78d4990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ed79346b0 0x564ed7936ad0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 --2- 172.21.15.26:0/2376908402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ed78c4270 0x564ed78d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 >> 172.21.15.26:0/2376908402 conn(0x564ed78302e0 msgr2=0x564ed78cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f4409ffb700 0 client.15915 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 shutdown_connections 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 1 -- 172.21.15.26:0/2376908402 wait complete. 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.922+0000 7f443c27db80 20 client.15915 trim_cache size 0 max 16384 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:14:24.925 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:14:24.926 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "840be7b8-0467-4f3d-8ff0-c22ef96e3429", 2023-10-06T21:14:24.926 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:14:24.926 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:14:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.925+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 msgr2=0x7f44100a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.925+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 0x7f44100a3b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f441c0098c0 tx=0x7f441c019040 comp rx=0 tx=0).stop 2023-10-06T21:14:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.925+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 msgr2=0x7f44241997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.925+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 0x7f44241997d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f440c006fd0 tx=0x7f440c00b7e0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 shutdown_connections 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f44100a16b0 0x7f44100a3b70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f442410d7a0 0x7f442419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44240ffbf0 0x7f4424199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 --2- 172.21.15.26:0/767928784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44241005a0 0x7f44241997d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.926+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 >> 172.21.15.26:0/767928784 conn(0x7f44240fb830 msgr2=0x7f442410f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.928+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 shutdown_connections 2023-10-06T21:14:24.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:24.928+0000 7f44289ff700 1 -- 172.21.15.26:0/767928784 wait complete. 2023-10-06T21:14:24.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:14:25.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:25 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:14:25.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:25 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:14:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.339+0000 7fb463bff700 1 -- 172.21.15.26:0/1125044924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c0ff080 msgr2=0x7fb45c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.339+0000 7fb463bff700 1 --2- 172.21.15.26:0/1125044924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c0ff080 0x7fb45c0ff460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb458009990 tx=0x7fb458009ca0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.341+0000 7fb463bff700 1 -- 172.21.15.26:0/1125044924 shutdown_connections 2023-10-06T21:14:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.341+0000 7fb463bff700 1 --2- 172.21.15.26:0/1125044924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c104220 0x7fb45c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.341+0000 7fb463bff700 1 --2- 172.21.15.26:0/1125044924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c0ff080 0x7fb45c0ff460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.341+0000 7fb463bff700 1 --2- 172.21.15.26:0/1125044924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.341+0000 7fb463bff700 1 -- 172.21.15.26:0/1125044924 >> 172.21.15.26:0/1125044924 conn(0x7fb45c076ed0 msgr2=0x7fb45c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.342+0000 7fb463bff700 1 -- 172.21.15.26:0/1125044924 shutdown_connections 2023-10-06T21:14:25.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.343+0000 7fb463bff700 1 -- 172.21.15.26:0/1125044924 wait complete. 2023-10-06T21:14:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.344+0000 7fb463bff700 1 Processor -- start 2023-10-06T21:14:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.346+0000 7fb463bff700 1 -- start start 2023-10-06T21:14:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c104220 0x7fb45c1b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1132d0 con 0x7fb45c0ff9a0 2023-10-06T21:14:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c113410 con 0x7fb45c104220 2023-10-06T21:14:25.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb463bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1b2490 con 0x7fb45c0ff080 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb46199b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb46199b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34130/0 (socket says 172.21.15.26:34130) 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb46199b700 1 -- 172.21.15.26:0/1657204601 learned_addr learned my addr 172.21.15.26:0/1657204601 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.347+0000 7fb46219c700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c104220 0x7fb45c1b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 -- 172.21.15.26:0/1657204601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c104220 msgr2=0x7fb45c1b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c104220 0x7fb45c1b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46119a700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 -- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 msgr2=0x7fb45c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 -- 172.21.15.26:0/1657204601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580095f0 con 0x7fb45c0ff080 2023-10-06T21:14:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46119a700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.348+0000 7fb46199b700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb458003b90 tx=0x7fb4580037c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.349+0000 7fb452ffd700 1 -- 172.21.15.26:0/1657204601 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45801d070 con 0x7fb45c0ff080 2023-10-06T21:14:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.349+0000 7fb452ffd700 1 -- 172.21.15.26:0/1657204601 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb458005580 con 0x7fb45c0ff080 2023-10-06T21:14:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.349+0000 7fb452ffd700 1 -- 172.21.15.26:0/1657204601 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800fe90 con 0x7fb45c0ff080 2023-10-06T21:14:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.350+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c1b2630 con 0x7fb45c0ff080 2023-10-06T21:14:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.350+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c1b2a60 con 0x7fb45c0ff080 2023-10-06T21:14:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.355+0000 7fb452ffd700 1 -- 172.21.15.26:0/1657204601 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb458022420 con 0x7fb45c0ff080 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.358+0000 7fb452ffd700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 0x7fb4480a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.358+0000 7fb477470b80 1 Processor -- start 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb452ffd700 1 -- 172.21.15.26:0/1657204601 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb4580ce2b0 con 0x7fb45c0ff080 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 -- start start 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb46119a700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 0x7fb4480a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 0x5622dbac0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dba0b460 con 0x5622dbaa29e0 2023-10-06T21:14:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db9fcc20 con 0x5622dbaa20c0 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.359+0000 7fb477470b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db9fa540 con 0x5622dbabe8a0 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46199b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46199b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39156/0 (socket says 172.21.15.26:39156) 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46199b700 1 -- 172.21.15.26:0/3306468878 learned_addr learned my addr 172.21.15.26:0/3306468878 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46219c700 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 0x5622dbac0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46219c700 1 -- 172.21.15.26:0/3306468878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 msgr2=0x5622dbaa24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.360+0000 7fb46219c700 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46219c700 1 -- 172.21.15.26:0/3306468878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 msgr2=0x5622dbabe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46219c700 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46219c700 1 -- 172.21.15.26:0/3306468878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622db820de0 con 0x5622dbabe8a0 2023-10-06T21:14:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46219c700 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 0x5622dbac0c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb454009f90 tx=0x7fb454009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf1238d62e76765 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46199b700 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb46119a700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 0x7fb4480a3cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005fd0 tx=0x7fb44c00e2f0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb447fff700 1 -- 172.21.15.26:0/3306468878 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454010040 con 0x5622dbabe8a0 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb447fff700 1 -- 172.21.15.26:0/3306468878 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb454014ae0 con 0x5622dbabe8a0 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.361+0000 7fb447fff700 1 -- 172.21.15.26:0/3306468878 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400ab50 con 0x5622dbabe8a0 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.362+0000 7fb477470b80 1 -- 172.21.15.26:0/3306468878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 msgr2=0x5622dbac0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.362+0000 7fb477470b80 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 0x5622dbac0c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb454009f90 tx=0x7fb454009630 comp rx=0 tx=0).stop 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.363+0000 7fb477470b80 1 -- 172.21.15.26:0/3306468878 shutdown_connections 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.363+0000 7fb477470b80 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbabe8a0 0x5622dbac0c90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.363+0000 7fb477470b80 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbaa24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.363+0000 7fb477470b80 1 --2- 172.21.15.26:0/3306468878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.363+0000 7fb477470b80 1 -- 172.21.15.26:0/3306468878 >> 172.21.15.26:0/3306468878 conn(0x5622dba17260 msgr2=0x5622dba9e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.364+0000 7fb477470b80 1 -- 172.21.15.26:0/3306468878 shutdown_connections 2023-10-06T21:14:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.364+0000 7fb477470b80 1 -- 172.21.15.26:0/3306468878 wait complete. 2023-10-06T21:14:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.365+0000 7fb477470b80 1 -- start start 2023-10-06T21:14:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.365+0000 7fb477470b80 1 Processor -- start 2023-10-06T21:14:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 0x5622dbabc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db9f5e30 con 0x5622dbaa29e0 2023-10-06T21:14:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db992710 con 0x5622dbabe8a0 2023-10-06T21:14:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db9923b0 con 0x5622dbaa20c0 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.366+0000 7fb477470b80 10 client.0 _ll_register_callbacks cb 0x5622db7ef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46219c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 0x5622dbabc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46219c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 0x5622dbabc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39172/0 (socket says 172.21.15.26:39172) 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46219c700 1 -- 172.21.15.26:0/1716601497 learned_addr learned my addr 172.21.15.26:0/1716601497 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46199b700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 -- 172.21.15.26:0/1716601497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 msgr2=0x5622dbabb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 -- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 msgr2=0x5622dbabc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 0x5622dbabc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.367+0000 7fb46119a700 1 -- 172.21.15.26:0/1716601497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c1b2a60 con 0x5622dbaa29e0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.368+0000 7fb46119a700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabbb40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb45c1a0860 tx=0x7fb44c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449e55a75b9a8441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.368+0000 7fb4457fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.368+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c023070 con 0x5622dbaa29e0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.368+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622dba95f10 con 0x5622dbaa29e0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.368+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb44c004020 con 0x5622dbaa29e0 2023-10-06T21:14:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb44c0041e0 con 0x5622dbaa29e0 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 10 client.15921 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fb454009bc0 con 0x5622dbaa29e0 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622dba95f10 con 0x5622dbaa29e0 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb44c026020 con 0x5622dbaa29e0 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 20 client.15921 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbace440 0x5622dbad0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 4 client.15921 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.369+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622db9eee10 con 0x5622dbace440 2023-10-06T21:14:25.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:25 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:14:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.385+0000 7fb46199b700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbace440 0x5622dbad0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.388+0000 7fb46199b700 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbace440 0x5622dbad0860 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1227b74f9248ffd server_cookie=7cfcc73d7b07d8ba in_seq=0 out_seq=0 2023-10-06T21:14:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.388+0000 7fb4457fa700 10 client.15921 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.391+0000 7fb4457fa700 1 -- 172.21.15.26:0/1716601497 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5622db9eee10 con 0x5622dbace440 2023-10-06T21:14:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.391+0000 7fb4457fa700 10 client.15921 handle_command_reply: tid=0 2023-10-06T21:14:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.391+0000 7fb477470b80 1 client.15921 shutdown 2023-10-06T21:14:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.392+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 msgr2=0x5622dbabbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.392+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabbb40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb45c1a0860 tx=0x7fb44c004750 comp rx=0 tx=0).stop 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 shutdown_connections 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbace440 0x5622dbad0860 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbabe8a0 0x5622dbabc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb4457fa700 0 client.15921 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 --2- 172.21.15.26:0/1716601497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbaa29e0 0x5622dbabbb40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 >> 172.21.15.26:0/1716601497 conn(0x5622dba17260 msgr2=0x5622dba9dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 shutdown_connections 2023-10-06T21:14:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 1 -- 172.21.15.26:0/1716601497 wait complete. 2023-10-06T21:14:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.393+0000 7fb477470b80 20 client.15921 trim_cache size 0 max 16384 2023-10-06T21:14:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.424+0000 7fb477470b80 1 Processor -- start 2023-10-06T21:14:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.428+0000 7fb477470b80 1 -- start start 2023-10-06T21:14:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbaba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622db876920 con 0x5622dbabe8a0 2023-10-06T21:14:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dba95d50 con 0x5622dbaa29e0 2023-10-06T21:14:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb477470b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dbab71d0 con 0x5622dbaa20c0 2023-10-06T21:14:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb46119a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb46199b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb46119a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39174/0 (socket says 172.21.15.26:39174) 2023-10-06T21:14:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.429+0000 7fb46119a700 1 -- 172.21.15.26:0/3527643164 learned_addr learned my addr 172.21.15.26:0/3527643164 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46119a700 1 -- 172.21.15.26:0/3527643164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 msgr2=0x5622dbabbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46119a700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46219c700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbaba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46119a700 1 -- 172.21.15.26:0/3527643164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 msgr2=0x5622dbaba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46119a700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbaba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46119a700 1 -- 172.21.15.26:0/3527643164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622db809c60 con 0x5622dbaa29e0 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46199b700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:14:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.430+0000 7fb46219c700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbaba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.431+0000 7fb46119a700 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb44c011a90 tx=0x7fb44c011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bade18d6aaec0005 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.431+0000 7fb450bff700 1 -- 172.21.15.26:0/3527643164 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c023040 con 0x5622dbaa29e0 2023-10-06T21:14:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.431+0000 7fb450bff700 1 -- 172.21.15.26:0/3527643164 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb44c019040 con 0x5622dbaa29e0 2023-10-06T21:14:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.431+0000 7fb450bff700 1 -- 172.21.15.26:0/3527643164 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c005030 con 0x5622dbaa29e0 2023-10-06T21:14:25.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 -- 172.21.15.26:0/3527643164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 msgr2=0x5622dbaba2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb44c011a90 tx=0x7fb44c011b40 comp rx=0 tx=0).stop 2023-10-06T21:14:25.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 -- 172.21.15.26:0/3527643164 shutdown_connections 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbaba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa20c0 0x5622dbabbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.432+0000 7fb477470b80 1 --2- 172.21.15.26:0/3527643164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa29e0 0x5622dbaba2b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.433+0000 7fb477470b80 1 -- 172.21.15.26:0/3527643164 >> 172.21.15.26:0/3527643164 conn(0x5622dba17260 msgr2=0x5622dbab9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.433+0000 7fb477470b80 1 -- 172.21.15.26:0/3527643164 shutdown_connections 2023-10-06T21:14:25.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.433+0000 7fb477470b80 1 -- 172.21.15.26:0/3527643164 wait complete. 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.433+0000 7fb477470b80 1 -- start start 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 Processor -- start 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 0x5622dbac3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbac3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dbac4470 con 0x5622dbabe8a0 2023-10-06T21:14:25.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dbac45e0 con 0x5622dbaa20c0 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.434+0000 7fb477470b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622dbb222a0 con 0x5622dbaa29e0 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb477470b80 10 client.0 _ll_register_callbacks cb 0x7fb44c00ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:39188/0 (socket says 172.21.15.26:39188) 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46119a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 0x5622dbac3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 -- 172.21.15.26:0/2077203417 learned_addr learned my addr 172.21.15.26:0/2077203417 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:25.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46219c700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbac3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 -- 172.21.15.26:0/2077203417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 msgr2=0x5622dbac3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 0x5622dbac3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 -- 172.21.15.26:0/2077203417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 msgr2=0x5622dbac3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbac3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 -- 172.21.15.26:0/2077203417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4580a1040 con 0x5622dbaa20c0 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46119a700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 0x5622dbac3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:25.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb46199b700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4580098e0 tx=0x7fb45801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f851d703a8c7c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.435+0000 7fb4457fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45809f420 con 0x5622dbaa20c0 2023-10-06T21:14:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb4580a35d0 con 0x5622dbaa20c0 2023-10-06T21:14:25.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb454014ae0 con 0x5622dbaa20c0 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45800f560 con 0x5622dbaa20c0 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb477470b80 10 client.26096 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.436+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fb454009bc0 con 0x5622dbaa20c0 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb454014ae0 con 0x5622dbaa20c0 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb4580b92c0 con 0x5622dbaa20c0 2023-10-06T21:14:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 10 client.26096 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 20 client.26096 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbb235a0 0x5622dbb259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 4 client.26096 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.437+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622db9eeb30 con 0x5622dbb235a0 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.438+0000 7fb46119a700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbb235a0 0x5622dbb259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb46119a700 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbb235a0 0x5622dbb259c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b5ee266d74dd6c server_cookie=5554d4da9372b4c9 in_seq=0 out_seq=0 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb4457fa700 10 client.26096 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb4457fa700 1 -- 172.21.15.26:0/2077203417 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5622db9eeb30 con 0x5622dbb235a0 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb4457fa700 10 client.26096 handle_command_reply: tid=0 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb477470b80 1 client.26096 shutdown 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 msgr2=0x5622dbac53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.440+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4580098e0 tx=0x7fb45801a040 comp rx=0 tx=0).stop 2023-10-06T21:14:25.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 shutdown_connections 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622dbabe8a0 0x5622dbac3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622dbb235a0 0x5622dbb259c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622dbaa20c0 0x5622dbac53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 --2- 172.21.15.26:0/2077203417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622dbaa29e0 0x5622dbac3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 >> 172.21.15.26:0/2077203417 conn(0x5622dba17260 msgr2=0x5622dbaba430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.441+0000 7fb4457fa700 0 client.26096 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.442+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 shutdown_connections 2023-10-06T21:14:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.443+0000 7fb477470b80 1 -- 172.21.15.26:0/2077203417 wait complete. 2023-10-06T21:14:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.444+0000 7fb477470b80 20 client.26096 trim_cache size 0 max 16384 2023-10-06T21:14:25.448 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "840be7b8-0467-4f3d-8ff0-c22ef96e3429": { 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "840be7b8-0467-4f3d-8ff0-c22ef96e3429", 2023-10-06T21:14:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:14:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:14:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:14:25.450 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:14:25.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 msgr2=0x7fb4480a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 0x7fb4480a3cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005fd0 tx=0x7fb44c00e2f0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 msgr2=0x7fb45c19f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb458003b90 tx=0x7fb4580037c0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 shutdown_connections 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb45c104220 0x7fb45c1b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb45c0ff080 0x7fb45c19f250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb4480a1800 0x7fb4480a3cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 --2- 172.21.15.26:0/1657204601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb45c0ff9a0 0x7fb45c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 >> 172.21.15.26:0/1657204601 conn(0x7fb45c076ed0 msgr2=0x7fb45c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 shutdown_connections 2023-10-06T21:14:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:25.445+0000 7fb463bff700 1 -- 172.21.15.26:0/1657204601 wait complete. 2023-10-06T21:14:25.479 INFO:tasks.cephfs.filesystem:scrub status for tag:840be7b8-0467-4f3d-8ff0-c22ef96e3429 - {'path': '/', 'tag': '840be7b8-0467-4f3d-8ff0-c22ef96e3429', 'options': 'recursive,force'} 2023-10-06T21:14:26.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:14:26.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:14:26.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:14:26.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.325 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:26 smithi026 ceph-mon[98490]: pgmap v127: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:26 smithi078 ceph-mon[134896]: pgmap v127: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2023-10-06T21:14:26.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:14:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:14:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:14:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:26 smithi167 ceph-mon[134494]: pgmap v127: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 37 MiB/s wr, 2.02k op/s 2023-10-06T21:14:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 225.0 s, 2142.0 tps, lat 46.773 ms stddev 47.507 2023-10-06T21:14:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:28 smithi167 ceph-mon[134494]: pgmap v128: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-06T21:14:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:28 smithi026 ceph-mon[98490]: pgmap v128: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-06T21:14:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:28 smithi078 ceph-mon[134896]: pgmap v128: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-06T21:14:30.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:30 smithi167 ceph-mon[134494]: pgmap v129: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-10-06T21:14:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:30 smithi026 ceph-mon[98490]: pgmap v129: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-10-06T21:14:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:30 smithi078 ceph-mon[134896]: pgmap v129: 97 pgs: 97 active+clean; 5.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2023-10-06T21:14:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 230.0 s, 675.4 tps, lat 41.288 ms stddev 43.573 2023-10-06T21:14:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:32 smithi167 ceph-mon[134494]: pgmap v130: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:14:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:32 smithi026 ceph-mon[98490]: pgmap v130: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:14:32.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:32 smithi078 ceph-mon[134896]: pgmap v130: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:14:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:33 smithi167 ceph-mon[134494]: pgmap v131: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 532 op/s 2023-10-06T21:14:34.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:33 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:33 smithi026 ceph-mon[98490]: pgmap v131: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 532 op/s 2023-10-06T21:14:34.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:33 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:34.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:33 smithi078 ceph-mon[134896]: pgmap v131: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 532 op/s 2023-10-06T21:14:34.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:33 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:14:35.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:35.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:35.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:14:35.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:14:35.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:35.287 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:34 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:35.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:35.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:35.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:14:35.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:14:35.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:35.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:34 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:35.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:14:35.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:14:35.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:14:35.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:14:35.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:35.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:34 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:14:36.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:36 smithi167 ceph-mon[134494]: pgmap v132: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 13 MiB/s wr, 576 op/s 2023-10-06T21:14:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.320+0000 7ff71a5ff700 1 -- 172.21.15.26:0/2745891489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 msgr2=0x7ff714107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.320+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/2745891489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 0x7ff714107eb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff7080098f0 tx=0x7ff708009c00 comp rx=0 tx=0).stop 2023-10-06T21:14:36.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:36 smithi026 ceph-mon[98490]: pgmap v132: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 13 MiB/s wr, 576 op/s 2023-10-06T21:14:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.323+0000 7ff71a5ff700 1 -- 172.21.15.26:0/2745891489 shutdown_connections 2023-10-06T21:14:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.323+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/2745891489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff71411a690 0x7ff71411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.323+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/2745891489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 0x7ff714107eb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.323+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/2745891489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff7141083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.323+0000 7ff71a5ff700 1 -- 172.21.15.26:0/2745891489 >> 172.21.15.26:0/2745891489 conn(0x7ff71406cfa0 msgr2=0x7ff71406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.326+0000 7ff71a5ff700 1 -- 172.21.15.26:0/2745891489 shutdown_connections 2023-10-06T21:14:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.326+0000 7ff71a5ff700 1 -- 172.21.15.26:0/2745891489 wait complete. 2023-10-06T21:14:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.327+0000 7ff71a5ff700 1 Processor -- start 2023-10-06T21:14:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 -- start start 2023-10-06T21:14:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 0x7ff714119b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff71411a690 0x7ff714113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714114250 con 0x7ff71410f620 2023-10-06T21:14:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7141143c0 con 0x7ff71410fa90 2023-10-06T21:14:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.328+0000 7ff71a5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714114580 con 0x7ff71411a690 2023-10-06T21:14:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37944/0 (socket says 172.21.15.26:37944) 2023-10-06T21:14:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 -- 172.21.15.26:0/4132541735 learned_addr learned my addr 172.21.15.26:0/4132541735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 -- 172.21.15.26:0/4132541735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff71411a690 msgr2=0x7ff714113a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff71411a690 0x7ff714113a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 -- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 msgr2=0x7ff714119b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 0x7ff714119b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.329+0000 7ff7137fe700 1 -- 172.21.15.26:0/4132541735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708009520 con 0x7ff71410fa90 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.330+0000 7ff7137fe700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e9c0 tx=0x7ff70400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.330+0000 7ff7117fa700 1 -- 172.21.15.26:0/4132541735 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70400cbd0 con 0x7ff71410fa90 2023-10-06T21:14:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.330+0000 7ff7117fa700 1 -- 172.21.15.26:0/4132541735 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70400cd30 con 0x7ff71410fa90 2023-10-06T21:14:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.330+0000 7ff7117fa700 1 -- 172.21.15.26:0/4132541735 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704005580 con 0x7ff71410fa90 2023-10-06T21:14:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.330+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7141bb2a0 con 0x7ff71410fa90 2023-10-06T21:14:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.331+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7141bb6d0 con 0x7ff71410fa90 2023-10-06T21:14:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.336+0000 7ff7117fa700 1 -- 172.21.15.26:0/4132541735 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff704040720 con 0x7ff71410fa90 2023-10-06T21:14:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.337+0000 7ff72de45b80 1 Processor -- start 2023-10-06T21:14:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.337+0000 7ff72de45b80 1 -- start start 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.337+0000 7ff7117fa700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 0x7ff6fc0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.337+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 0x557c0044efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.337+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0045a3d0 0x557c0045c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff72de45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c003b8400 con 0x557c0044f520 2023-10-06T21:14:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff72de45b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c003a9bc0 con 0x557c0044ec00 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff72de45b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c003a74e0 con 0x557c0045a3d0 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff7117fa700 1 -- 172.21.15.26:0/4132541735 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff704014070 con 0x7ff71410fa90 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff718b9c700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 0x7ff6fc0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff718b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0045a3d0 0x557c0045c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37294/0 (socket says 172.21.15.26:37294) 2023-10-06T21:14:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.338+0000 7ff7137fe700 1 -- 172.21.15.26:0/2491536370 learned_addr learned my addr 172.21.15.26:0/2491536370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff713fff700 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 0x557c0044efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 -- 172.21.15.26:0/2491536370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0045a3d0 msgr2=0x557c0045c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0045a3d0 0x557c0045c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 -- 172.21.15.26:0/2491536370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 msgr2=0x557c0044efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 0x557c0044efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 -- 172.21.15.26:0/2491536370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c001cdd80 con 0x557c0044f520 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff713fff700 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 0x557c0044efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff718b9c700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 0x7ff6fc0a3bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff71406a250 tx=0x7ff70c007e30 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.339+0000 7ff7137fe700 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff704096770 tx=0x7ff70409a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852f170618670de4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.340+0000 7ff6fa7fc700 1 -- 172.21.15.26:0/2491536370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b1070 con 0x557c0044f520 2023-10-06T21:14:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.340+0000 7ff6fa7fc700 1 -- 172.21.15.26:0/2491536370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70409de50 con 0x557c0044f520 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.340+0000 7ff6fa7fc700 1 -- 172.21.15.26:0/2491536370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704099f60 con 0x557c0044f520 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 -- 172.21.15.26:0/2491536370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 msgr2=0x557c00459e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff704096770 tx=0x7ff70409a480 comp rx=0 tx=0).stop 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 -- 172.21.15.26:0/2491536370 shutdown_connections 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0045a3d0 0x557c0045c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0044ec00 0x557c0044efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2491536370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044f520 0x557c00459e90 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff704096770 tx=0x7ff70409a480 comp rx=0 tx=0).stop 2023-10-06T21:14:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.341+0000 7ff72de45b80 1 -- 172.21.15.26:0/2491536370 >> 172.21.15.26:0/2491536370 conn(0x557c003c4200 msgr2=0x557c0044b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.343+0000 7ff72de45b80 1 -- 172.21.15.26:0/2491536370 shutdown_connections 2023-10-06T21:14:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.343+0000 7ff72de45b80 1 -- 172.21.15.26:0/2491536370 wait complete. 2023-10-06T21:14:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.344+0000 7ff72de45b80 1 -- start start 2023-10-06T21:14:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.344+0000 7ff72de45b80 1 Processor -- start 2023-10-06T21:14:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0045a3d0 0x557c00457890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c00457dd0 0x557c00465fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c003a2dd0 con 0x557c0045a3d0 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0033f6b0 con 0x557c00457dd0 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0033f350 con 0x557c0044ec00 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.345+0000 7ff72de45b80 10 client.0 _ll_register_callbacks cb 0x557c0019c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54666/0 (socket says 172.21.15.26:54666) 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 -- 172.21.15.26:0/2369281945 learned_addr learned my addr 172.21.15.26:0/2369281945 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff718b9c700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c00457dd0 0x557c00465fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 -- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c00457dd0 msgr2=0x557c00465fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c00457dd0 0x557c00465fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 -- 172.21.15.26:0/2369281945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0045a3d0 msgr2=0x557c00457890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0045a3d0 0x557c00457890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.346+0000 7ff713fff700 1 -- 172.21.15.26:0/2369281945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7141bb6d0 con 0x557c0044ec00 2023-10-06T21:14:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff713fff700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff70800bf70 tx=0x7ff70800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72192c8e5d1589f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff6f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70801c040 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c00442d50 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff708005040 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff708005200 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff72de45b80 10 client.26098 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.347+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557c001b6c00 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70802f080 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff708015020 con 0x557c0044ec00 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 20 client.26098 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c0046a640 0x557c0046ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 4 client.26098 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c0039bdb0 con 0x557c0046a640 2023-10-06T21:14:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.348+0000 7ff7137fe700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c0046a640 0x557c0046ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.349+0000 7ff7137fe700 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c0046a640 0x557c0046ca60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f6328d230a7fdf server_cookie=6e06270df3c52d26 in_seq=0 out_seq=0 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.349+0000 7ff6f37fe700 10 client.26098 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.349+0000 7ff6f37fe700 1 -- 172.21.15.26:0/2369281945 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c0039bdb0 con 0x557c0046a640 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.349+0000 7ff6f37fe700 10 client.26098 handle_command_reply: tid=0 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 client.26098 shutdown 2023-10-06T21:14:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 msgr2=0x557c00457350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff70800bf70 tx=0x7ff70800b610 comp rx=0 tx=0).stop 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 shutdown_connections 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c0046a640 0x557c0046ca60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c00457dd0 0x557c00465fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c00457350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 --2- 172.21.15.26:0/2369281945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0045a3d0 0x557c00457890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 >> 172.21.15.26:0/2369281945 conn(0x557c003c4200 msgr2=0x557c0044ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff6f37fe700 0 client.26098 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 shutdown_connections 2023-10-06T21:14:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.350+0000 7ff72de45b80 1 -- 172.21.15.26:0/2369281945 wait complete. 2023-10-06T21:14:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.351+0000 7ff72de45b80 20 client.26098 trim_cache size 0 max 16384 2023-10-06T21:14:36.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:36 smithi078 ceph-mon[134896]: pgmap v132: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 13 MiB/s wr, 576 op/s 2023-10-06T21:14:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.378+0000 7ff72de45b80 1 Processor -- start 2023-10-06T21:14:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 -- start start 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 0x557c00460f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c00454600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c002238c0 con 0x557c0044ec00 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c00450420 con 0x557c0045a3d0 2023-10-06T21:14:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.381+0000 7ff72de45b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c00447860 con 0x557c00457dd0 2023-10-06T21:14:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:54682/0 (socket says 172.21.15.26:54682) 2023-10-06T21:14:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 -- 172.21.15.26:0/1812745677 learned_addr learned my addr 172.21.15.26:0/1812745677 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff718b9c700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c00454600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff713fff700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 0x557c00460f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 -- 172.21.15.26:0/1812745677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 msgr2=0x557c00454600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c00454600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.382+0000 7ff7137fe700 1 -- 172.21.15.26:0/1812745677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 msgr2=0x557c00460f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff7137fe700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 0x557c00460f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff7137fe700 1 -- 172.21.15.26:0/1812745677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c001f41f0 con 0x557c00457dd0 2023-10-06T21:14:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff713fff700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 0x557c00460f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff7137fe700 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff704099c90 tx=0x7ff704009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17e9ac4f7cdfd24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff6faffd700 1 -- 172.21.15.26:0/1812745677 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70407d070 con 0x557c00457dd0 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff6faffd700 1 -- 172.21.15.26:0/1812745677 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff704096040 con 0x557c00457dd0 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.383+0000 7ff6faffd700 1 -- 172.21.15.26:0/1812745677 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b1040 con 0x557c00457dd0 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.384+0000 7ff72de45b80 1 -- 172.21.15.26:0/1812745677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 msgr2=0x557c00465d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.384+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff704099c90 tx=0x7ff704009550 comp rx=0 tx=0).stop 2023-10-06T21:14:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.385+0000 7ff72de45b80 1 -- 172.21.15.26:0/1812745677 shutdown_connections 2023-10-06T21:14:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.385+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c00454600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.385+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c0044ec00 0x557c00460f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.385+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1812745677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c00457dd0 0x557c00465d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.385+0000 7ff72de45b80 1 -- 172.21.15.26:0/1812745677 >> 172.21.15.26:0/1812745677 conn(0x557c003c4200 msgr2=0x557c00465670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.387+0000 7ff72de45b80 1 -- 172.21.15.26:0/1812745677 shutdown_connections 2023-10-06T21:14:36.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.388+0000 7ff72de45b80 1 -- 172.21.15.26:0/1812745677 wait complete. 2023-10-06T21:14:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.388+0000 7ff72de45b80 1 -- start start 2023-10-06T21:14:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.389+0000 7ff72de45b80 1 Processor -- start 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c004b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c00457dd0 0x557c004b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c004bd5e0 con 0x557c00457dd0 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c004bd750 con 0x557c0045a3d0 2023-10-06T21:14:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c004bd910 con 0x557c0044ec00 2023-10-06T21:14:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.390+0000 7ff72de45b80 10 client.0 _ll_register_callbacks cb 0x7ff7040091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.391+0000 7ff718b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.391+0000 7ff718b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37988/0 (socket says 172.21.15.26:37988) 2023-10-06T21:14:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.391+0000 7ff718b9c700 1 -- 172.21.15.26:0/1704257425 learned_addr learned my addr 172.21.15.26:0/1704257425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.393+0000 7ff718b9c700 1 -- 172.21.15.26:0/1704257425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 msgr2=0x557c004b23e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.393+0000 7ff718b9c700 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c004b23e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.393+0000 7ff718b9c700 1 -- 172.21.15.26:0/1704257425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c00457dd0 msgr2=0x557c004b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.393+0000 7ff718b9c700 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c00457dd0 0x557c004b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.393+0000 7ff718b9c700 1 -- 172.21.15.26:0/1704257425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c001cdd80 con 0x557c0045a3d0 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.394+0000 7ff718b9c700 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff70c015d30 tx=0x7ff70c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23d1b192a8a53d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.394+0000 7ff6f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.395+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c01aca0 con 0x557c0045a3d0 2023-10-06T21:14:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.395+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff70c00f040 con 0x557c0045a3d0 2023-10-06T21:14:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.395+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff70409de50 con 0x557c0045a3d0 2023-10-06T21:14:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.395+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70c01ae00 con 0x557c0045a3d0 2023-10-06T21:14:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 10 client.26114 fetch_fsmap learned FSMap version 35 2023-10-06T21:14:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557c001cc250 con 0x557c0045a3d0 2023-10-06T21:14:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70409de50 con 0x557c0045a3d0 2023-10-06T21:14:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff70c026af0 con 0x557c0045a3d0 2023-10-06T21:14:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T21:14:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 20 client.26114 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c004bf110 0x557c004c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 4 client.26114 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:14:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.396+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c0039bad0 con 0x557c004bf110 2023-10-06T21:14:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.397+0000 7ff713fff700 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c004bf110 0x557c004c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.398+0000 7ff713fff700 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c004bf110 0x557c004c1570 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46292abcd8c61e8d server_cookie=9bd322b495cd12b4 in_seq=0 out_seq=0 2023-10-06T21:14:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.398+0000 7ff6f37fe700 10 client.26114 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.415+0000 7ff6f37fe700 1 -- 172.21.15.26:0/1704257425 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c0039bad0 con 0x557c004bf110 2023-10-06T21:14:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.415+0000 7ff6f37fe700 10 client.26114 handle_command_reply: tid=0 2023-10-06T21:14:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.415+0000 7ff72de45b80 1 client.26114 shutdown 2023-10-06T21:14:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 msgr2=0x557c004b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff70c015d30 tx=0x7ff70c004ad0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 shutdown_connections 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557c004bf110 0x557c004c1570 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c0045a3d0 0x557c004b2e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c0044ec00 0x557c004b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 --2- 172.21.15.26:0/1704257425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c00457dd0 0x557c004b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.416+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 >> 172.21.15.26:0/1704257425 conn(0x557c003c4200 msgr2=0x557c0044a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.418+0000 7ff6f37fe700 0 client.26114 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.418+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 shutdown_connections 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.418+0000 7ff72de45b80 1 -- 172.21.15.26:0/1704257425 wait complete. 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.418+0000 7ff72de45b80 20 client.26114 trim_cache size 0 max 16384 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:14:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:14:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:14:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 msgr2=0x7ff6fc0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 0x7ff6fc0a3bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff71406a250 tx=0x7ff70c007e30 comp rx=0 tx=0).stop 2023-10-06T21:14:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 msgr2=0x7ff71411a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e9c0 tx=0x7ff70400ed80 comp rx=0 tx=0).stop 2023-10-06T21:14:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 shutdown_connections 2023-10-06T21:14:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff71411a690 0x7ff714113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6fc0a1730 0x7ff6fc0a3bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff71410f620 0x7ff714119b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 --2- 172.21.15.26:0/4132541735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71410fa90 0x7ff71411a080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 >> 172.21.15.26:0/4132541735 conn(0x7ff71406cfa0 msgr2=0x7ff71411bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 shutdown_connections 2023-10-06T21:14:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:36.420+0000 7ff71a5ff700 1 -- 172.21.15.26:0/4132541735 wait complete. 2023-10-06T21:14:37.130 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 235.0 s, 1143.3 tps, lat 150.133 ms stddev 790.377 2023-10-06T21:14:38.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:38 smithi167 ceph-mon[134494]: pgmap v133: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.0 MiB/s wr, 267 op/s 2023-10-06T21:14:38.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:38 smithi026 ceph-mon[98490]: pgmap v133: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.0 MiB/s wr, 267 op/s 2023-10-06T21:14:38.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:38 smithi078 ceph-mon[134896]: pgmap v133: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.0 MiB/s wr, 267 op/s 2023-10-06T21:14:40.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:40 smithi167 ceph-mon[134494]: pgmap v134: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-10-06T21:14:40.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:40 smithi026 ceph-mon[98490]: pgmap v134: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-10-06T21:14:40.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:40 smithi078 ceph-mon[134896]: pgmap v134: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-10-06T21:14:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 240.0 s, 2115.4 tps, lat 47.680 ms stddev 48.097 2023-10-06T21:14:42.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:42 smithi026 ceph-mon[98490]: pgmap v135: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 555 op/s 2023-10-06T21:14:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:42 smithi078 ceph-mon[134896]: pgmap v135: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 555 op/s 2023-10-06T21:14:42.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:42 smithi167 ceph-mon[134494]: pgmap v135: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 MiB/s wr, 555 op/s 2023-10-06T21:14:44.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:44 smithi026 ceph-mon[98490]: pgmap v136: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.2 MiB/s wr, 386 op/s 2023-10-06T21:14:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:44 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:14:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:44 smithi078 ceph-mon[134896]: pgmap v136: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.2 MiB/s wr, 386 op/s 2023-10-06T21:14:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:44 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:14:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:44 smithi167 ceph-mon[134494]: pgmap v136: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.2 MiB/s wr, 386 op/s 2023-10-06T21:14:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:44 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:14:46.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:46 smithi078 ceph-mon[134896]: pgmap v137: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 587 op/s 2023-10-06T21:14:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:46 smithi167 ceph-mon[134494]: pgmap v137: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 587 op/s 2023-10-06T21:14:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:46 smithi026 ceph-mon[98490]: pgmap v137: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 587 op/s 2023-10-06T21:14:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 245.0 s, 2031.7 tps, lat 49.111 ms stddev 49.816 2023-10-06T21:14:48.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:48 smithi078 ceph-mon[134896]: pgmap v138: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-06T21:14:48.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:48 smithi167 ceph-mon[134494]: pgmap v138: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-06T21:14:48.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:48 smithi026 ceph-mon[98490]: pgmap v138: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 618 op/s 2023-10-06T21:14:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.698+0000 7ff05a3ff700 1 -- 172.21.15.26:0/3315709817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 msgr2=0x7ff05411ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.698+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/3315709817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff05411ca80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00b0a0 tx=0x7ff04c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 -- 172.21.15.26:0/3315709817 shutdown_connections 2023-10-06T21:14:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/3315709817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff05411ca80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/3315709817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff054107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/3315709817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05410fa90 0x7ff0541083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 -- 172.21.15.26:0/3315709817 >> 172.21.15.26:0/3315709817 conn(0x7ff05406cfa0 msgr2=0x7ff05406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 -- 172.21.15.26:0/3315709817 shutdown_connections 2023-10-06T21:14:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.699+0000 7ff05a3ff700 1 -- 172.21.15.26:0/3315709817 wait complete. 2023-10-06T21:14:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.701+0000 7ff05a3ff700 1 Processor -- start 2023-10-06T21:14:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.701+0000 7ff05a3ff700 1 -- start start 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.701+0000 7ff05a3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05a3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05410fa90 0x7ff0541a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05a3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff0541aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05a3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541aaff0 con 0x7ff05410fa90 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05a3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541ab160 con 0x7ff05410f620 2023-10-06T21:14:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05a3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541ab320 con 0x7ff05411a690 2023-10-06T21:14:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff053fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff053fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57326/0 (socket says 172.21.15.26:57326) 2023-10-06T21:14:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff053fff700 1 -- 172.21.15.26:0/2674561678 learned_addr learned my addr 172.21.15.26:0/2674561678 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.702+0000 7ff05899c700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff0541aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 -- 172.21.15.26:0/2674561678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 msgr2=0x7ff0541aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff0541aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 -- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05410fa90 msgr2=0x7ff0541a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05410fa90 0x7ff0541a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 -- 172.21.15.26:0/2674561678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c009d00 con 0x7ff05410f620 2023-10-06T21:14:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.703+0000 7ff053fff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff04400d950 tx=0x7ff04400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.704+0000 7ff0517fa700 1 -- 172.21.15.26:0/2674561678 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044009b40 con 0x7ff05410f620 2023-10-06T21:14:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.704+0000 7ff0517fa700 1 -- 172.21.15.26:0/2674561678 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff044009ca0 con 0x7ff05410f620 2023-10-06T21:14:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.704+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0541ab600 con 0x7ff05410f620 2023-10-06T21:14:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.704+0000 7ff0517fa700 1 -- 172.21.15.26:0/2674561678 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400f620 con 0x7ff05410f620 2023-10-06T21:14:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.704+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0541b2e40 con 0x7ff05410f620 2023-10-06T21:14:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.714+0000 7ff06dd48b80 1 Processor -- start 2023-10-06T21:14:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 -- start start 2023-10-06T21:14:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27298520 0x55cc272a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc272a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc27201400 con 0x55cc27298520 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc271f2bc0 con 0x55cc27297c00 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.715+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc271f04e0 con 0x55cc272a33d0 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57340/0 (socket says 172.21.15.26:57340) 2023-10-06T21:14:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 -- 172.21.15.26:0/2744692253 learned_addr learned my addr 172.21.15.26:0/2744692253 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 -- 172.21.15.26:0/2744692253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 msgr2=0x55cc272a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff0537fe700 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc272a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc272a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 -- 172.21.15.26:0/2744692253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27298520 msgr2=0x55cc272a2e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27298520 0x55cc272a2e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff05899c700 1 -- 172.21.15.26:0/2744692253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc27016d80 con 0x55cc27297c00 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.716+0000 7ff0537fe700 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc272a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.717+0000 7ff05899c700 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff04c015000 tx=0x7ff04c009170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbad660a7b5ab2d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.718+0000 7ff0517fa700 1 -- 172.21.15.26:0/2674561678 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff044010460 con 0x7ff05410f620 2023-10-06T21:14:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.719+0000 7ff03a7fc700 1 -- 172.21.15.26:0/2744692253 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00e050 con 0x55cc27297c00 2023-10-06T21:14:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.719+0000 7ff03a7fc700 1 -- 172.21.15.26:0/2744692253 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04c009480 con 0x55cc27297c00 2023-10-06T21:14:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.719+0000 7ff03a7fc700 1 -- 172.21.15.26:0/2744692253 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c005580 con 0x55cc27297c00 2023-10-06T21:14:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2744692253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 msgr2=0x55cc27297fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff04c015000 tx=0x7ff04c009170 comp rx=0 tx=0).stop 2023-10-06T21:14:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2744692253 shutdown_connections 2023-10-06T21:14:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc272a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc27297fe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2744692253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27298520 0x55cc272a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.720+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2744692253 >> 172.21.15.26:0/2744692253 conn(0x55cc2720d200 msgr2=0x55cc27294080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.721+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2744692253 shutdown_connections 2023-10-06T21:14:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.721+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2744692253 wait complete. 2023-10-06T21:14:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.722+0000 7ff06dd48b80 1 -- start start 2023-10-06T21:14:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.722+0000 7ff06dd48b80 1 Processor -- start 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27298520 0x55cc272a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc272a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc271ebdd0 con 0x55cc272a33d0 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc271886b0 con 0x55cc27297c00 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc27188350 con 0x55cc27298520 2023-10-06T21:14:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.723+0000 7ff06dd48b80 10 client.0 _ll_register_callbacks cb 0x55cc26fe57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57352/0 (socket says 172.21.15.26:57352) 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 -- 172.21.15.26:0/1284675040 learned_addr learned my addr 172.21.15.26:0/1284675040 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff053fff700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27298520 0x55cc272a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 -- 172.21.15.26:0/1284675040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27298520 msgr2=0x55cc272a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27298520 0x55cc272a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 -- 172.21.15.26:0/1284675040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 msgr2=0x55cc272a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc272a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc272a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.724+0000 7ff05899c700 1 -- 172.21.15.26:0/1284675040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05411d3b0 con 0x55cc27297c00 2023-10-06T21:14:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.725+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc272a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.728+0000 7ff05899c700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006080 tx=0x7ff04c007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dcc51e56b10359 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.728+0000 7ff0337fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.728+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00e040 con 0x55cc27297c00 2023-10-06T21:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.728+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff04c0055d0 con 0x55cc27297c00 2023-10-06T21:14:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.729+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc2728bd50 con 0x55cc27297c00 2023-10-06T21:14:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.730+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04c007aa0 con 0x55cc27297c00 2023-10-06T21:14:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.730+0000 7ff06dd48b80 10 client.26126 fetch_fsmap learned FSMap version 36 2023-10-06T21:14:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.730+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cc26fffc00 con 0x55cc27297c00 2023-10-06T21:14:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.730+0000 7ff0517fa700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 0x7ff03c0a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.730+0000 7ff0517fa700 1 -- 172.21.15.26:0/2674561678 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff0440cd430 con 0x7ff05410f620 2023-10-06T21:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.731+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc2728bd50 con 0x55cc27297c00 2023-10-06T21:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.731+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff04c00b0d0 con 0x55cc27297c00 2023-10-06T21:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff053fff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 0x7ff03c0a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:14:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 20 client.26126 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc272b2ac0 0x55cc272b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 4 client.26126 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.732+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc271e4db0 con 0x55cc272b2ac0 2023-10-06T21:14:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.733+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc272b2ac0 0x55cc272b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.734+0000 7ff053fff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 0x7ff03c0a3b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55cc272a1770 tx=0x7ff044096040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.734+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc272b2ac0 0x55cc272b4ee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f484df38acbccde server_cookie=e8edd8b00aa673f0 in_seq=0 out_seq=0 2023-10-06T21:14:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.734+0000 7ff0337fe700 10 client.26126 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.735+0000 7ff0337fe700 1 -- 172.21.15.26:0/1284675040 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc271e4db0 con 0x55cc272b2ac0 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.735+0000 7ff0337fe700 10 client.26126 handle_command_reply: tid=0 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 client.26126 shutdown 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 msgr2=0x55cc272a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff04c006080 tx=0x7ff04c007780 comp rx=0 tx=0).stop 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 shutdown_connections 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc272a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc272b2ac0 0x55cc272b4ee0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27297c00 0x55cc272a01c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1284675040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27298520 0x55cc272a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.736+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 >> 172.21.15.26:0/1284675040 conn(0x55cc2720d200 msgr2=0x55cc27293c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.737+0000 7ff0337fe700 0 client.26126 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.737+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 shutdown_connections 2023-10-06T21:14:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.737+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1284675040 wait complete. 2023-10-06T21:14:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.738+0000 7ff06dd48b80 20 client.26126 trim_cache size 0 max 16384 2023-10-06T21:14:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.766+0000 7ff06dd48b80 1 Processor -- start 2023-10-06T21:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.768+0000 7ff06dd48b80 1 -- start start 2023-10-06T21:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 0x55cc2729abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc2729b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc2706c8c0 con 0x55cc27297c00 2023-10-06T21:14:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc27299730 con 0x55cc27298520 2023-10-06T21:14:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc272b5590 con 0x55cc272a33d0 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff0537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff0537fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57356/0 (socket says 172.21.15.26:57356) 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.769+0000 7ff0537fe700 1 -- 172.21.15.26:0/2680706489 learned_addr learned my addr 172.21.15.26:0/2680706489 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff05899c700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc2729b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff05899c700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 0x55cc2729abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 -- 172.21.15.26:0/2680706489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 msgr2=0x55cc2729b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc2729b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 -- 172.21.15.26:0/2680706489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 msgr2=0x55cc2729abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 0x55cc2729abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 -- 172.21.15.26:0/2680706489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc2703d1f0 con 0x55cc27298520 2023-10-06T21:14:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff05899c700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 0x55cc2729abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:14:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.770+0000 7ff0537fe700 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff048000c00 tx=0x7ff048016f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13814f7ccf0e46cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.773+0000 7ff03affd700 1 -- 172.21.15.26:0/2680706489 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800b040 con 0x55cc27298520 2023-10-06T21:14:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.773+0000 7ff03affd700 1 -- 172.21.15.26:0/2680706489 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04801a070 con 0x55cc27298520 2023-10-06T21:14:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.773+0000 7ff03affd700 1 -- 172.21.15.26:0/2680706489 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048011380 con 0x55cc27298520 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2680706489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 msgr2=0x55cc2729f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff048000c00 tx=0x7ff048016f00 comp rx=0 tx=0).stop 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2680706489 shutdown_connections 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc272a33d0 0x55cc2729b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc27297c00 0x55cc2729abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/2680706489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc2729f730 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2680706489 >> 172.21.15.26:0/2680706489 conn(0x55cc2720d200 msgr2=0x55cc272ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2680706489 shutdown_connections 2023-10-06T21:14:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.774+0000 7ff06dd48b80 1 -- 172.21.15.26:0/2680706489 wait complete. 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.775+0000 7ff06dd48b80 1 -- start start 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.775+0000 7ff06dd48b80 1 Processor -- start 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 0x55cc272a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc272a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc273058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc272a85f0 con 0x55cc272a33d0 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc27316e00 con 0x55cc27298520 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc27316f40 con 0x55cc27297c00 2023-10-06T21:14:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.776+0000 7ff06dd48b80 10 client.0 _ll_register_callbacks cb 0x7ff04c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.777+0000 7ff05899c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 0x55cc272a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.777+0000 7ff05899c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 0x55cc272a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:37042/0 (socket says 172.21.15.26:37042) 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.777+0000 7ff05899c700 1 -- 172.21.15.26:0/1712681266 learned_addr learned my addr 172.21.15.26:0/1712681266 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.777+0000 7ff05899c700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc273058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.777+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc272a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 -- 172.21.15.26:0/1712681266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 msgr2=0x55cc272a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 0x55cc272a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 -- 172.21.15.26:0/1712681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 msgr2=0x55cc273058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc273058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 -- 172.21.15.26:0/1712681266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff048008560 con 0x55cc27298520 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0537fe700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc272a7db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff048008fd0 tx=0x7ff048013d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7999043a73a4fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.778+0000 7ff0337fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.779+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048016150 con 0x55cc27298520 2023-10-06T21:14:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.779+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff0480162b0 con 0x55cc27298520 2023-10-06T21:14:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.783+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff04c009480 con 0x55cc27298520 2023-10-06T21:14:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.783+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0480164b0 con 0x55cc27298520 2023-10-06T21:14:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.784+0000 7ff06dd48b80 10 client.26132 fetch_fsmap learned FSMap version 36 2023-10-06T21:14:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.784+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cc27015250 con 0x55cc27298520 2023-10-06T21:14:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff04c009480 con 0x55cc27298520 2023-10-06T21:14:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff048006380 con 0x55cc27298520 2023-10-06T21:14:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 10 client.26132 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 20 client.26132 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc27318580 0x55cc2731a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 4 client.26132 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc271e4ad0 con 0x55cc27318580 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.785+0000 7ff05899c700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc27318580 0x55cc2731a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.786+0000 7ff0337fe700 10 client.26132 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.786+0000 7ff05899c700 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc27318580 0x55cc2731a9a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a81d6c1da98ecb server_cookie=28f5c4f81ad68500 in_seq=0 out_seq=0 2023-10-06T21:14:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.935+0000 7ff0337fe700 1 -- 172.21.15.26:0/1712681266 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc271e4ad0 con 0x55cc27318580 2023-10-06T21:14:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.935+0000 7ff0337fe700 10 client.26132 handle_command_reply: tid=0 2023-10-06T21:14:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.936+0000 7ff06dd48b80 1 client.26132 shutdown 2023-10-06T21:14:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 msgr2=0x55cc272a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc272a7db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff048008fd0 tx=0x7ff048013d10 comp rx=0 tx=0).stop 2023-10-06T21:14:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 shutdown_connections 2023-10-06T21:14:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc272a33d0 0x55cc273058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc27297c00 0x55cc272a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc27318580 0x55cc2731a9a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 --2- 172.21.15.26:0/1712681266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc27298520 0x55cc272a7db0 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff048008fd0 tx=0x7ff048013d10 comp rx=0 tx=0).stop 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 >> 172.21.15.26:0/1712681266 conn(0x55cc2720d200 msgr2=0x55cc272a1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.938+0000 7ff0337fe700 0 client.26132 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.939+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 shutdown_connections 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.939+0000 7ff06dd48b80 1 -- 172.21.15.26:0/1712681266 wait complete. 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.939+0000 7ff06dd48b80 20 client.26132 trim_cache size 0 max 16384 2023-10-06T21:14:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:14:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:14:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:14:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:14:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.940+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 msgr2=0x7ff03c0a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.940+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 0x7ff03c0a3b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55cc272a1770 tx=0x7ff044096040 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.940+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 msgr2=0x7ff0541a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.940+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff04400d950 tx=0x7ff04400dd10 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 shutdown_connections 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05411a690 0x7ff0541aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff05410f620 0x7ff0541a5e00 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff04400d950 tx=0x7ff04400dd10 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff03c0a1640 0x7ff03c0a3b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 --2- 172.21.15.26:0/2674561678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05410fa90 0x7ff0541a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 >> 172.21.15.26:0/2674561678 conn(0x7ff05406cfa0 msgr2=0x7ff0541190f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 shutdown_connections 2023-10-06T21:14:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:14:49.941+0000 7ff05a3ff700 1 -- 172.21.15.26:0/2674561678 wait complete. 2023-10-06T21:14:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:50 smithi078 ceph-mon[134896]: pgmap v139: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-06T21:14:50.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:50 smithi167 ceph-mon[134494]: pgmap v139: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-06T21:14:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:50 smithi026 ceph-mon[98490]: pgmap v139: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 612 op/s 2023-10-06T21:14:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 250.0 s, 1311.1 tps, lat 75.827 ms stddev 84.081 2023-10-06T21:14:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:52 smithi078 ceph-mon[134896]: pgmap v140: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-06T21:14:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:52 smithi167 ceph-mon[134494]: pgmap v140: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-06T21:14:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:52 smithi026 ceph-mon[98490]: pgmap v140: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-10-06T21:14:54.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:54 smithi167 ceph-mon[134494]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 681 op/s 2023-10-06T21:14:54.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:54 smithi026 ceph-mon[98490]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 681 op/s 2023-10-06T21:14:54.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:54 smithi078 ceph-mon[134896]: pgmap v141: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 681 op/s 2023-10-06T21:14:55.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:14:55.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2173075638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681061e0 msgr2=0x7f1c6810dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2173075638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681061e0 0x7f1c6810dd60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b0a0 tx=0x7f1c6000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2173075638 shutdown_connections 2023-10-06T21:14:55.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2173075638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681061e0 0x7f1c6810dd60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2173075638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c681010c0 0x7f1c681014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2173075638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c68101a70 0x7f1c68105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.866+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2173075638 >> 172.21.15.26:0/2173075638 conn(0x7f1c680fc960 msgr2=0x7f1c680fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.867+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2173075638 shutdown_connections 2023-10-06T21:14:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.867+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2173075638 wait complete. 2023-10-06T21:14:55.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.868+0000 7f1c6d1ff700 1 Processor -- start 2023-10-06T21:14:55.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 -- start start 2023-10-06T21:14:55.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c681061e0 0x7f1c6810ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c6810d4c0 0x7f1c681b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681b23b0 con 0x7f1c681061e0 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681b2520 con 0x7f1c681010c0 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6d1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681b26e0 con 0x7f1c6810d4c0 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c66d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c66d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52752/0 (socket says 172.21.15.26:52752) 2023-10-06T21:14:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2874087800 learned_addr learned my addr 172.21.15.26:0/2874087800 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.869+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c6810d4c0 0x7f1c681b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2874087800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c6810d4c0 msgr2=0x7f1c681b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c6810d4c0 0x7f1c681b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c681061e0 msgr2=0x7f1c6810ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c681061e0 0x7f1c6810ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2874087800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009520 con 0x7f1c681010c0 2023-10-06T21:14:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.870+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800eb30 tx=0x7f1c5800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.871+0000 7f1c4ffff700 1 -- 172.21.15.26:0/2874087800 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800cc90 con 0x7f1c681010c0 2023-10-06T21:14:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.871+0000 7f1c4ffff700 1 -- 172.21.15.26:0/2874087800 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c580044e0 con 0x7f1c681010c0 2023-10-06T21:14:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.871+0000 7f1c4ffff700 1 -- 172.21.15.26:0/2874087800 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58003e70 con 0x7f1c681010c0 2023-10-06T21:14:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.871+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60009d00 con 0x7f1c681010c0 2023-10-06T21:14:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.871+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c681b2cf0 con 0x7f1c681010c0 2023-10-06T21:14:55.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.878+0000 7f1c80a84b80 1 Processor -- start 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 -- start start 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff30c0 0x55985bff34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c000c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bf5c460 con 0x55985bff30c0 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bf4dc20 con 0x55985bff39e0 2023-10-06T21:14:55.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.879+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bf4b540 con 0x55985bffe890 2023-10-06T21:14:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.880+0000 7f1c4ffff700 1 -- 172.21.15.26:0/2874087800 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1c58003fd0 con 0x7f1c681010c0 2023-10-06T21:14:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.880+0000 7f1c66d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.880+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff30c0 0x55985bff34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.880+0000 7f1c66d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52758/0 (socket says 172.21.15.26:52758) 2023-10-06T21:14:55.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.880+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2147715473 learned_addr learned my addr 172.21.15.26:0/2147715473 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c6659c700 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c000c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2147715473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 msgr2=0x55985c000c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c000c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2147715473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff30c0 msgr2=0x55985bff34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff30c0 0x55985bff34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.881+0000 7f1c66d9d700 1 -- 172.21.15.26:0/2147715473 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55985bd71de0 con 0x55985bff39e0 2023-10-06T21:14:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.882+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cac10 tx=0x7f1c580d5670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44cd38166706c587 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.882+0000 7f1c4d7fa700 1 -- 172.21.15.26:0/2147715473 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580cf310 con 0x55985bff39e0 2023-10-06T21:14:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.882+0000 7f1c4d7fa700 1 -- 172.21.15.26:0/2147715473 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c580cdcc0 con 0x55985bff39e0 2023-10-06T21:14:55.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.882+0000 7f1c4d7fa700 1 -- 172.21.15.26:0/2147715473 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580dc8c0 con 0x55985bff39e0 2023-10-06T21:14:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.883+0000 7f1c4ffff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 0x7f1c500a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.883+0000 7f1c4ffff700 1 -- 172.21.15.26:0/2874087800 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c58014070 con 0x7f1c681010c0 2023-10-06T21:14:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.883+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2147715473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 msgr2=0x55985bffe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.883+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c580cac10 tx=0x7f1c580d5670 comp rx=0 tx=0).stop 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2147715473 shutdown_connections 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c000c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff30c0 0x55985bff34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2147715473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985bffe350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2147715473 >> 172.21.15.26:0/2147715473 conn(0x55985bf68260 msgr2=0x55985bfef450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c6759e700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 0x7f1c500a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2147715473 shutdown_connections 2023-10-06T21:14:55.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.884+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2147715473 wait complete. 2023-10-06T21:14:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.885+0000 7f1c80a84b80 1 -- start start 2023-10-06T21:14:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.885+0000 7f1c6759e700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 0x7f1c500a3bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c6810d930 tx=0x7f1c60006230 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.886+0000 7f1c80a84b80 1 Processor -- start 2023-10-06T21:14:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985bffe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985bffb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bf46e30 con 0x55985bff39e0 2023-10-06T21:14:55.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bee3710 con 0x55985bffe890 2023-10-06T21:14:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bee33b0 con 0x55985bff30c0 2023-10-06T21:14:55.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.887+0000 7f1c80a84b80 10 client.0 _ll_register_callbacks cb 0x55985bd40800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:55.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.890+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.890+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52768/0 (socket says 172.21.15.26:52768) 2023-10-06T21:14:55.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.890+0000 7f1c6759e700 1 -- 172.21.15.26:0/3490398278 learned_addr learned my addr 172.21.15.26:0/3490398278 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985bffe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6659c700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985bffb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6759e700 1 -- 172.21.15.26:0/3490398278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 msgr2=0x55985bffe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6759e700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985bffe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6759e700 1 -- 172.21.15.26:0/3490398278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 msgr2=0x55985bffb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6759e700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985bffb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6759e700 1 -- 172.21.15.26:0/3490398278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c681b2cf0 con 0x55985bffe890 2023-10-06T21:14:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.891+0000 7f1c6659c700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985bffb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:14:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.892+0000 7f1c6759e700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c60015000 tx=0x7f1c6000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e22a7a031cd5f86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.892+0000 7f1c4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:14:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000e040 con 0x55985bffe890 2023-10-06T21:14:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c6000bab0 con 0x55985bffe890 2023-10-06T21:14:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55985bfe6f10 con 0x55985bffe890 2023-10-06T21:14:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c600056d0 con 0x55985bffe890 2023-10-06T21:14:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c80a84b80 10 client.26144 fetch_fsmap learned FSMap version 36 2023-10-06T21:14:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.893+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55985bd5ac60 con 0x55985bffe890 2023-10-06T21:14:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.894+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c60037030 con 0x55985bffe890 2023-10-06T21:14:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.894+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1c6001d420 con 0x55985bffe890 2023-10-06T21:14:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.894+0000 7f1c80a84b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:14:55.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.894+0000 7f1c80a84b80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.894+0000 7f1c80a84b80 20 client.26144 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.895+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c01f3c0 0x55985c0217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.895+0000 7f1c80a84b80 4 client.26144 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:14:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.895+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55985bf3fe10 con 0x55985c01f3c0 2023-10-06T21:14:55.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.895+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c01f3c0 0x55985c0217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.896+0000 7f1c4aff5700 10 client.26144 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:55.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.896+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c01f3c0 0x55985c0217e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c950b2be800156 server_cookie=d36a9c9caaf8c416 in_seq=0 out_seq=0 2023-10-06T21:14:55.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.897+0000 7f1c4aff5700 1 -- 172.21.15.26:0/3490398278 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55985bf3fe10 con 0x55985c01f3c0 2023-10-06T21:14:55.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.897+0000 7f1c4aff5700 10 client.26144 handle_command_reply: tid=0 2023-10-06T21:14:55.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.897+0000 7f1c80a84b80 1 client.26144 shutdown 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.897+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 msgr2=0x55985bffbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.897+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c60015000 tx=0x7f1c6000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 shutdown_connections 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c01f3c0 0x55985c0217e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bffe890 0x55985bffbc90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985bffe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/3490398278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985bffb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 >> 172.21.15.26:0/3490398278 conn(0x55985bf68260 msgr2=0x55985bfeefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c4aff5700 0 client.26144 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:55.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 shutdown_connections 2023-10-06T21:14:55.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.898+0000 7f1c80a84b80 1 -- 172.21.15.26:0/3490398278 wait complete. 2023-10-06T21:14:55.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.900+0000 7f1c80a84b80 20 client.26144 trim_cache size 0 max 16384 2023-10-06T21:14:55.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.932+0000 7f1c80a84b80 1 Processor -- start 2023-10-06T21:14:55.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.932+0000 7f1c80a84b80 1 -- start start 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.932+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985c0053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bffe890 0x55985c0213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bdc7920 con 0x55985bffe890 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985c01bcf0 con 0x55985bff39e0 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985bff0020 con 0x55985bff30c0 2023-10-06T21:14:55.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c6659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52784/0 (socket says 172.21.15.26:52784) 2023-10-06T21:14:55.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.933+0000 7f1c6659c700 1 -- 172.21.15.26:0/2465186470 learned_addr learned my addr 172.21.15.26:0/2465186470 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:55.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.936+0000 7f1c6759e700 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bffe890 0x55985c0213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.937+0000 7f1c6659c700 1 -- 172.21.15.26:0/2465186470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 msgr2=0x55985c0053e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:14:55.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.937+0000 7f1c6659c700 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985c0053e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.937+0000 7f1c6659c700 1 -- 172.21.15.26:0/2465186470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bffe890 msgr2=0x55985c0213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.937+0000 7f1c6659c700 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bffe890 0x55985c0213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.937+0000 7f1c6659c700 1 -- 172.21.15.26:0/2465186470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55985bd98250 con 0x55985bff39e0 2023-10-06T21:14:55.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.938+0000 7f1c6659c700 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00b8a0 tx=0x7f1c5c00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0bcb49026f4a41a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.938+0000 7f1c4dffb700 1 -- 172.21.15.26:0/2465186470 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00f6e0 con 0x55985bff39e0 2023-10-06T21:14:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.938+0000 7f1c4dffb700 1 -- 172.21.15.26:0/2465186470 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c5c00fd00 con 0x55985bff39e0 2023-10-06T21:14:55.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.938+0000 7f1c4dffb700 1 -- 172.21.15.26:0/2465186470 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c014e50 con 0x55985bff39e0 2023-10-06T21:14:55.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.939+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2465186470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 msgr2=0x55985c009b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.939+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00b8a0 tx=0x7f1c5c00bbb0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2465186470 shutdown_connections 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bffe890 0x55985c0213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bff30c0 0x55985c0053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/2465186470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff39e0 0x55985c009b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2465186470 >> 172.21.15.26:0/2465186470 conn(0x55985bf68260 msgr2=0x55985bff4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.940+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2465186470 shutdown_connections 2023-10-06T21:14:55.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.941+0000 7f1c80a84b80 1 -- 172.21.15.26:0/2465186470 wait complete. 2023-10-06T21:14:55.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.941+0000 7f1c80a84b80 1 -- start start 2023-10-06T21:14:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.943+0000 7f1c80a84b80 1 Processor -- start 2023-10-06T21:14:55.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff30c0 0x55985c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985c003b00 con 0x55985bff39e0 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985c003c70 con 0x55985bff30c0 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55985c003e30 con 0x55985bffe890 2023-10-06T21:14:55.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c80a84b80 10 client.0 _ll_register_callbacks cb 0x7f1c60003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c6759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:37994/0 (socket says 172.21.15.26:37994) 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c6759e700 1 -- 172.21.15.26:0/814182766 learned_addr learned my addr 172.21.15.26:0/814182766 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.944+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff30c0 0x55985c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 -- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff30c0 msgr2=0x55985c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff30c0 0x55985c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 -- 172.21.15.26:0/814182766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 msgr2=0x55985c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:14:55.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 -- 172.21.15.26:0/814182766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c00e040 con 0x55985bffe890 2023-10-06T21:14:55.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c6759e700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c6001d880 tx=0x7f1c600099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5cd0821d887083 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:14:55.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.945+0000 7f1c4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.946+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000e030 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.946+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c6001de30 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.946+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c580cdcc0 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.946+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c600055d0 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 10 client.26131 fetch_fsmap learned FSMap version 36 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55985bd702b0 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c580cdcc0 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1c6002ace0 con 0x55985bffe890 2023-10-06T21:14:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 20 client.26131 populate_metadata read hostname 'smithi026' 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c074160 0x55985c076580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 4 client.26131 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.947+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55985bf3fb30 con 0x55985c074160 2023-10-06T21:14:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.949+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c074160 0x55985c076580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:14:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.950+0000 7f1c4aff5700 10 client.26131 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.950+0000 7f1c66d9d700 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c074160 0x55985c076580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec86aef14a77a96a server_cookie=7dcd1c45d2c3c785 in_seq=0 out_seq=0 2023-10-06T21:14:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.950+0000 7f1c4aff5700 1 -- 172.21.15.26:0/814182766 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55985bf3fb30 con 0x55985c074160 2023-10-06T21:14:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.950+0000 7f1c4aff5700 10 client.26131 handle_command_reply: tid=0 2023-10-06T21:14:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.951+0000 7f1c80a84b80 1 client.26131 shutdown 2023-10-06T21:14:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.951+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 msgr2=0x55985c0033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.951+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1c6001d880 tx=0x7f1c600099e0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 shutdown_connections 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55985bffe890 0x55985c0033c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55985c074160 0x55985c076580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55985bff30c0 0x55985c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 --2- 172.21.15.26:0/814182766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55985bff39e0 0x55985c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c4aff5700 0 client.26131 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:14:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.952+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 >> 172.21.15.26:0/814182766 conn(0x55985bf68260 msgr2=0x55985c009b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.954+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 shutdown_connections 2023-10-06T21:14:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.954+0000 7f1c80a84b80 1 -- 172.21.15.26:0/814182766 wait complete. 2023-10-06T21:14:55.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.956+0000 7f1c80a84b80 20 client.26131 trim_cache size 0 max 16384 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "840be7b8-0467-4f3d-8ff0-c22ef96e3429": { 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "840be7b8-0467-4f3d-8ff0-c22ef96e3429", 2023-10-06T21:14:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 msgr2=0x7f1c500a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 0x7f1c500a3bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c6810d930 tx=0x7f1c60006230 comp rx=0 tx=0).stop 2023-10-06T21:14:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 msgr2=0x7f1c6810c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800eb30 tx=0x7f1c5800ee40 comp rx=0 tx=0).stop 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 shutdown_connections 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c6810d4c0 0x7f1c681b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c681010c0 0x7f1c6810c950 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800eb30 tx=0x7f1c5800ee40 comp rx=0 tx=0).stop 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1c500a16f0 0x7f1c500a3bb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 --2- 172.21.15.26:0/2874087800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c681061e0 0x7f1c6810ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:14:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.957+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 >> 172.21.15.26:0/2874087800 conn(0x7f1c680fc960 msgr2=0x7f1c680fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:14:55.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.980+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 shutdown_connections 2023-10-06T21:14:55.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:14:55.988+0000 7f1c6d1ff700 1 -- 172.21.15.26:0/2874087800 wait complete. 2023-10-06T21:14:56.026 INFO:tasks.cephfs.filesystem:scrub status for tag:840be7b8-0467-4f3d-8ff0-c22ef96e3429 - {'path': '/', 'tag': '840be7b8-0467-4f3d-8ff0-c22ef96e3429', 'options': 'recursive,force'} 2023-10-06T21:14:56.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:56 smithi026 ceph-mon[98490]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 985 op/s 2023-10-06T21:14:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:56 smithi167 ceph-mon[134494]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 985 op/s 2023-10-06T21:14:56.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:56 smithi078 ceph-mon[134896]: pgmap v142: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 985 op/s 2023-10-06T21:14:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 255.0 s, 1574.2 tps, lat 63.636 ms stddev 66.451 2023-10-06T21:14:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:14:58 smithi167 ceph-mon[134494]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 886 op/s 2023-10-06T21:14:58.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:14:58 smithi026 ceph-mon[98490]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 886 op/s 2023-10-06T21:14:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:14:58 smithi078 ceph-mon[134896]: pgmap v143: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 886 op/s 2023-10-06T21:15:00.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:00 smithi026 ceph-mon[98490]: pgmap v144: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 812 op/s 2023-10-06T21:15:00.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:00 smithi078 ceph-mon[134896]: pgmap v144: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 812 op/s 2023-10-06T21:15:00.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:00 smithi167 ceph-mon[134494]: pgmap v144: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 812 op/s 2023-10-06T21:15:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 260.0 s, 1123.4 tps, lat 84.994 ms stddev 114.509 2023-10-06T21:15:02.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:02 smithi026 ceph-mon[98490]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2023-10-06T21:15:02.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:02 smithi078 ceph-mon[134896]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2023-10-06T21:15:02.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:02 smithi167 ceph-mon[134494]: pgmap v145: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 -- 172.21.15.26:0/1794504759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c20102060 msgr2=0x7f0c20102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 --2- 172.21.15.26:0/1794504759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c20102060 0x7f0c20102440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009990 tx=0x7f0c10009ca0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 -- 172.21.15.26:0/1794504759 shutdown_connections 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 --2- 172.21.15.26:0/1794504759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c2010b480 0x7f0c2010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 --2- 172.21.15.26:0/1794504759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c20102060 0x7f0c20102440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 --2- 172.21.15.26:0/1794504759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102980 0x7f0c2010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 -- 172.21.15.26:0/1794504759 >> 172.21.15.26:0/1794504759 conn(0x7f0c200fb860 msgr2=0x7f0c200fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 -- 172.21.15.26:0/1794504759 shutdown_connections 2023-10-06T21:15:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.145+0000 7f0c259ff700 1 -- 172.21.15.26:0/1794504759 wait complete. 2023-10-06T21:15:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.147+0000 7f0c259ff700 1 Processor -- start 2023-10-06T21:15:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.147+0000 7f0c259ff700 1 -- start start 2023-10-06T21:15:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c20102980 0x7f0c201003d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2010b480 0x7f0c201a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a56e0 con 0x7f0c20102980 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a5820 con 0x7f0c20102060 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c259ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201a5990 con 0x7f0c2010b480 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60292/0 (socket says 172.21.15.26:60292) 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 -- 172.21.15.26:0/200800060 learned_addr learned my addr 172.21.15.26:0/200800060 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2010b480 0x7f0c201a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c20102980 0x7f0c201003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 -- 172.21.15.26:0/200800060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2010b480 msgr2=0x7f0c201a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2010b480 0x7f0c201a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 -- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c20102980 msgr2=0x7f0c201003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c20102980 0x7f0c201003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.148+0000 7f0c1effd700 1 -- 172.21.15.26:0/200800060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100095f0 con 0x7f0c20102060 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c1effd700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c10000c00 tx=0x7f0c10005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c249fd700 1 -- 172.21.15.26:0/200800060 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1001d070 con 0x7f0c20102060 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c249fd700 1 -- 172.21.15.26:0/200800060 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c10022470 con 0x7f0c20102060 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c201a5c10 con 0x7f0c20102060 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c249fd700 1 -- 172.21.15.26:0/200800060 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10017650 con 0x7f0c20102060 2023-10-06T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.149+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c201a6060 con 0x7f0c20102060 2023-10-06T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.153+0000 7f0c249fd700 1 -- 172.21.15.26:0/200800060 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0c1000fad0 con 0x7f0c20102060 2023-10-06T21:15:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.155+0000 7f0c249fd700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 0x7f0c080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.155+0000 7f0c249fd700 1 -- 172.21.15.26:0/200800060 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c100ce570 con 0x7f0c20102060 2023-10-06T21:15:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.155+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 0x7f0c080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.155+0000 7f0c3932fb80 1 Processor -- start 2023-10-06T21:15:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.155+0000 7f0c3932fb80 1 -- start start 2023-10-06T21:15:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 0x7f0c080a3ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c201016b0 tx=0x7f0c14009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bcf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bc62400 con 0x55ca7bcf8c00 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bc53bc0 con 0x55ca7bd043d0 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bc514e0 con 0x55ca7bcf9520 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:40332/0 (socket says 172.21.15.26:40332) 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/3246150835 learned_addr learned my addr 172.21.15.26:0/3246150835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.156+0000 7f0c1effd700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bcf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/3246150835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 msgr2=0x55ca7bd067c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd067c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/3246150835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 msgr2=0x55ca7bcf8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bcf8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/3246150835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7ba77d80 con 0x55ca7bcf9520 2023-10-06T21:15:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:15:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011da0 tx=0x7f0c14014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f12dc6ee24c91a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c05ffb700 1 -- 172.21.15.26:0/3246150835 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14004a30 con 0x55ca7bcf9520 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c05ffb700 1 -- 172.21.15.26:0/3246150835 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c140054e0 con 0x55ca7bcf9520 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.157+0000 7f0c05ffb700 1 -- 172.21.15.26:0/3246150835 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14003a40 con 0x55ca7bcf9520 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3246150835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 msgr2=0x55ca7bd03e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011da0 tx=0x7f0c14014000 comp rx=0 tx=0).stop 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3246150835 shutdown_connections 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bcf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3246150835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf9520 0x55ca7bd03e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3246150835 >> 172.21.15.26:0/3246150835 conn(0x55ca7bc6e200 msgr2=0x55ca7bcf5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.158+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3246150835 shutdown_connections 2023-10-06T21:15:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.159+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3246150835 wait complete. 2023-10-06T21:15:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.159+0000 7f0c3932fb80 1 -- start start 2023-10-06T21:15:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.159+0000 7f0c3932fb80 1 Processor -- start 2023-10-06T21:15:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bd01c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bc4cdd0 con 0x55ca7bcf9520 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bbe96b0 con 0x55ca7bcf8c00 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bbe9350 con 0x55ca7bd043d0 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c3932fb80 10 client.0 _ll_register_callbacks cb 0x55ca7ba467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.160+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60314/0 (socket says 172.21.15.26:60314) 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 -- 172.21.15.26:0/3331670245 learned_addr learned my addr 172.21.15.26:0/3331670245 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bd01c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 -- 172.21.15.26:0/3331670245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 msgr2=0x55ca7bd01c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bd01c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 -- 172.21.15.26:0/3331670245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 msgr2=0x55ca7bd016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 -- 172.21.15.26:0/3331670245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18003ae0 con 0x55ca7bcf8c00 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bd01c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:15:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0c1effd700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0c10022c10 tx=0x7f0c1009b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c254493ecddb7f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.161+0000 7f0bfeffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1009bc90 con 0x55ca7bcf8c00 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c1009b590 con 0x55ca7bcf8c00 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca7bcecd50 con 0x55ca7bcf8c00 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c1009f6e0 con 0x55ca7bcf8c00 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0c3932fb80 10 client.26162 fetch_fsmap learned FSMap version 36 2023-10-06T21:15:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f0c14017070 con 0x55ca7bcf8c00 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.162+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca7bcecd50 con 0x55ca7bcf8c00 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0c100b9020 con 0x55ca7bcf8c00 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 10 client.26162 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 20 client.26162 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd13c70 0x55ca7bd16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 4 client.26162 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca7bc45db0 con 0x55ca7bd13c70 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.163+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd13c70 0x55ca7bd16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.164+0000 7f0bfeffd700 10 client.26162 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.164+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd13c70 0x55ca7bd16090 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736999ce663cdfd2 server_cookie=b8bf89c56ae0ae49 in_seq=0 out_seq=0 2023-10-06T21:15:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0bfeffd700 1 -- 172.21.15.26:0/3331670245 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ca7bc45db0 con 0x55ca7bd13c70 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0bfeffd700 10 client.26162 handle_command_reply: tid=0 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 client.26162 shutdown 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 msgr2=0x55ca7bd01180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0c10022c10 tx=0x7f0c1009b410 comp rx=0 tx=0).stop 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 shutdown_connections 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bd01c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd13c70 0x55ca7bd16090 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd01180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/3331670245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 >> 172.21.15.26:0/3331670245 conn(0x55ca7bc6e200 msgr2=0x55ca7bcf4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.165+0000 7f0bfeffd700 0 client.26162 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.166+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 shutdown_connections 2023-10-06T21:15:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.166+0000 7f0c3932fb80 1 -- 172.21.15.26:0/3331670245 wait complete. 2023-10-06T21:15:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.166+0000 7f0c3932fb80 20 client.26162 trim_cache size 0 max 16384 2023-10-06T21:15:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.185+0000 7f0c3932fb80 1 Processor -- start 2023-10-06T21:15:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.185+0000 7f0c3932fb80 1 -- start start 2023-10-06T21:15:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd0af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bcfe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bacd8c0 con 0x55ca7bcf8c00 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bcf1860 con 0x55ca7bcf9520 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bd0b450 con 0x55ca7bd043d0 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd0af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bcfe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60330/0 (socket says 172.21.15.26:60330) 2023-10-06T21:15:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/1404485079 learned_addr learned my addr 172.21.15.26:0/1404485079 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bcfe3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:40362/0 (socket says 172.21.15.26:40362) 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/1404485079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 msgr2=0x55ca7bcfe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bcfe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/1404485079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 msgr2=0x55ca7bd0af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd0af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.186+0000 7f0c1e7fc700 1 -- 172.21.15.26:0/1404485079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca7ba60c00 con 0x55ca7bcf9520 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.187+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011da0 tx=0x7f0c14017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a76d24233fa461 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.187+0000 7f0c067fc700 1 -- 172.21.15.26:0/1404485079 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1402c040 con 0x55ca7bcf9520 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.187+0000 7f0c067fc700 1 -- 172.21.15.26:0/1404485079 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c140315a0 con 0x55ca7bcf9520 2023-10-06T21:15:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.187+0000 7f0c067fc700 1 -- 172.21.15.26:0/1404485079 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14004910 con 0x55ca7bcf9520 2023-10-06T21:15:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 -- 172.21.15.26:0/1404485079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 msgr2=0x55ca7bd00940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011da0 tx=0x7f0c14017000 comp rx=0 tx=0).stop 2023-10-06T21:15:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 -- 172.21.15.26:0/1404485079 shutdown_connections 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bd043d0 0x55ca7bcfe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd0af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/1404485079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bcf9520 0x55ca7bd00940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 -- 172.21.15.26:0/1404485079 >> 172.21.15.26:0/1404485079 conn(0x55ca7bc6e200 msgr2=0x55ca7bcfc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 -- 172.21.15.26:0/1404485079 shutdown_connections 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.188+0000 7f0c3932fb80 1 -- 172.21.15.26:0/1404485079 wait complete. 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.189+0000 7f0c3932fb80 1 -- start start 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.189+0000 7f0c3932fb80 1 Processor -- start 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd08970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bd09610 con 0x55ca7bcf9520 2023-10-06T21:15:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bd66fa0 con 0x55ca7bd043d0 2023-10-06T21:15:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7bd67160 con 0x55ca7bcf8c00 2023-10-06T21:15:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c3932fb80 10 client.0 _ll_register_callbacks cb 0x7f0c100978d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c1f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c1effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd08970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c1f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60342/0 (socket says 172.21.15.26:60342) 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c1f7fe700 1 -- 172.21.15.26:0/2795662647 learned_addr learned my addr 172.21.15.26:0/2795662647 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.190+0000 7f0c1e7fc700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 -- 172.21.15.26:0/2795662647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 msgr2=0x55ca7bd08970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd08970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 -- 172.21.15.26:0/2795662647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 msgr2=0x55ca7bd08eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd08eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 -- 172.21.15.26:0/2795662647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca7ba77d80 con 0x55ca7bd043d0 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1effd700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:15:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c1f7fe700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c18007b50 tx=0x7f0c18007b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a948b643246e11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0bfeffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180189b0 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c18018b10 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.191+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c140054e0 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c18018cd0 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 10 client.26168 fetch_fsmap learned FSMap version 36 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ca7ba9e1f0 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1802d080 con 0x55ca7bd043d0 2023-10-06T21:15:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f0c18029510 con 0x55ca7bd043d0 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 10 client.26168 fetch_fsmap finished waiting for FSMap version 36 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 20 client.26168 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd68a90 0x55ca7bd6aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 4 client.26168 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.192+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca7bc45ad0 con 0x55ca7bd68a90 2023-10-06T21:15:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.196+0000 7f0c1effd700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd68a90 0x55ca7bd6aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.197+0000 7f0c1effd700 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd68a90 0x55ca7bd6aeb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1196e627cd655342 server_cookie=e59270704be18dfe in_seq=0 out_seq=0 2023-10-06T21:15:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.197+0000 7f0bfeffd700 10 client.26168 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.213+0000 7f0bfeffd700 1 -- 172.21.15.26:0/2795662647 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca7bc45ad0 con 0x55ca7bd68a90 2023-10-06T21:15:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.213+0000 7f0bfeffd700 10 client.26168 handle_command_reply: tid=0 2023-10-06T21:15:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.213+0000 7f0c3932fb80 1 client.26168 shutdown 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 msgr2=0x55ca7bd66a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c18007b50 tx=0x7f0c18007b80 comp rx=0 tx=0).stop 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 shutdown_connections 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ca7bd68a90 0x55ca7bd6aeb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca7bd043d0 0x55ca7bd66a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca7bcf8c00 0x55ca7bd08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 --2- 172.21.15.26:0/2795662647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca7bcf9520 0x55ca7bd08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 >> 172.21.15.26:0/2795662647 conn(0x55ca7bc6e200 msgr2=0x55ca7bcfab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0bfeffd700 0 client.26168 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 shutdown_connections 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.214+0000 7f0c3932fb80 1 -- 172.21.15.26:0/2795662647 wait complete. 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.215+0000 7f0c3932fb80 20 client.26168 trim_cache size 0 max 16384 2023-10-06T21:15:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:15:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:15:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:15:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:15:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 msgr2=0x7f0c080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 0x7f0c080a3ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c201016b0 tx=0x7f0c14009260 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 msgr2=0x7f0c200ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c10000c00 tx=0x7f0c10005740 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 shutdown_connections 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2010b480 0x7f0c201a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c20102060 0x7f0c200ffe90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0c080a15f0 0x7f0c080a3ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 --2- 172.21.15.26:0/200800060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c20102980 0x7f0c201003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 >> 172.21.15.26:0/200800060 conn(0x7f0c200fb860 msgr2=0x7f0c2010d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 shutdown_connections 2023-10-06T21:15:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:03.216+0000 7f0c259ff700 1 -- 172.21.15.26:0/200800060 wait complete. 2023-10-06T21:15:04.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:04 smithi167 ceph-mon[134494]: pgmap v146: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 810 op/s 2023-10-06T21:15:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:04 smithi026 ceph-mon[98490]: pgmap v146: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 810 op/s 2023-10-06T21:15:04.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:04 smithi078 ceph-mon[134896]: pgmap v146: 97 pgs: 97 active+clean; 5.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 19 MiB/s wr, 810 op/s 2023-10-06T21:15:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:06 smithi167 ceph-mon[134494]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-06T21:15:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:06 smithi026 ceph-mon[98490]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-06T21:15:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:06 smithi078 ceph-mon[134896]: pgmap v147: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-06T21:15:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 265.0 s, 609.7 tps, lat 170.993 ms stddev 246.688 2023-10-06T21:15:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:08 smithi167 ceph-mon[134494]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-10-06T21:15:08.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:08 smithi026 ceph-mon[98490]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-10-06T21:15:08.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:08 smithi078 ceph-mon[134896]: pgmap v148: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.87k op/s 2023-10-06T21:15:09.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:09 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:09 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:09.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:09 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:10.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:10 smithi167 ceph-mon[134494]: pgmap v149: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-10-06T21:15:10.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:10 smithi026 ceph-mon[98490]: pgmap v149: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-10-06T21:15:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:10 smithi078 ceph-mon[134896]: pgmap v149: 97 pgs: 97 active+clean; 5.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 32 MiB/s wr, 1.77k op/s 2023-10-06T21:15:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 270.0 s, 1698.8 tps, lat 59.059 ms stddev 65.517 2023-10-06T21:15:12.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:12 smithi167 ceph-mon[134494]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 43 MiB/s wr, 2.39k op/s 2023-10-06T21:15:12.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:12 smithi026 ceph-mon[98490]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 43 MiB/s wr, 2.39k op/s 2023-10-06T21:15:12.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:12 smithi078 ceph-mon[134896]: pgmap v150: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 43 MiB/s wr, 2.39k op/s 2023-10-06T21:15:14.290 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:14 smithi167 ceph-mon[134494]: pgmap v151: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 1.98k op/s 2023-10-06T21:15:14.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:14 smithi026 ceph-mon[98490]: pgmap v151: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 1.98k op/s 2023-10-06T21:15:14.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:14 smithi078 ceph-mon[134896]: pgmap v151: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 34 MiB/s wr, 1.98k op/s 2023-10-06T21:15:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 -- 172.21.15.26:0/66436577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc107840 msgr2=0x7f10bc107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 --2- 172.21.15.26:0/66436577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc107840 0x7f10bc107c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f10b80099c0 tx=0x7f10b8009cd0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 -- 172.21.15.26:0/66436577 shutdown_connections 2023-10-06T21:15:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 --2- 172.21.15.26:0/66436577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 0x7f10bc114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 --2- 172.21.15.26:0/66436577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc107840 0x7f10bc107c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 --2- 172.21.15.26:0/66436577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc1081f0 0x7f10bc10c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 -- 172.21.15.26:0/66436577 >> 172.21.15.26:0/66436577 conn(0x7f10bc076d50 msgr2=0x7f10bc105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 -- 172.21.15.26:0/66436577 shutdown_connections 2023-10-06T21:15:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.464+0000 7f10c2bff700 1 -- 172.21.15.26:0/66436577 wait complete. 2023-10-06T21:15:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.466+0000 7f10c2bff700 1 Processor -- start 2023-10-06T21:15:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.472+0000 7f10c2bff700 1 -- start start 2023-10-06T21:15:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc107840 0x7f10bc081260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 0x7f10bc0817e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc07acf0 con 0x7f10bc10c880 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc07ae60 con 0x7f10bc07a100 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.473+0000 7f10c2bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc07edf0 con 0x7f10bc107840 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.474+0000 7f10c119c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.474+0000 7f10c119c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41194/0 (socket says 172.21.15.26:41194) 2023-10-06T21:15:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.474+0000 7f10c119c700 1 -- 172.21.15.26:0/889090043 learned_addr learned my addr 172.21.15.26:0/889090043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.482+0000 7f10c099b700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc107840 0x7f10bc081260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.485+0000 7f10b3fff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 0x7f10bc0817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.495+0000 7f10c119c700 1 -- 172.21.15.26:0/889090043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc107840 msgr2=0x7f10bc081260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.495+0000 7f10c119c700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc107840 0x7f10bc081260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.495+0000 7f10c119c700 1 -- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 msgr2=0x7f10bc0817e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.495+0000 7f10c119c700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 0x7f10bc0817e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.495+0000 7f10c119c700 1 -- 172.21.15.26:0/889090043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80095f0 con 0x7f10bc07a100 2023-10-06T21:15:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.507+0000 7f10c119c700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f10b4007f10 tx=0x7f10b400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.508+0000 7f10b1ffb700 1 -- 172.21.15.26:0/889090043 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4017070 con 0x7f10bc07a100 2023-10-06T21:15:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.508+0000 7f10b1ffb700 1 -- 172.21.15.26:0/889090043 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b400f040 con 0x7f10bc07a100 2023-10-06T21:15:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.508+0000 7f10b1ffb700 1 -- 172.21.15.26:0/889090043 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4005740 con 0x7f10bc07a100 2023-10-06T21:15:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.509+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc07f0d0 con 0x7f10bc07a100 2023-10-06T21:15:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.509+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc07f590 con 0x7f10bc07a100 2023-10-06T21:15:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.521+0000 7f10d6489b80 1 Processor -- start 2023-10-06T21:15:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.521+0000 7f10b1ffb700 1 -- 172.21.15.26:0/889090043 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f10b400dc60 con 0x7f10bc07a100 2023-10-06T21:15:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.521+0000 7f10d6489b80 1 -- start start 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a052492e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a0524957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0523f1400 con 0x55a0524933d0 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0523e2bc0 con 0x55a052487c00 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.522+0000 7f10d6489b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0523e04e0 con 0x55a052488520 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.525+0000 7f10b1ffb700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 0x7f10a40a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.525+0000 7f10b1ffb700 1 -- 172.21.15.26:0/889090043 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f10b40cbea0 con 0x7f10bc07a100 2023-10-06T21:15:16.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:16 smithi167 ceph-mon[134494]: pgmap v152: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 35 MiB/s wr, 2.13k op/s 2023-10-06T21:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.530+0000 7f10b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.530+0000 7f10c119c700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 0x7f10a40a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.530+0000 7f10b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41204/0 (socket says 172.21.15.26:41204) 2023-10-06T21:15:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.530+0000 7f10b3fff700 1 -- 172.21.15.26:0/847609938 learned_addr learned my addr 172.21.15.26:0/847609938 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.531+0000 7f10c119c700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 0x7f10a40a3c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10b40a0bf0 tx=0x7f10b400df60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.533+0000 7f10c099b700 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a052492e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 -- 172.21.15.26:0/847609938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 msgr2=0x55a052492e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a052492e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 -- 172.21.15.26:0/847609938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 msgr2=0x55a0524957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a0524957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 -- 172.21.15.26:0/847609938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a052206d80 con 0x55a052487c00 2023-10-06T21:15:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.534+0000 7f10b3fff700 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00cc90 tx=0x7f10ac0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36251c35a42a42cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.535+0000 7f10a2ffd700 1 -- 172.21.15.26:0/847609938 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac011c00 con 0x55a052487c00 2023-10-06T21:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.535+0000 7f10a2ffd700 1 -- 172.21.15.26:0/847609938 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10ac011d60 con 0x55a052487c00 2023-10-06T21:15:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.535+0000 7f10a2ffd700 1 -- 172.21.15.26:0/847609938 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac012450 con 0x55a052487c00 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 -- 172.21.15.26:0/847609938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 msgr2=0x55a052487fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00cc90 tx=0x7f10ac0113e0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 -- 172.21.15.26:0/847609938 shutdown_connections 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a0524957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052487c00 0x55a052487fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 --2- 172.21.15.26:0/847609938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a052492e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.536+0000 7f10d6489b80 1 -- 172.21.15.26:0/847609938 >> 172.21.15.26:0/847609938 conn(0x55a0523fd200 msgr2=0x55a052484080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.539+0000 7f10d6489b80 1 -- 172.21.15.26:0/847609938 shutdown_connections 2023-10-06T21:15:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.540+0000 7f10d6489b80 1 -- 172.21.15.26:0/847609938 wait complete. 2023-10-06T21:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.540+0000 7f10d6489b80 1 -- start start 2023-10-06T21:15:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.541+0000 7f10d6489b80 1 Processor -- start 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a052490700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 0x55a052490c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0523dbdd0 con 0x55a052487c00 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0523786b0 con 0x55a052488520 2023-10-06T21:15:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a052378350 con 0x55a0524933d0 2023-10-06T21:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.542+0000 7f10d6489b80 10 client.0 _ll_register_callbacks cb 0x55a0521d57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.543+0000 7f10c099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a052490700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.543+0000 7f10c099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a052490700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41216/0 (socket says 172.21.15.26:41216) 2023-10-06T21:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.543+0000 7f10c099b700 1 -- 172.21.15.26:0/2318963385 learned_addr learned my addr 172.21.15.26:0/2318963385 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.544+0000 7f10b3fff700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 0x55a052490c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 -- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 msgr2=0x55a052490700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a052490700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 -- 172.21.15.26:0/2318963385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 msgr2=0x55a0524901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 -- 172.21.15.26:0/2318963385 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac00c970 con 0x55a0524933d0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f10b3fff700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 0x55a052490c40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10ac018000 tx=0x7f10ac0050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122678086e8b603a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.545+0000 7f109bfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0055b0 con 0x55a0524933d0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a05247bd50 con 0x55a0524933d0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f10ac004030 con 0x55a0524933d0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10ac005710 con 0x55a0524933d0 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f10d6489b80 10 client.26161 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.546+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a0521efc00 con 0x55a0524933d0 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a05247bd50 con 0x55a0524933d0 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f10ac025030 con 0x55a0524933d0 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 20 client.26161 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:16.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524a2fd0 0x55a0524a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 4 client.26161 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:16.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.547+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0523d4db0 con 0x55a0524a2fd0 2023-10-06T21:15:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.548+0000 7f10c099b700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524a2fd0 0x55a0524a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.549+0000 7f10c099b700 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524a2fd0 0x55a0524a53f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f113163c7974e server_cookie=1e260750acbcf0fe in_seq=0 out_seq=0 2023-10-06T21:15:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.552+0000 7f109bfff700 10 client.26161 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.552+0000 7f109bfff700 1 -- 172.21.15.26:0/2318963385 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a0523d4db0 con 0x55a0524a2fd0 2023-10-06T21:15:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.552+0000 7f109bfff700 10 client.26161 handle_command_reply: tid=0 2023-10-06T21:15:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.552+0000 7f10d6489b80 1 client.26161 shutdown 2023-10-06T21:15:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.553+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 msgr2=0x55a052490c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.553+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 0x55a052490c40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10ac018000 tx=0x7f10ac0050d0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 shutdown_connections 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a0524933d0 0x55a052490c40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524a2fd0 0x55a0524a53f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2318963385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a052490700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 >> 172.21.15.26:0/2318963385 conn(0x55a0523fd200 msgr2=0x55a052483c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.554+0000 7f109bfff700 0 client.26161 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.555+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 shutdown_connections 2023-10-06T21:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.555+0000 7f10d6489b80 1 -- 172.21.15.26:0/2318963385 wait complete. 2023-10-06T21:15:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.555+0000 7f10d6489b80 20 client.26161 trim_cache size 0 max 16384 2023-10-06T21:15:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:16 smithi026 ceph-mon[98490]: pgmap v152: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 35 MiB/s wr, 2.13k op/s 2023-10-06T21:15:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.578+0000 7f10d6489b80 1 Processor -- start 2023-10-06T21:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 -- start start 2023-10-06T21:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052487c00 0x55a052499f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a05248b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05225c8c0 con 0x55a0524933d0 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05249a490 con 0x55a052488520 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.579+0000 7f10d6489b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a05248e720 con 0x55a052487c00 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41230/0 (socket says 172.21.15.26:41230) 2023-10-06T21:15:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 -- 172.21.15.26:0/2989667967 learned_addr learned my addr 172.21.15.26:0/2989667967 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10b3fff700 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052487c00 0x55a052499f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 -- 172.21.15.26:0/2989667967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052487c00 msgr2=0x55a052499f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052487c00 0x55a052499f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 -- 172.21.15.26:0/2989667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 msgr2=0x55a05248b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a05248b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.580+0000 7f10c099b700 1 -- 172.21.15.26:0/2989667967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a05222d1f0 con 0x55a052488520 2023-10-06T21:15:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.581+0000 7f10c099b700 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10b80098e0 tx=0x7f10b8009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f944fd405f11a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.582+0000 7f10a37fe700 1 -- 172.21.15.26:0/2989667967 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b802a070 con 0x55a052488520 2023-10-06T21:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.582+0000 7f10a37fe700 1 -- 172.21.15.26:0/2989667967 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b802e070 con 0x55a052488520 2023-10-06T21:15:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.582+0000 7f10a37fe700 1 -- 172.21.15.26:0/2989667967 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b801d040 con 0x55a052488520 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 -- 172.21.15.26:0/2989667967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 msgr2=0x55a05249e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10b80098e0 tx=0x7f10b8009530 comp rx=0 tx=0).stop 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 -- 172.21.15.26:0/2989667967 shutdown_connections 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a0524933d0 0x55a05248b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052487c00 0x55a052499f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 --2- 172.21.15.26:0/2989667967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a052488520 0x55a05249e6a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 -- 172.21.15.26:0/2989667967 >> 172.21.15.26:0/2989667967 conn(0x55a0523fd200 msgr2=0x55a0524827b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 -- 172.21.15.26:0/2989667967 shutdown_connections 2023-10-06T21:15:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.583+0000 7f10d6489b80 1 -- 172.21.15.26:0/2989667967 wait complete. 2023-10-06T21:15:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.584+0000 7f10d6489b80 1 -- start start 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.584+0000 7f10d6489b80 1 Processor -- start 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0524933d0 0x55a0524eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0524ef380 con 0x55a052487c00 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0524ef4f0 con 0x55a0524933d0 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0524ef6b0 con 0x55a052488520 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.585+0000 7f10d6489b80 10 client.0 _ll_register_callbacks cb 0x7f10b8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.586+0000 7f10c099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.586+0000 7f10c099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:35708/0 (socket says 172.21.15.26:35708) 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.586+0000 7f10c099b700 1 -- 172.21.15.26:0/797832733 learned_addr learned my addr 172.21.15.26:0/797832733 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.586+0000 7f10b3fff700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.587+0000 7f10c099b700 1 -- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0524933d0 msgr2=0x55a0524eedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.587+0000 7f10c099b700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0524933d0 0x55a0524eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.587+0000 7f10c099b700 1 -- 172.21.15.26:0/797832733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 msgr2=0x55a0524f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.587+0000 7f10c099b700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.587+0000 7f10c099b700 1 -- 172.21.15.26:0/797832733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc114c10 con 0x55a052488520 2023-10-06T21:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.588+0000 7f10c099b700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10b80095c0 tx=0x7f10b8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18ad09bbae29257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.589+0000 7f109bfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.589+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b802a050 con 0x55a052488520 2023-10-06T21:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.589+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f10b800bad0 con 0x55a052488520 2023-10-06T21:15:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.589+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10ac011d60 con 0x55a052488520 2023-10-06T21:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.591+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b801d040 con 0x55a052488520 2023-10-06T21:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.591+0000 7f10d6489b80 10 client.26167 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.591+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a052205250 con 0x55a052488520 2023-10-06T21:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.592+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b8044080 con 0x55a052488520 2023-10-06T21:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.592+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f10b80099f0 con 0x55a052488520 2023-10-06T21:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 20 client.26167 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524f7c10 0x55a0524fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 4 client.26167 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a0523d4ad0 con 0x55a0524f7c10 2023-10-06T21:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.593+0000 7f10b3fff700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524f7c10 0x55a0524fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.594+0000 7f10b3fff700 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524f7c10 0x55a0524fa030 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3611dcf8c53e05bd server_cookie=db3463872c168d02 in_seq=0 out_seq=0 2023-10-06T21:15:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.594+0000 7f109bfff700 10 client.26167 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:16.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:16 smithi078 ceph-mon[134896]: pgmap v152: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 35 MiB/s wr, 2.13k op/s 2023-10-06T21:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.620+0000 7f109bfff700 1 -- 172.21.15.26:0/797832733 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a0523d4ad0 con 0x55a0524f7c10 2023-10-06T21:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.620+0000 7f109bfff700 10 client.26167 handle_command_reply: tid=0 2023-10-06T21:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.620+0000 7f10d6489b80 1 client.26167 shutdown 2023-10-06T21:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.620+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 msgr2=0x55a0524f05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.620+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10b80095c0 tx=0x7f10b8005740 comp rx=0 tx=0).stop 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 shutdown_connections 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a0524f7c10 0x55a0524fa030 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0524933d0 0x55a0524eedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a052487c00 0x55a0524f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 --2- 172.21.15.26:0/797832733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a052488520 0x55a0524f05a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 >> 172.21.15.26:0/797832733 conn(0x55a0523fd200 msgr2=0x55a05249f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.621+0000 7f109bfff700 0 client.26167 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.623+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 shutdown_connections 2023-10-06T21:15:16.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.624+0000 7f10d6489b80 1 -- 172.21.15.26:0/797832733 wait complete. 2023-10-06T21:15:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.625+0000 7f10d6489b80 20 client.26167 trim_cache size 0 max 16384 2023-10-06T21:15:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:15:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:15:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:15:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:15:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 msgr2=0x7f10a40a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 0x7f10a40a3c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10b40a0bf0 tx=0x7f10b400df60 comp rx=0 tx=0).stop 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 msgr2=0x7f10bc07a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f10b4007f10 tx=0x7f10b400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 shutdown_connections 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10bc07a100 0x7f10bc07a5b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10bc107840 0x7f10bc081260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f10a40a17c0 0x7f10a40a3c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 --2- 172.21.15.26:0/889090043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc10c880 0x7f10bc0817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 >> 172.21.15.26:0/889090043 conn(0x7f10bc076d50 msgr2=0x7f10bc077130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.626+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 shutdown_connections 2023-10-06T21:15:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:16.627+0000 7f10c2bff700 1 -- 172.21.15.26:0/889090043 wait complete. 2023-10-06T21:15:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 275.0 s, 1928.2 tps, lat 51.764 ms stddev 56.311 2023-10-06T21:15:18.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:18 smithi167 ceph-mon[134494]: pgmap v153: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-10-06T21:15:18.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:18 smithi026 ceph-mon[98490]: pgmap v153: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-10-06T21:15:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:18 smithi078 ceph-mon[134896]: pgmap v153: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 26 MiB/s wr, 1.41k op/s 2023-10-06T21:15:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:20 smithi167 ceph-mon[134494]: pgmap v154: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-06T21:15:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:20 smithi026 ceph-mon[98490]: pgmap v154: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-06T21:15:20.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:20 smithi078 ceph-mon[134896]: pgmap v154: 97 pgs: 97 active+clean; 5.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-06T21:15:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 280.0 s, 1984.2 tps, lat 50.852 ms stddev 55.163 2023-10-06T21:15:22.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:22 smithi167 ceph-mon[134494]: pgmap v155: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2023-10-06T21:15:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:22 smithi026 ceph-mon[98490]: pgmap v155: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2023-10-06T21:15:22.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:22 smithi078 ceph-mon[134896]: pgmap v155: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2023-10-06T21:15:24.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:24 smithi167 ceph-mon[134494]: pgmap v156: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 612 op/s 2023-10-06T21:15:24.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:24 smithi026 ceph-mon[98490]: pgmap v156: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 612 op/s 2023-10-06T21:15:24.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:24 smithi078 ceph-mon[134896]: pgmap v156: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 612 op/s 2023-10-06T21:15:26.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:15:26.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.449+0000 7fb3673ff700 1 -- 172.21.15.26:0/2651111459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 msgr2=0x7fb3600ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.449+0000 7fb3673ff700 1 --2- 172.21.15.26:0/2651111459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3600ff6d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0098f0 tx=0x7fb35c009c00 comp rx=0 tx=0).stop 2023-10-06T21:15:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.450+0000 7fb3673ff700 1 -- 172.21.15.26:0/2651111459 shutdown_connections 2023-10-06T21:15:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.450+0000 7fb3673ff700 1 --2- 172.21.15.26:0/2651111459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb360067360 0x7fb36010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.450+0000 7fb3673ff700 1 --2- 172.21.15.26:0/2651111459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3600ff6d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.450+0000 7fb3673ff700 1 --2- 172.21.15.26:0/2651111459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3600ffca0 0x7fb360104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.450+0000 7fb3673ff700 1 -- 172.21.15.26:0/2651111459 >> 172.21.15.26:0/2651111459 conn(0x7fb360076ed0 msgr2=0x7fb3600fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.451+0000 7fb3673ff700 1 -- 172.21.15.26:0/2651111459 shutdown_connections 2023-10-06T21:15:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.451+0000 7fb3673ff700 1 -- 172.21.15.26:0/2651111459 wait complete. 2023-10-06T21:15:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 Processor -- start 2023-10-06T21:15:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 -- start start 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3601997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 0x7fb36019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36019e680 con 0x7fb3600ff2f0 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36019e7f0 con 0x7fb360067360 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.453+0000 7fb3673ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36019e9b0 con 0x7fb36019dc00 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49628/0 (socket says 172.21.15.26:49628) 2023-10-06T21:15:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 -- 172.21.15.26:0/1902291827 learned_addr learned my addr 172.21.15.26:0/1902291827 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36499a700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3601997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36599c700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 0x7fb36019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 -- 172.21.15.26:0/1902291827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 msgr2=0x7fb36019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 0x7fb36019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 -- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 msgr2=0x7fb3601997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3601997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.454+0000 7fb36519b700 1 -- 172.21.15.26:0/1902291827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009520 con 0x7fb360067360 2023-10-06T21:15:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.455+0000 7fb36599c700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 0x7fb36019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.455+0000 7fb36519b700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009810 tx=0x7fb35c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.455+0000 7fb3567fc700 1 -- 172.21.15.26:0/1902291827 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c01c070 con 0x7fb360067360 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.455+0000 7fb3567fc700 1 -- 172.21.15.26:0/1902291827 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb35c005500 con 0x7fb360067360 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.455+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3601a5eb0 con 0x7fb360067360 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.456+0000 7fb3567fc700 1 -- 172.21.15.26:0/1902291827 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c004a40 con 0x7fb360067360 2023-10-06T21:15:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.456+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3601a6270 con 0x7fb360067360 2023-10-06T21:15:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.463+0000 7fb37add5b80 1 Processor -- start 2023-10-06T21:15:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.464+0000 7fb3567fc700 1 -- 172.21.15.26:0/1902291827 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb35c020070 con 0x7fb360067360 2023-10-06T21:15:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.464+0000 7fb37add5b80 1 -- start start 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.464+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1670c0 0x556f6e1674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.464+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e17ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.464+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb37add5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e0d0460 con 0x556f6e1670c0 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb37add5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e0c1c20 con 0x556f6e1679e0 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb37add5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e0bf540 con 0x556f6e17b0a0 2023-10-06T21:15:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb36499a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb36499a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44664/0 (socket says 172.21.15.26:44664) 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.465+0000 7fb36499a700 1 -- 172.21.15.26:0/514683368 learned_addr learned my addr 172.21.15.26:0/514683368 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.466+0000 7fb36499a700 1 -- 172.21.15.26:0/514683368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 msgr2=0x556f6e17ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.466+0000 7fb36499a700 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e17ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.466+0000 7fb36499a700 1 -- 172.21.15.26:0/514683368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1670c0 msgr2=0x556f6e1674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.466+0000 7fb36499a700 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1670c0 0x556f6e1674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.466+0000 7fb36499a700 1 -- 172.21.15.26:0/514683368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6dee5de0 con 0x556f6e17b0a0 2023-10-06T21:15:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.467+0000 7fb36499a700 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb35000abf0 tx=0x7fb35000af00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdae100be68a9dd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.467+0000 7fb34b7fe700 1 -- 172.21.15.26:0/514683368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350013070 con 0x556f6e17b0a0 2023-10-06T21:15:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.467+0000 7fb34b7fe700 1 -- 172.21.15.26:0/514683368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb35000ed80 con 0x556f6e17b0a0 2023-10-06T21:15:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.467+0000 7fb34b7fe700 1 -- 172.21.15.26:0/514683368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000d6a0 con 0x556f6e17b0a0 2023-10-06T21:15:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.467+0000 7fb3567fc700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 0x7fb34c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.468+0000 7fb3567fc700 1 -- 172.21.15.26:0/1902291827 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb35c0cde90 con 0x7fb360067360 2023-10-06T21:15:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.468+0000 7fb37add5b80 1 -- 172.21.15.26:0/514683368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 msgr2=0x556f6e17d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb35000abf0 tx=0x7fb35000af00 comp rx=0 tx=0).stop 2023-10-06T21:15:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb36599c700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 0x7fb34c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 -- 172.21.15.26:0/514683368 shutdown_connections 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e17d490 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1670c0 0x556f6e1674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 --2- 172.21.15.26:0/514683368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e17ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.469+0000 7fb37add5b80 1 -- 172.21.15.26:0/514683368 >> 172.21.15.26:0/514683368 conn(0x556f6e0dc260 msgr2=0x556f6e163450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.470+0000 7fb37add5b80 1 -- 172.21.15.26:0/514683368 shutdown_connections 2023-10-06T21:15:26.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.470+0000 7fb37add5b80 1 -- 172.21.15.26:0/514683368 wait complete. 2023-10-06T21:15:26.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.471+0000 7fb37add5b80 1 -- start start 2023-10-06T21:15:26.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.471+0000 7fb37add5b80 1 Processor -- start 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e177e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb36599c700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 0x7fb34c0a3b70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb358009fd0 tx=0x7fb358009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e1788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e0bae30 con 0x556f6e17b0a0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e057710 con 0x556f6e1679e0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e0573b0 con 0x556f6e1670c0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb37add5b80 10 client.0 _ll_register_callbacks cb 0x556f6deb4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb36499a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb36499a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49640/0 (socket says 172.21.15.26:49640) 2023-10-06T21:15:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb36499a700 1 -- 172.21.15.26:0/2803791065 learned_addr learned my addr 172.21.15.26:0/2803791065 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.473+0000 7fb36519b700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e177e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 -- 172.21.15.26:0/2803791065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 msgr2=0x556f6e177e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e177e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 -- 172.21.15.26:0/2803791065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 msgr2=0x556f6e1788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e1788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 -- 172.21.15.26:0/2803791065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35000a8d0 con 0x556f6e1679e0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb36499a700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb350005e10 tx=0x7fb350017b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba01df08b9b6911 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350013030 con 0x556f6e1679e0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.474+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f6e15af10 con 0x556f6e1679e0 2023-10-06T21:15:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb350014070 con 0x556f6e1679e0 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb350020840 con 0x556f6e1679e0 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb37add5b80 10 client.26192 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556f6decec60 con 0x556f6e1679e0 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f6e15af10 con 0x556f6e1679e0 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb35000da50 con 0x556f6e1679e0 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.475+0000 7fb37add5b80 10 client.26192 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb37add5b80 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb37add5b80 20 client.26192 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e18aca0 0x556f6e18d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb37add5b80 4 client.26192 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f6e0b3e10 con 0x556f6e18aca0 2023-10-06T21:15:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.476+0000 7fb36519b700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e18aca0 0x556f6e18d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.478+0000 7fb36519b700 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e18aca0 0x556f6e18d0c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d6b11e38b33b53 server_cookie=415c1fa908d5eb7e in_seq=0 out_seq=0 2023-10-06T21:15:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.478+0000 7fb348ff9700 10 client.26192 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.479+0000 7fb348ff9700 1 -- 172.21.15.26:0/2803791065 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f6e0b3e10 con 0x556f6e18aca0 2023-10-06T21:15:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.479+0000 7fb348ff9700 10 client.26192 handle_command_reply: tid=0 2023-10-06T21:15:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.479+0000 7fb37add5b80 1 client.26192 shutdown 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 msgr2=0x556f6e178370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb350005e10 tx=0x7fb350017b40 comp rx=0 tx=0).stop 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 shutdown_connections 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e1788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e177e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e18aca0 0x556f6e18d0c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 --2- 172.21.15.26:0/2803791065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e178370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb348ff9700 0 client.26192 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 >> 172.21.15.26:0/2803791065 conn(0x556f6e0dc260 msgr2=0x556f6e162fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.480+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 shutdown_connections 2023-10-06T21:15:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.481+0000 7fb37add5b80 1 -- 172.21.15.26:0/2803791065 wait complete. 2023-10-06T21:15:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.482+0000 7fb37add5b80 20 client.26192 trim_cache size 0 max 16384 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.516+0000 7fb37add5b80 1 Processor -- start 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.516+0000 7fb37add5b80 1 -- start start 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1670c0 0x556f6e181c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e175640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6df3b920 con 0x556f6e1679e0 2023-10-06T21:15:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e1682e0 con 0x556f6e1670c0 2023-10-06T21:15:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.517+0000 7fb37add5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e17aee0 con 0x556f6e17b0a0 2023-10-06T21:15:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.518+0000 7fb36499a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.518+0000 7fb36499a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43332/0 (socket says 172.21.15.26:43332) 2023-10-06T21:15:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.518+0000 7fb36499a700 1 -- 172.21.15.26:0/1080717752 learned_addr learned my addr 172.21.15.26:0/1080717752 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.519+0000 7fb36519b700 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1670c0 0x556f6e181c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.519+0000 7fb36499a700 1 -- 172.21.15.26:0/1080717752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 msgr2=0x556f6e175640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.520+0000 7fb36499a700 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e175640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.520+0000 7fb36499a700 1 -- 172.21.15.26:0/1080717752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1670c0 msgr2=0x556f6e181c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.520+0000 7fb36499a700 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1670c0 0x556f6e181c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.520+0000 7fb36499a700 1 -- 172.21.15.26:0/1080717752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6df0c250 con 0x556f6e1679e0 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.520+0000 7fb36499a700 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35000b010 tx=0x7fb350001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1905713b04ae976 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.521+0000 7fb34bfff700 1 -- 172.21.15.26:0/1080717752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350013040 con 0x556f6e1679e0 2023-10-06T21:15:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.521+0000 7fb34bfff700 1 -- 172.21.15.26:0/1080717752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb350005e80 con 0x556f6e1679e0 2023-10-06T21:15:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.521+0000 7fb34bfff700 1 -- 172.21.15.26:0/1080717752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000d840 con 0x556f6e1679e0 2023-10-06T21:15:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 -- 172.21.15.26:0/1080717752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 msgr2=0x556f6e175100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35000b010 tx=0x7fb350001e70 comp rx=0 tx=0).stop 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 -- 172.21.15.26:0/1080717752 shutdown_connections 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e17b0a0 0x556f6e175640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1670c0 0x556f6e181c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1080717752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e1679e0 0x556f6e175100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 -- 172.21.15.26:0/1080717752 >> 172.21.15.26:0/1080717752 conn(0x556f6e0dc260 msgr2=0x556f6e186fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 -- 172.21.15.26:0/1080717752 shutdown_connections 2023-10-06T21:15:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.522+0000 7fb37add5b80 1 -- 172.21.15.26:0/1080717752 wait complete. 2023-10-06T21:15:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.523+0000 7fb37add5b80 1 -- start start 2023-10-06T21:15:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.523+0000 7fb37add5b80 1 Processor -- start 2023-10-06T21:15:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e1815a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e181ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e17fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e180200 con 0x556f6e17b0a0 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e180370 con 0x556f6e1679e0 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6e180530 con 0x556f6e1670c0 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.524+0000 7fb37add5b80 10 client.0 _ll_register_callbacks cb 0x7fb35000aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36519b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e1815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36519b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e1815a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44698/0 (socket says 172.21.15.26:44698) 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36519b700 1 -- 172.21.15.26:0/1959825279 learned_addr learned my addr 172.21.15.26:0/1959825279 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e181ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 -- 172.21.15.26:0/1959825279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 msgr2=0x556f6e1815a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e1815a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 -- 172.21.15.26:0/1959825279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 msgr2=0x556f6e17fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e17fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 -- 172.21.15.26:0/1959825279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f6decec60 con 0x556f6e1679e0 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb36499a700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e181ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ddb0 tx=0x7fb350017d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d671676c0819d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.525+0000 7fb348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.526+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350013040 con 0x556f6e1679e0 2023-10-06T21:15:26.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.526+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb35001f4d0 con 0x556f6e1679e0 2023-10-06T21:15:26.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.527+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb35000ed80 con 0x556f6e1679e0 2023-10-06T21:15:26.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.528+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3500054d0 con 0x556f6e1679e0 2023-10-06T21:15:26.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.528+0000 7fb37add5b80 10 client.26198 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:26.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.528+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556f6dee42b0 con 0x556f6e1679e0 2023-10-06T21:15:26.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb35000ed80 con 0x556f6e1679e0 2023-10-06T21:15:26.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb35001f9a0 con 0x556f6e1679e0 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 20 client.26198 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e16ae70 0x556f6e16d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 4 client.26198 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f6e0b3b30 con 0x556f6e16ae70 2023-10-06T21:15:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.529+0000 7fb36519b700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e16ae70 0x556f6e16d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:26.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:26 smithi167 ceph-mon[134494]: pgmap v157: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 776 op/s 2023-10-06T21:15:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.530+0000 7fb36519b700 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e16ae70 0x556f6e16d290 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2f5dd22cb749b8 server_cookie=40fd9da910fe571d in_seq=0 out_seq=0 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.530+0000 7fb348ff9700 10 client.26198 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.530+0000 7fb348ff9700 1 -- 172.21.15.26:0/1959825279 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556f6e0b3b30 con 0x556f6e16ae70 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.530+0000 7fb348ff9700 10 client.26198 handle_command_reply: tid=0 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.530+0000 7fb37add5b80 1 client.26198 shutdown 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 msgr2=0x556f6e181ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e181ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ddb0 tx=0x7fb350017d20 comp rx=0 tx=0).stop 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 shutdown_connections 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556f6e17b0a0 0x556f6e17fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f6e1670c0 0x556f6e1815a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556f6e16ae70 0x556f6e16d290 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 --2- 172.21.15.26:0/1959825279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556f6e1679e0 0x556f6e181ae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 >> 172.21.15.26:0/1959825279 conn(0x556f6e0dc260 msgr2=0x556f6e177d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.531+0000 7fb348ff9700 0 client.26198 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.535+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 shutdown_connections 2023-10-06T21:15:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.535+0000 7fb37add5b80 1 -- 172.21.15.26:0/1959825279 wait complete. 2023-10-06T21:15:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.535+0000 7fb37add5b80 20 client.26198 trim_cache size 0 max 16384 2023-10-06T21:15:26.539 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:15:26.539 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:15:26.539 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:15:26.539 INFO:teuthology.orchestra.run.smithi026.stdout: "840be7b8-0467-4f3d-8ff0-c22ef96e3429": { 2023-10-06T21:15:26.539 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:15:26.540 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "840be7b8-0467-4f3d-8ff0-c22ef96e3429", 2023-10-06T21:15:26.540 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:15:26.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:15:26.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:15:26.541 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:15:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 msgr2=0x7fb34c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 0x7fb34c0a3b70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb358009fd0 tx=0x7fb358009920 comp rx=0 tx=0).stop 2023-10-06T21:15:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 msgr2=0x7fb360199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009810 tx=0x7fb35c00b790 comp rx=0 tx=0).stop 2023-10-06T21:15:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 shutdown_connections 2023-10-06T21:15:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb36019dc00 0x7fb36019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb360067360 0x7fb360199260 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb34c0a16b0 0x7fb34c0a3b70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 --2- 172.21.15.26:0/1902291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3600ff2f0 0x7fb3601997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 >> 172.21.15.26:0/1902291827 conn(0x7fb360076ed0 msgr2=0x7fb360103550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.536+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 shutdown_connections 2023-10-06T21:15:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:26.537+0000 7fb3673ff700 1 -- 172.21.15.26:0/1902291827 wait complete. 2023-10-06T21:15:26.564 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:26 smithi026 ceph-mon[98490]: pgmap v157: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 776 op/s 2023-10-06T21:15:26.569 INFO:tasks.cephfs.filesystem:scrub status for tag:840be7b8-0467-4f3d-8ff0-c22ef96e3429 - {'path': '/', 'tag': '840be7b8-0467-4f3d-8ff0-c22ef96e3429', 'options': 'recursive,force'} 2023-10-06T21:15:26.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:26 smithi078 ceph-mon[134896]: pgmap v157: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 776 op/s 2023-10-06T21:15:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 285.0 s, 2328.0 tps, lat 42.784 ms stddev 48.201 2023-10-06T21:15:28.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:28 smithi167 ceph-mon[134494]: pgmap v158: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 741 op/s 2023-10-06T21:15:28.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:28 smithi026 ceph-mon[98490]: pgmap v158: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 741 op/s 2023-10-06T21:15:28.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:28 smithi078 ceph-mon[134896]: pgmap v158: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 741 op/s 2023-10-06T21:15:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.871+0000 7fdfa07ff700 1 -- 172.21.15.26:0/3380451330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf98107840 msgr2=0x7fdf98107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.871+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/3380451330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf98107840 0x7fdf98107c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf940098f0 tx=0x7fdf94009c00 comp rx=0 tx=0).stop 2023-10-06T21:15:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.872+0000 7fdfa07ff700 1 -- 172.21.15.26:0/3380451330 shutdown_connections 2023-10-06T21:15:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.872+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/3380451330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf9810c880 0x7fdf98114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.873+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/3380451330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf98107840 0x7fdf98107c20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.873+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/3380451330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf981081f0 0x7fdf9810c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.873+0000 7fdfa07ff700 1 -- 172.21.15.26:0/3380451330 >> 172.21.15.26:0/3380451330 conn(0x7fdf98076d50 msgr2=0x7fdf98105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.875+0000 7fdfa07ff700 1 -- 172.21.15.26:0/3380451330 shutdown_connections 2023-10-06T21:15:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.875+0000 7fdfa07ff700 1 -- 172.21.15.26:0/3380451330 wait complete. 2023-10-06T21:15:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.877+0000 7fdfa07ff700 1 Processor -- start 2023-10-06T21:15:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.877+0000 7fdfa07ff700 1 -- start start 2023-10-06T21:15:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9810c880 0x7fdf980729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9806b230 0x7fdf9806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9806be20 con 0x7fdf9806b230 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9806bf90 con 0x7fdf98107840 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.878+0000 7fdfa07ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9806ffc0 con 0x7fdf9810c880 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49682/0 (socket says 172.21.15.26:49682) 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 -- 172.21.15.26:0/1785958806 learned_addr learned my addr 172.21.15.26:0/1785958806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9806b230 0x7fdf9806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9810c880 0x7fdf980729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 -- 172.21.15.26:0/1785958806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9810c880 msgr2=0x7fdf980729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9810c880 0x7fdf980729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 -- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9806b230 msgr2=0x7fdf9806b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9806b230 0x7fdf9806b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.879+0000 7fdf9e59b700 1 -- 172.21.15.26:0/1785958806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94009520 con 0x7fdf98107840 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.880+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400bb20 tx=0x7fdf9400bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.881+0000 7fdf8f7fe700 1 -- 172.21.15.26:0/1785958806 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf940054e0 con 0x7fdf98107840 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.881+0000 7fdf8f7fe700 1 -- 172.21.15.26:0/1785958806 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf9400f930 con 0x7fdf98107840 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.881+0000 7fdf8f7fe700 1 -- 172.21.15.26:0/1785958806 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94021920 con 0x7fdf98107840 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.881+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf98070240 con 0x7fdf98107840 2023-10-06T21:15:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.881+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf980706a0 con 0x7fdf98107840 2023-10-06T21:15:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.890+0000 7fdfb418cb80 1 Processor -- start 2023-10-06T21:15:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.891+0000 7fdf8f7fe700 1 -- 172.21.15.26:0/1785958806 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf94003bd0 con 0x7fdf98107840 2023-10-06T21:15:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.891+0000 7fdfb418cb80 1 -- start start 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d677fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 0x55ed4d682e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d5e1400 con 0x55ed4d678520 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d5d2bc0 con 0x55ed4d6833d0 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.892+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d5d04e0 con 0x55ed4d677c00 2023-10-06T21:15:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.893+0000 7fdf9e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 0x55ed4d682e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.893+0000 7fdf9e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 0x55ed4d682e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43366/0 (socket says 172.21.15.26:43366) 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.893+0000 7fdf9e59b700 1 -- 172.21.15.26:0/4258422284 learned_addr learned my addr 172.21.15.26:0/4258422284 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/4258422284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 msgr2=0x55ed4d677fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d677fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/4258422284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 msgr2=0x55ed4d682e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 0x55ed4d682e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.894+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/4258422284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed4d3f6d80 con 0x55ed4d6833d0 2023-10-06T21:15:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.895+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6857c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806a590 tx=0x7fdf88009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe190923c32965a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.896+0000 7fdf8cff9700 1 -- 172.21.15.26:0/4258422284 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88011c80 con 0x55ed4d6833d0 2023-10-06T21:15:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.896+0000 7fdf8cff9700 1 -- 172.21.15.26:0/4258422284 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88011de0 con 0x55ed4d6833d0 2023-10-06T21:15:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.896+0000 7fdf8cff9700 1 -- 172.21.15.26:0/4258422284 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88005280 con 0x55ed4d6833d0 2023-10-06T21:15:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.897+0000 7fdfb418cb80 1 -- 172.21.15.26:0/4258422284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 msgr2=0x55ed4d6857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.897+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6857c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf9806a590 tx=0x7fdf88009f60 comp rx=0 tx=0).stop 2023-10-06T21:15:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.898+0000 7fdfb418cb80 1 -- 172.21.15.26:0/4258422284 shutdown_connections 2023-10-06T21:15:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.898+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6857c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.898+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d677fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.898+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/4258422284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d678520 0x55ed4d682e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.898+0000 7fdfb418cb80 1 -- 172.21.15.26:0/4258422284 >> 172.21.15.26:0/4258422284 conn(0x55ed4d5ed200 msgr2=0x55ed4d674080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.899+0000 7fdfb418cb80 1 -- 172.21.15.26:0/4258422284 shutdown_connections 2023-10-06T21:15:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.900+0000 7fdfb418cb80 1 -- 172.21.15.26:0/4258422284 wait complete. 2023-10-06T21:15:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.901+0000 7fdfb418cb80 1 -- start start 2023-10-06T21:15:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.901+0000 7fdfb418cb80 1 Processor -- start 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d680140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d680680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d680bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdf8f7fe700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 0x7fdf840a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d5cbdd0 con 0x55ed4d677c00 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d5686b0 con 0x55ed4d6833d0 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d568350 con 0x55ed4d678520 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdfb418cb80 10 client.0 _ll_register_callbacks cb 0x55ed4d3c57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdf8f7fe700 1 -- 172.21.15.26:0/1785958806 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf940ce510 con 0x7fdf98107840 2023-10-06T21:15:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdf9e59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d680680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdf9ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d680bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.903+0000 7fdf9ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d680bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49692/0 (socket says 172.21.15.26:49692) 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9ed9c700 1 -- 172.21.15.26:0/3554283087 learned_addr learned my addr 172.21.15.26:0/3554283087 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 0x7fdf840a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d680140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9e59b700 1 -- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 msgr2=0x55ed4d680bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d680bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9e59b700 1 -- 172.21.15.26:0/3554283087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 msgr2=0x55ed4d680140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d680140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9e59b700 1 -- 172.21.15.26:0/3554283087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf880098e0 con 0x55ed4d678520 2023-10-06T21:15:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.904+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d680140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d680680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf94097a90 tx=0x7fdf940d1c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56fd553cc7da3a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:15:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94086070 con 0x55ed4d678520 2023-10-06T21:15:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf9409bc90 con 0x55ed4d678520 2023-10-06T21:15:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed4d66bd50 con 0x55ed4d678520 2023-10-06T21:15:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 0x7fdf840a3a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf88004bd0 tx=0x7fdf88004c30 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf9409b360 con 0x55ed4d678520 2023-10-06T21:15:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdfb418cb80 10 client.26188 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.906+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ed4d3dfc00 con 0x55ed4d678520 2023-10-06T21:15:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed4d66bd50 con 0x55ed4d678520 2023-10-06T21:15:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdf9409aa80 con 0x55ed4d678520 2023-10-06T21:15:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 10 client.26188 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 20 client.26188 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d692c40 0x55ed4d695060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 4 client.26188 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.907+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed4d5c4db0 con 0x55ed4d692c40 2023-10-06T21:15:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.908+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d692c40 0x55ed4d695060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.910+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d692c40 0x55ed4d695060 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d6608e7b648e7b server_cookie=c3420287b8636a71 in_seq=0 out_seq=0 2023-10-06T21:15:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.910+0000 7fdf81ffb700 10 client.26188 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.911+0000 7fdf81ffb700 1 -- 172.21.15.26:0/3554283087 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed4d5c4db0 con 0x55ed4d692c40 2023-10-06T21:15:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.911+0000 7fdf81ffb700 10 client.26188 handle_command_reply: tid=0 2023-10-06T21:15:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.911+0000 7fdfb418cb80 1 client.26188 shutdown 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 msgr2=0x55ed4d680680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d680680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdf94097a90 tx=0x7fdf940d1c10 comp rx=0 tx=0).stop 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 shutdown_connections 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d692c40 0x55ed4d695060 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d680bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d680140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3554283087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d680680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 >> 172.21.15.26:0/3554283087 conn(0x55ed4d5ed200 msgr2=0x55ed4d673c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdf81ffb700 0 client.26188 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 shutdown_connections 2023-10-06T21:15:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3554283087 wait complete. 2023-10-06T21:15:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.912+0000 7fdfb418cb80 20 client.26188 trim_cache size 0 max 16384 2023-10-06T21:15:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.943+0000 7fdfb418cb80 1 Processor -- start 2023-10-06T21:15:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.943+0000 7fdfb418cb80 1 -- start start 2023-10-06T21:15:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d689f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d6833d0 0x55ed4d67b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d44c8c0 con 0x55ed4d6833d0 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d67b080 con 0x55ed4d678520 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d63e860 con 0x55ed4d677c00 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdf9dd9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdf9dd9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49700/0 (socket says 172.21.15.26:49700) 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.944+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/3894104627 learned_addr learned my addr 172.21.15.26:0/3894104627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d6833d0 0x55ed4d67b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/3894104627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 msgr2=0x55ed4d689f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d689f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/3894104627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d6833d0 msgr2=0x55ed4d67b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d6833d0 0x55ed4d67b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.945+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/3894104627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed4d41d1f0 con 0x55ed4d678520 2023-10-06T21:15:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.946+0000 7fdf9dd9a700 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf880047f0 tx=0x7fdf88021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad9112a76896ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.946+0000 7fdf8d7fa700 1 -- 172.21.15.26:0/3894104627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf880048f0 con 0x55ed4d678520 2023-10-06T21:15:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.946+0000 7fdf8d7fa700 1 -- 172.21.15.26:0/3894104627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88005590 con 0x55ed4d678520 2023-10-06T21:15:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.946+0000 7fdf8d7fa700 1 -- 172.21.15.26:0/3894104627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88023e10 con 0x55ed4d678520 2023-10-06T21:15:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.947+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3894104627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 msgr2=0x55ed4d67f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.947+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf880047f0 tx=0x7fdf88021000 comp rx=0 tx=0).stop 2023-10-06T21:15:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.948+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3894104627 shutdown_connections 2023-10-06T21:15:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.948+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d6833d0 0x55ed4d67b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.948+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d677c00 0x55ed4d689f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.948+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/3894104627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d678520 0x55ed4d67f960 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.948+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3894104627 >> 172.21.15.26:0/3894104627 conn(0x55ed4d5ed200 msgr2=0x55ed4d68e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.949+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3894104627 shutdown_connections 2023-10-06T21:15:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.949+0000 7fdfb418cb80 1 -- 172.21.15.26:0/3894104627 wait complete. 2023-10-06T21:15:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.950+0000 7fdfb418cb80 1 -- start start 2023-10-06T21:15:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.950+0000 7fdfb418cb80 1 Processor -- start 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d687730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d687c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d688580 con 0x55ed4d677c00 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d6e5fb0 con 0x55ed4d6833d0 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed4d6e60f0 con 0x55ed4d678520 2023-10-06T21:15:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.951+0000 7fdfb418cb80 10 client.0 _ll_register_callbacks cb 0x55ed4d3c57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.952+0000 7fdf9dd9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d687c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.952+0000 7fdf9dd9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d687c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44724/0 (socket says 172.21.15.26:44724) 2023-10-06T21:15:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.952+0000 7fdf9dd9a700 1 -- 172.21.15.26:0/2793150625 learned_addr learned my addr 172.21.15.26:0/2793150625 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.952+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.952+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d687730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 -- 172.21.15.26:0/2793150625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 msgr2=0x55ed4d687c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d687c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 -- 172.21.15.26:0/2793150625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 msgr2=0x55ed4d687730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d687730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 -- 172.21.15.26:0/2793150625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8801d040 con 0x55ed4d6833d0 2023-10-06T21:15:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d687730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:15:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf9ed9c700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6e5a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009cd0 tx=0x7fdf90004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74bb8bfb36d9e92f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.953+0000 7fdf81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.954+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000e040 con 0x55ed4d6833d0 2023-10-06T21:15:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.954+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf90017410 con 0x55ed4d6833d0 2023-10-06T21:15:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.954+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf88011de0 con 0x55ed4d6833d0 2023-10-06T21:15:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.958+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf90007e70 con 0x55ed4d6833d0 2023-10-06T21:15:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.958+0000 7fdfb418cb80 10 client.26216 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.958+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ed4d3f5250 con 0x55ed4d6833d0 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.959+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf88011de0 con 0x55ed4d6833d0 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.959+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fdf9002c070 con 0x55ed4d6833d0 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 10 client.26216 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 20 client.26216 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d6e7770 0x55ed4d6e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 4 client.26216 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:15:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.960+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed4d5c4ad0 con 0x55ed4d6e7770 2023-10-06T21:15:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.961+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d6e7770 0x55ed4d6e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.962+0000 7fdf9e59b700 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d6e7770 0x55ed4d6e9b90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c769fe22e1e2f3 server_cookie=32a3c10cef202c00 in_seq=0 out_seq=0 2023-10-06T21:15:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:29.962+0000 7fdf81ffb700 10 client.26216 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.092+0000 7fdf81ffb700 1 -- 172.21.15.26:0/2793150625 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed4d5c4ad0 con 0x55ed4d6e7770 2023-10-06T21:15:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.092+0000 7fdf81ffb700 10 client.26216 handle_command_reply: tid=0 2023-10-06T21:15:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.092+0000 7fdfb418cb80 1 client.26216 shutdown 2023-10-06T21:15:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.092+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 msgr2=0x55ed4d6e5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.092+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6e5a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009cd0 tx=0x7fdf90004e30 comp rx=0 tx=0).stop 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 shutdown_connections 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed4d6e7770 0x55ed4d6e9b90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed4d6833d0 0x55ed4d6e5a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed4d677c00 0x55ed4d687730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 --2- 172.21.15.26:0/2793150625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed4d678520 0x55ed4d687c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdf81ffb700 0 client.26216 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 >> 172.21.15.26:0/2793150625 conn(0x55ed4d5ed200 msgr2=0x55ed4d680960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 shutdown_connections 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.093+0000 7fdfb418cb80 1 -- 172.21.15.26:0/2793150625 wait complete. 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.094+0000 7fdfb418cb80 20 client.26216 trim_cache size 0 max 16384 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:15:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:15:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 msgr2=0x7fdf840a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 0x7fdf840a3a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf88004bd0 tx=0x7fdf88004c30 comp rx=0 tx=0).stop 2023-10-06T21:15:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 msgr2=0x7fdf98072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400bb20 tx=0x7fdf9400bb50 comp rx=0 tx=0).stop 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 shutdown_connections 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf840a1580 0x7fdf840a3a40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9806b230 0x7fdf9806b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf98107840 0x7fdf98072430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 --2- 172.21.15.26:0/1785958806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf9810c880 0x7fdf980729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 >> 172.21.15.26:0/1785958806 conn(0x7fdf98076d50 msgr2=0x7fdf980771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.096+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 shutdown_connections 2023-10-06T21:15:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:30.097+0000 7fdfa07ff700 1 -- 172.21.15.26:0/1785958806 wait complete. 2023-10-06T21:15:30.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:30 smithi167 ceph-mon[134494]: pgmap v159: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 613 op/s 2023-10-06T21:15:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:30 smithi026 ceph-mon[98490]: pgmap v159: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 613 op/s 2023-10-06T21:15:30.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:30 smithi078 ceph-mon[134896]: pgmap v159: 97 pgs: 97 active+clean; 5.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 613 op/s 2023-10-06T21:15:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 290.0 s, 1788.6 tps, lat 55.350 ms stddev 68.531 2023-10-06T21:15:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:32 smithi167 ceph-mon[134494]: pgmap v160: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 924 op/s 2023-10-06T21:15:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:32 smithi026 ceph-mon[98490]: pgmap v160: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 924 op/s 2023-10-06T21:15:32.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:32 smithi078 ceph-mon[134896]: pgmap v160: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 924 op/s 2023-10-06T21:15:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:34 smithi167 ceph-mon[134494]: pgmap v161: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 584 op/s 2023-10-06T21:15:34.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:34 smithi026 ceph-mon[98490]: pgmap v161: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 584 op/s 2023-10-06T21:15:34.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:34 smithi078 ceph-mon[134896]: pgmap v161: 97 pgs: 97 active+clean; 6.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 584 op/s 2023-10-06T21:15:35.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:35 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:35.327 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:35 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:35.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:35 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:15:36.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:36 smithi167 ceph-mon[134494]: pgmap v162: 97 pgs: 97 active+clean; 6.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 847 op/s 2023-10-06T21:15:36.329 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:36 smithi026 ceph-mon[98490]: pgmap v162: 97 pgs: 97 active+clean; 6.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 847 op/s 2023-10-06T21:15:36.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:36 smithi078 ceph-mon[134896]: pgmap v162: 97 pgs: 97 active+clean; 6.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 847 op/s 2023-10-06T21:15:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 295.0 s, 1603.6 tps, lat 63.091 ms stddev 67.173 2023-10-06T21:15:37.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:37 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:37.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:37 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:37.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:37 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:15:37.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:37 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:37.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:37 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:37.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:37 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:15:37.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:37 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:15:37.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:37 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:15:37.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:37 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:15:38.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:38 smithi167 ceph-mon[134494]: pgmap v163: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 828 op/s 2023-10-06T21:15:38.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:38 smithi026 ceph-mon[98490]: pgmap v163: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 828 op/s 2023-10-06T21:15:38.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:38 smithi078 ceph-mon[134896]: pgmap v163: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 828 op/s 2023-10-06T21:15:40.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:40 smithi167 ceph-mon[134494]: pgmap v164: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 718 op/s 2023-10-06T21:15:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:40 smithi026 ceph-mon[98490]: pgmap v164: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 718 op/s 2023-10-06T21:15:40.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:40 smithi078 ceph-mon[134896]: pgmap v164: 97 pgs: 97 active+clean; 6.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 718 op/s 2023-10-06T21:15:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 300.0 s, 1801.2 tps, lat 55.286 ms stddev 60.579 2023-10-06T21:15:42.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:42 smithi167 ceph-mon[134494]: pgmap v165: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-06T21:15:42.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:42 smithi026 ceph-mon[98490]: pgmap v165: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-06T21:15:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:42 smithi078 ceph-mon[134896]: pgmap v165: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.364+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/873635411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c411a690 msgr2=0x7fe9c411ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.364+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/873635411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c411a690 0x7fe9c411ca80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009990 tx=0x7fe9b4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/873635411 shutdown_connections 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/873635411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c411a690 0x7fe9c411ca80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/873635411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c410f620 0x7fe9c4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/873635411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410fa90 0x7fe9c41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/873635411 >> 172.21.15.26:0/873635411 conn(0x7fe9c406cfa0 msgr2=0x7fe9c406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/873635411 shutdown_connections 2023-10-06T21:15:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.366+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/873635411 wait complete. 2023-10-06T21:15:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.368+0000 7fe9cc7ff700 1 Processor -- start 2023-10-06T21:15:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.368+0000 7fe9cc7ff700 1 -- start start 2023-10-06T21:15:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410f620 0x7fe9c4114850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c411a690 0x7fe9c41ba490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41baad0 con 0x7fe9c410fa90 2023-10-06T21:15:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41bac40 con 0x7fe9c410f620 2023-10-06T21:15:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.369+0000 7fe9cc7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41bae00 con 0x7fe9c411a690 2023-10-06T21:15:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.370+0000 7fe9c9d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35588/0 (socket says 172.21.15.26:35588) 2023-10-06T21:15:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/975043385 learned_addr learned my addr 172.21.15.26:0/975043385 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410f620 0x7fe9c4114850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c411a690 0x7fe9c41ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/975043385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c411a690 msgr2=0x7fe9c41ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c411a690 0x7fe9c41ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/975043385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410f620 msgr2=0x7fe9c4114850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410f620 0x7fe9c4114850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.371+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/975043385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40095f0 con 0x7fe9c410fa90 2023-10-06T21:15:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.372+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00ada0 tx=0x7fe9bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.373+0000 7fe9bb7fe700 1 -- 172.21.15.26:0/975043385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00f070 con 0x7fe9c410fa90 2023-10-06T21:15:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.373+0000 7fe9bb7fe700 1 -- 172.21.15.26:0/975043385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc00ae90 con 0x7fe9c410fa90 2023-10-06T21:15:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.373+0000 7fe9bb7fe700 1 -- 172.21.15.26:0/975043385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0053e0 con 0x7fe9c410fa90 2023-10-06T21:15:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.373+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c41bb0e0 con 0x7fe9c410fa90 2023-10-06T21:15:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.373+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c41bb5a0 con 0x7fe9c410fa90 2023-10-06T21:15:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 Processor -- start 2023-10-06T21:15:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9bb7fe700 1 -- 172.21.15.26:0/975043385 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe9bc00c980 con 0x7fe9c410fa90 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 -- start start 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f09c00 0x5559e0f09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0e73400 con 0x5559e0f153d0 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0e64bc0 con 0x5559e0f09c00 2023-10-06T21:15:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.381+0000 7fe9e005db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0e624e0 con 0x5559e0f0a520 2023-10-06T21:15:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.382+0000 7fe9ca59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.382+0000 7fe9ca59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:48904/0 (socket says 172.21.15.26:48904) 2023-10-06T21:15:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.382+0000 7fe9ca59b700 1 -- 172.21.15.26:0/1349014653 learned_addr learned my addr 172.21.15.26:0/1349014653 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.382+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f09c00 0x5559e0f09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.382+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 -- 172.21.15.26:0/1349014653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f09c00 msgr2=0x5559e0f09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f09c00 0x5559e0f09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 -- 172.21.15.26:0/1349014653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 msgr2=0x5559e0f177c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f177c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 -- 172.21.15.26:0/1349014653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e0c88d80 con 0x5559e0f0a520 2023-10-06T21:15:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009a50 tx=0x7fe9c0009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d7f961370dbf53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9b8ff9700 1 -- 172.21.15.26:0/1349014653 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00095d0 con 0x5559e0f0a520 2023-10-06T21:15:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9b8ff9700 1 -- 172.21.15.26:0/1349014653 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c000ed90 con 0x5559e0f0a520 2023-10-06T21:15:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.383+0000 7fe9b8ff9700 1 -- 172.21.15.26:0/1349014653 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00123d0 con 0x5559e0f0a520 2023-10-06T21:15:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.384+0000 7fe9bb7fe700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 0x7fe9b00a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.384+0000 7fe9bb7fe700 1 -- 172.21.15.26:0/975043385 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe9bc0cc760 con 0x7fe9c410fa90 2023-10-06T21:15:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.384+0000 7fe9e005db80 1 -- 172.21.15.26:0/1349014653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 msgr2=0x5559e0f14e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.384+0000 7fe9e005db80 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009a50 tx=0x7fe9c0009d60 comp rx=0 tx=0).stop 2023-10-06T21:15:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.386+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 0x7fe9b00a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 -- 172.21.15.26:0/1349014653 shutdown_connections 2023-10-06T21:15:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f177c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f09c00 0x5559e0f09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 --2- 172.21.15.26:0/1349014653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f14e90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 -- 172.21.15.26:0/1349014653 >> 172.21.15.26:0/1349014653 conn(0x5559e0e7f200 msgr2=0x5559e0f06080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.387+0000 7fe9e005db80 1 -- 172.21.15.26:0/1349014653 shutdown_connections 2023-10-06T21:15:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.389+0000 7fe9e005db80 1 -- 172.21.15.26:0/1349014653 wait complete. 2023-10-06T21:15:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.390+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 0x7fe9b00a3c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4003800 tx=0x7fe9b4019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.390+0000 7fe9e005db80 1 -- start start 2023-10-06T21:15:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.390+0000 7fe9e005db80 1 Processor -- start 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f09c00 0x5559e0f14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0e5ddd0 con 0x5559e0f09c00 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0dfa6b0 con 0x5559e0f153d0 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0dfa350 con 0x5559e0f0a520 2023-10-06T21:15:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.391+0000 7fe9e005db80 10 client.0 _ll_register_callbacks cb 0x5559e0c577a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9cad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9cad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42176/0 (socket says 172.21.15.26:42176) 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9cad9c700 1 -- 172.21.15.26:0/2559384147 learned_addr learned my addr 172.21.15.26:0/2559384147 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f09c00 0x5559e0f14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 msgr2=0x5559e0f127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/2559384147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f09c00 msgr2=0x5559e0f14e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f09c00 0x5559e0f14e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.392+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/2559384147 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c0009730 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.393+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f120a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c9e60 tx=0x7fe9bc004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e66eeb3d610b5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.393+0000 7fe9adffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.393+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc099be0 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.393+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe9bc004ad0 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.393+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559e0efdd50 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.394+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc0192b0 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.394+0000 7fe9e005db80 10 client.26200 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.394+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5559e0c71c00 con 0x5559e0f0a520 2023-10-06T21:15:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.394+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559e0efdd50 con 0x5559e0f0a520 2023-10-06T21:15:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.394+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe9bc098c70 con 0x5559e0f0a520 2023-10-06T21:15:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 10 client.26200 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 20 client.26200 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f24e50 0x5559e0f27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 4 client.26200 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.395+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559e0e56db0 con 0x5559e0f24e50 2023-10-06T21:15:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.396+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f24e50 0x5559e0f27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.397+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f24e50 0x5559e0f27270 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a9e7995ea292e6 server_cookie=59c9882614ee2fae in_seq=0 out_seq=0 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.397+0000 7fe9adffb700 10 client.26200 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9adffb700 1 -- 172.21.15.26:0/2559384147 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559e0e56db0 con 0x5559e0f24e50 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9adffb700 10 client.26200 handle_command_reply: tid=0 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 client.26200 shutdown 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 msgr2=0x5559e0f120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f120a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c9e60 tx=0x7fe9bc004dc0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 shutdown_connections 2023-10-06T21:15:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f24e50 0x5559e0f27270 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f09c00 0x5559e0f14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2559384147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f0a520 0x5559e0f120a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 >> 172.21.15.26:0/2559384147 conn(0x5559e0e7f200 msgr2=0x5559e0f05c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.398+0000 7fe9adffb700 0 client.26200 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.399+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 shutdown_connections 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.399+0000 7fe9e005db80 1 -- 172.21.15.26:0/2559384147 wait complete. 2023-10-06T21:15:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.399+0000 7fe9e005db80 20 client.26200 trim_cache size 0 max 16384 2023-10-06T21:15:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.424+0000 7fe9e005db80 1 Processor -- start 2023-10-06T21:15:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.424+0000 7fe9e005db80 1 -- start start 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 0x5559e0f14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f0d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0cde8c0 con 0x5559e0f0a520 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0f0df10 con 0x5559e0f153d0 2023-10-06T21:15:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.425+0000 7fe9e005db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0efdbb0 con 0x5559e0f09c00 2023-10-06T21:15:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:48924/0 (socket says 172.21.15.26:48924) 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 -- 172.21.15.26:0/2742525137 learned_addr learned my addr 172.21.15.26:0/2742525137 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f0d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 -- 172.21.15.26:0/2742525137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 msgr2=0x5559e0f0d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f0d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 0x5559e0f14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 -- 172.21.15.26:0/2742525137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 msgr2=0x5559e0f14950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 0x5559e0f14950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9ca59b700 1 -- 172.21.15.26:0/2742525137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559e0caf1f0 con 0x5559e0f09c00 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.426+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 0x5559e0f14950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.427+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009a20 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7321ebdd23820339 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.427+0000 7fe9b97fa700 1 -- 172.21.15.26:0/2742525137 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00255a0 con 0x5559e0f09c00 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.427+0000 7fe9b97fa700 1 -- 172.21.15.26:0/2742525137 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c0013b00 con 0x5559e0f09c00 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.427+0000 7fe9b97fa700 1 -- 172.21.15.26:0/2742525137 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0025ae0 con 0x5559e0f09c00 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.428+0000 7fe9e005db80 1 -- 172.21.15.26:0/2742525137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 msgr2=0x5559e0f1bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.428+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009a20 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 -- 172.21.15.26:0/2742525137 shutdown_connections 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f153d0 0x5559e0f0d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f1bf10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 --2- 172.21.15.26:0/2742525137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f0a520 0x5559e0f14950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 -- 172.21.15.26:0/2742525137 >> 172.21.15.26:0/2742525137 conn(0x5559e0e7f200 msgr2=0x5559e0f20670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 -- 172.21.15.26:0/2742525137 shutdown_connections 2023-10-06T21:15:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.429+0000 7fe9e005db80 1 -- 172.21.15.26:0/2742525137 wait complete. 2023-10-06T21:15:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.430+0000 7fe9e005db80 1 -- start start 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.430+0000 7fe9e005db80 1 Processor -- start 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f0f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f0e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0f0e860 con 0x5559e0f153d0 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0f0e9d0 con 0x5559e0f0a520 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559e0f0eb90 con 0x5559e0f09c00 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9e005db80 10 client.0 _ll_register_callbacks cb 0x5559e0c577a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9c9d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9ca59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f0f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9c9d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42198/0 (socket says 172.21.15.26:42198) 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/3016965366 learned_addr learned my addr 172.21.15.26:0/3016965366 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.431+0000 7fe9cad9c700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f0e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.432+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/3016965366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 msgr2=0x5559e0f0f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.432+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f0f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.432+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/3016965366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 msgr2=0x5559e0f0e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.432+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f0e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.432+0000 7fe9c9d9a700 1 -- 172.21.15.26:0/3016965366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559e0c88d80 con 0x5559e0f0a520 2023-10-06T21:15:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.434+0000 7fe9c9d9a700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00a0a0 tx=0x7fe9bc0cef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fc8417fc992f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.434+0000 7fe9adffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.434+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0992c0 con 0x5559e0f0a520 2023-10-06T21:15:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.434+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe9bc00ce40 con 0x5559e0f0a520 2023-10-06T21:15:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.435+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c000ed90 con 0x5559e0f0a520 2023-10-06T21:15:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.435+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc004ad0 con 0x5559e0f0a520 2023-10-06T21:15:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 10 client.26228 fetch_fsmap learned FSMap version 37 2023-10-06T21:15:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5559e0c87250 con 0x5559e0f0a520 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c000ed90 con 0x5559e0f0a520 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe9bc0b5050 con 0x5559e0f0a520 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 10 client.26228 fetch_fsmap finished waiting for FSMap version 37 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 20 client.26228 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f796d0 0x5559e0f7baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 4 client.26228 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:15:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.436+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559e0e56ad0 con 0x5559e0f796d0 2023-10-06T21:15:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.437+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f796d0 0x5559e0f7baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.437+0000 7fe9ca59b700 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f796d0 0x5559e0f7baf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0931660cf0d127 server_cookie=5bbf476366d3f1f8 in_seq=0 out_seq=0 2023-10-06T21:15:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.437+0000 7fe9adffb700 10 client.26228 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.457+0000 7fe9adffb700 1 -- 172.21.15.26:0/3016965366 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559e0e56ad0 con 0x5559e0f796d0 2023-10-06T21:15:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.457+0000 7fe9adffb700 10 client.26228 handle_command_reply: tid=0 2023-10-06T21:15:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.458+0000 7fe9e005db80 1 client.26228 shutdown 2023-10-06T21:15:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 msgr2=0x5559e0f0fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00a0a0 tx=0x7fe9bc0cef10 comp rx=0 tx=0).stop 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 shutdown_connections 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5559e0f153d0 0x5559e0f0e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559e0f09c00 0x5559e0f0f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5559e0f796d0 0x5559e0f7baf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 --2- 172.21.15.26:0/3016965366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559e0f0a520 0x5559e0f0fad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 >> 172.21.15.26:0/3016965366 conn(0x5559e0e7f200 msgr2=0x5559e0f0d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9adffb700 0 client.26228 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 shutdown_connections 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.459+0000 7fe9e005db80 1 -- 172.21.15.26:0/3016965366 wait complete. 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.460+0000 7fe9e005db80 20 client.26228 trim_cache size 0 max 16384 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:15:43.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:15:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.461+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 msgr2=0x7fe9b00a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.461+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 0x7fe9b00a3c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4003800 tx=0x7fe9b4019040 comp rx=0 tx=0).stop 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.461+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 msgr2=0x7fe9c4114d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.461+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00ada0 tx=0x7fe9bc00c330 comp rx=0 tx=0).stop 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 shutdown_connections 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9c411a690 0x7fe9c41ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c410f620 0x7fe9c4114850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe9b00a17c0 0x7fe9b00a3c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 --2- 172.21.15.26:0/975043385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9c410fa90 0x7fe9c4114d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.463+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 >> 172.21.15.26:0/975043385 conn(0x7fe9c406cfa0 msgr2=0x7fe9c40718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.464+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 shutdown_connections 2023-10-06T21:15:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:43.464+0000 7fe9cc7ff700 1 -- 172.21.15.26:0/975043385 wait complete. 2023-10-06T21:15:44.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:44 smithi167 ceph-mon[134494]: pgmap v166: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 720 op/s 2023-10-06T21:15:44.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:44 smithi026 ceph-mon[98490]: pgmap v166: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 720 op/s 2023-10-06T21:15:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:44 smithi078 ceph-mon[134896]: pgmap v166: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 MiB/s wr, 720 op/s 2023-10-06T21:15:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:46 smithi167 ceph-mon[134494]: pgmap v167: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 956 op/s 2023-10-06T21:15:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:46 smithi026 ceph-mon[98490]: pgmap v167: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 956 op/s 2023-10-06T21:15:46.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:46 smithi078 ceph-mon[134896]: pgmap v167: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 956 op/s 2023-10-06T21:15:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 305.0 s, 1694.4 tps, lat 59.240 ms stddev 65.879 2023-10-06T21:15:48.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:48 smithi167 ceph-mon[134494]: pgmap v168: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 802 op/s 2023-10-06T21:15:48.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:48 smithi026 ceph-mon[98490]: pgmap v168: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 802 op/s 2023-10-06T21:15:48.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:48 smithi078 ceph-mon[134896]: pgmap v168: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 18 MiB/s wr, 802 op/s 2023-10-06T21:15:50.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:50 smithi026 ceph-mon[98490]: pgmap v169: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 656 op/s 2023-10-06T21:15:50.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:50 smithi078 ceph-mon[134896]: pgmap v169: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 656 op/s 2023-10-06T21:15:50.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:50 smithi167 ceph-mon[134494]: pgmap v169: 97 pgs: 97 active+clean; 6.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 656 op/s 2023-10-06T21:15:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 310.0 s, 1361.0 tps, lat 73.103 ms stddev 121.283 2023-10-06T21:15:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:52 smithi078 ceph-mon[134896]: pgmap v170: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 2.18k op/s 2023-10-06T21:15:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:52 smithi167 ceph-mon[134494]: pgmap v170: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 2.18k op/s 2023-10-06T21:15:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:52 smithi026 ceph-mon[98490]: pgmap v170: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 2.18k op/s 2023-10-06T21:15:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:54 smithi078 ceph-mon[134896]: pgmap v171: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-06T21:15:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:54 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:54.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:54 smithi167 ceph-mon[134494]: pgmap v171: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-06T21:15:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:54 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:54 smithi026 ceph-mon[98490]: pgmap v171: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.86k op/s 2023-10-06T21:15:54.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:54 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:56.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:15:56.683 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:56 smithi026 ceph-mon[98490]: pgmap v172: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 39 MiB/s wr, 2.06k op/s 2023-10-06T21:15:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 -- 172.21.15.26:0/2193705364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 msgr2=0x7f40281014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 --2- 172.21.15.26:0/2193705364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281014a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f401c009990 tx=0x7f401c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 -- 172.21.15.26:0/2193705364 shutdown_connections 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 --2- 172.21.15.26:0/2193705364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f402810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 --2- 172.21.15.26:0/2193705364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281014a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 --2- 172.21.15.26:0/2193705364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028101a70 0x7f4028105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.705+0000 7f402edff700 1 -- 172.21.15.26:0/2193705364 >> 172.21.15.26:0/2193705364 conn(0x7f40280fc940 msgr2=0x7f40280fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.706+0000 7f402edff700 1 -- 172.21.15.26:0/2193705364 shutdown_connections 2023-10-06T21:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.706+0000 7f402edff700 1 -- 172.21.15.26:0/2193705364 wait complete. 2023-10-06T21:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.708+0000 7f402edff700 1 Processor -- start 2023-10-06T21:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.708+0000 7f402edff700 1 -- start start 2023-10-06T21:15:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f40281a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402810caa0 0x7f402810cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402810d520 con 0x7f402810caa0 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402810d660 con 0x7f40281010c0 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402edff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402810d820 con 0x7f40281061e0 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56222/0 (socket says 172.21.15.26:56222) 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.709+0000 7f402cb9b700 1 -- 172.21.15.26:0/1881701172 learned_addr learned my addr 172.21.15.26:0/1881701172 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f4027fff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f40281a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402d39c700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402810caa0 0x7f402810cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 -- 172.21.15.26:0/1881701172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 msgr2=0x7f40281a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f40281a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 -- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402810caa0 msgr2=0x7f402810cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402810caa0 0x7f402810cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 -- 172.21.15.26:0/1881701172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0095f0 con 0x7f40281010c0 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f4027fff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f40281a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.710+0000 7f402cb9b700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f401c005750 tx=0x7f401c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.712+0000 7f4025ffb700 1 -- 172.21.15.26:0/1881701172 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c01d070 con 0x7f40281010c0 2023-10-06T21:15:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.712+0000 7f4025ffb700 1 -- 172.21.15.26:0/1881701172 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c022470 con 0x7f40281010c0 2023-10-06T21:15:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.712+0000 7f4025ffb700 1 -- 172.21.15.26:0/1881701172 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c017650 con 0x7f40281010c0 2023-10-06T21:15:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.712+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40281aa3a0 con 0x7f40281010c0 2023-10-06T21:15:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.712+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281aa830 con 0x7f40281010c0 2023-10-06T21:15:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.719+0000 7f4042761b80 1 Processor -- start 2023-10-06T21:15:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.719+0000 7f4042761b80 1 -- start start 2023-10-06T21:15:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df4404fc00 0x55df4404ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 0x55df4405ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 0x55df4405d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43fb9400 con 0x55df44050520 2023-10-06T21:15:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43faabc0 con 0x55df4404fc00 2023-10-06T21:15:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4042761b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43fa84e0 con 0x55df4405b3d0 2023-10-06T21:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4027fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 0x55df4405d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4027fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 0x55df4405d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2023-10-06T21:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.720+0000 7f4027fff700 1 -- 172.21.15.26:0/4020553444 learned_addr learned my addr 172.21.15.26:0/4020553444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.721+0000 7f402d39c700 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df4404fc00 0x55df4404ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.722+0000 7f4025ffb700 1 -- 172.21.15.26:0/1881701172 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f401c00fb40 con 0x7f40281010c0 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.722+0000 7f402cb9b700 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 0x55df4405ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 -- 172.21.15.26:0/4020553444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 msgr2=0x55df4405d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 0x55df4405d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 -- 172.21.15.26:0/4020553444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df4404fc00 msgr2=0x55df4404ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df4404fc00 0x55df4404ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 -- 172.21.15.26:0/4020553444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df43dced80 con 0x55df44050520 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.723+0000 7f402cb9b700 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 0x55df4405ae90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f401c0cb7e0 tx=0x7f401c0d63c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a135bd49197fdfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.724+0000 7f4012ffd700 1 -- 172.21.15.26:0/4020553444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0d6ce0 con 0x55df44050520 2023-10-06T21:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.724+0000 7f4012ffd700 1 -- 172.21.15.26:0/4020553444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c0d6e40 con 0x55df44050520 2023-10-06T21:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.724+0000 7f4012ffd700 1 -- 172.21.15.26:0/4020553444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0d0e20 con 0x55df44050520 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.725+0000 7f4025ffb700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 0x7f40180a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.725+0000 7f4025ffb700 1 -- 172.21.15.26:0/1881701172 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f401c0ce0e0 con 0x7f40281010c0 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.725+0000 7f402d39c700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 0x7f40180a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.725+0000 7f4042761b80 1 -- 172.21.15.26:0/4020553444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 msgr2=0x55df4405ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.725+0000 7f4042761b80 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 0x55df4405ae90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f401c0cb7e0 tx=0x7f401c0d63c0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.726+0000 7f4042761b80 1 -- 172.21.15.26:0/4020553444 shutdown_connections 2023-10-06T21:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.726+0000 7f4042761b80 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4405b3d0 0x55df4405d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.726+0000 7f4042761b80 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df4404fc00 0x55df4404ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.726+0000 7f4042761b80 1 --2- 172.21.15.26:0/4020553444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df44050520 0x55df4405ae90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.726+0000 7f4042761b80 1 -- 172.21.15.26:0/4020553444 >> 172.21.15.26:0/4020553444 conn(0x55df43fc5200 msgr2=0x55df4404c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.728+0000 7f4042761b80 1 -- 172.21.15.26:0/4020553444 shutdown_connections 2023-10-06T21:15:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.728+0000 7f4042761b80 1 -- 172.21.15.26:0/4020553444 wait complete. 2023-10-06T21:15:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.729+0000 7f4042761b80 1 -- start start 2023-10-06T21:15:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.729+0000 7f402d39c700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 0x7f40180a3c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4020009420 tx=0x7f4020009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.729+0000 7f4042761b80 1 Processor -- start 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df4405b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df44066830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43fa3dd0 con 0x55df4405b3d0 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43f406b0 con 0x55df44050520 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43f40350 con 0x55df4404fc00 2023-10-06T21:15:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4042761b80 10 client.0 _ll_register_callbacks cb 0x55df43d9d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4027fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4027fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56246/0 (socket says 172.21.15.26:56246) 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f4027fff700 1 -- 172.21.15.26:0/3030374112 learned_addr learned my addr 172.21.15.26:0/3030374112 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.730+0000 7f402cb9b700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df4405b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402d39c700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df44066830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402cb9b700 1 -- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 msgr2=0x55df4405bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402cb9b700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402cb9b700 1 -- 172.21.15.26:0/3030374112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 msgr2=0x55df44066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402cb9b700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df44066830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.731+0000 7f402cb9b700 1 -- 172.21.15.26:0/3030374112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281aa830 con 0x55df4404fc00 2023-10-06T21:15:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.732+0000 7f402cb9b700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df4405b7b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f401c097b70 tx=0x7f401c0d1920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d98c041388b84a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.732+0000 7f4007fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:15:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.732+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0ce2f0 con 0x55df4404fc00 2023-10-06T21:15:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.732+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f401c0d0320 con 0x55df4404fc00 2023-10-06T21:15:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.732+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df44043d50 con 0x55df4404fc00 2023-10-06T21:15:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.733+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c097440 con 0x55df4404fc00 2023-10-06T21:15:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.733+0000 7f4042761b80 10 client.26215 fetch_fsmap learned FSMap version 38 2023-10-06T21:15:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.733+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55df43db7c00 con 0x55df4404fc00 2023-10-06T21:15:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df44043d50 con 0x55df4404fc00 2023-10-06T21:15:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f401c097810 con 0x55df4404fc00 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 20 client.26215 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df4406a6a0 0x55df4406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 4 client.26215 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.735+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df43f9cdb0 con 0x55df4406a6a0 2023-10-06T21:15:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.736+0000 7f4027fff700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df4406a6a0 0x55df4406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.736+0000 7f4007fff700 10 client.26215 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.736+0000 7f4027fff700 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df4406a6a0 0x55df4406ca90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b787be8f851a2a server_cookie=27fd53f1b12f84a0 in_seq=0 out_seq=0 2023-10-06T21:15:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.737+0000 7f4007fff700 1 -- 172.21.15.26:0/3030374112 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df43f9cdb0 con 0x55df4406a6a0 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.737+0000 7f4007fff700 10 client.26215 handle_command_reply: tid=0 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.737+0000 7f4042761b80 1 client.26215 shutdown 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.737+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 msgr2=0x55df4405b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.737+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df4405b7b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f401c097b70 tx=0x7f401c0d1920 comp rx=0 tx=0).stop 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 shutdown_connections 2023-10-06T21:15:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df44066830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df4405b7b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df4406a6a0 0x55df4406ca90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 --2- 172.21.15.26:0/3030374112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 >> 172.21.15.26:0/3030374112 conn(0x55df43fc5200 msgr2=0x55df4404bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.738+0000 7f4007fff700 0 client.26215 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.739+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 shutdown_connections 2023-10-06T21:15:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.739+0000 7f4042761b80 1 -- 172.21.15.26:0/3030374112 wait complete. 2023-10-06T21:15:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.739+0000 7f4042761b80 20 client.26215 trim_cache size 0 max 16384 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.759+0000 7f4042761b80 1 Processor -- start 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.759+0000 7f4042761b80 1 -- start start 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43e248c0 con 0x55df4405b3d0 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4406cdc0 con 0x55df44050520 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4042761b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df44056310 con 0x55df4404fc00 2023-10-06T21:15:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4027fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f402d39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:58640/0 (socket says 172.21.15.26:58640) 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f4027fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56248/0 (socket says 172.21.15.26:56248) 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.760+0000 7f402cb9b700 1 -- 172.21.15.26:0/2008257717 learned_addr learned my addr 172.21.15.26:0/2008257717 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 -- 172.21.15.26:0/2008257717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 msgr2=0x55df4405b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 -- 172.21.15.26:0/2008257717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 msgr2=0x55df4405bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 -- 172.21.15.26:0/2008257717 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df43df51f0 con 0x55df4404fc00 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.761+0000 7f402cb9b700 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f401c09ae00 tx=0x7f401c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e330d277a7d3b838 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.763+0000 7f40137fe700 1 -- 172.21.15.26:0/2008257717 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0d68c0 con 0x55df4404fc00 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.763+0000 7f40137fe700 1 -- 172.21.15.26:0/2008257717 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c09aed0 con 0x55df4404fc00 2023-10-06T21:15:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.763+0000 7f40137fe700 1 -- 172.21.15.26:0/2008257717 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0225e0 con 0x55df4404fc00 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 -- 172.21.15.26:0/2008257717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 msgr2=0x55df44054c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f401c09ae00 tx=0x7f401c022c90 comp rx=0 tx=0).stop 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 -- 172.21.15.26:0/2008257717 shutdown_connections 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44054c20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 --2- 172.21.15.26:0/2008257717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 -- 172.21.15.26:0/2008257717 >> 172.21.15.26:0/2008257717 conn(0x55df43fc5200 msgr2=0x55df4406c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 -- 172.21.15.26:0/2008257717 shutdown_connections 2023-10-06T21:15:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.764+0000 7f4042761b80 1 -- 172.21.15.26:0/2008257717 wait complete. 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 -- start start 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 Processor -- start 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44060750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4405e650 con 0x55df4405b3d0 2023-10-06T21:15:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4405e7c0 con 0x55df44050520 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.765+0000 7f4042761b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df440bd4a0 con 0x55df4404fc00 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4042761b80 10 client.0 _ll_register_callbacks cb 0x7f401c0d1df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44060750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f402cb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44060750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:58642/0 (socket says 172.21.15.26:58642) 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f402cb9b700 1 -- 172.21.15.26:0/1178628228 learned_addr learned my addr 172.21.15.26:0/1178628228 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f402d39c700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 -- 172.21.15.26:0/1178628228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 msgr2=0x55df44060750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44060750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 -- 172.21.15.26:0/1178628228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 msgr2=0x55df4405df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 -- 172.21.15.26:0/1178628228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df43db7c00 con 0x55df44050520 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4027fff700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405da00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55df4406d340 tx=0x7f401401dd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf23b4172595d6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.766+0000 7f4007fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014017040 con 0x55df44050520 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f401c0d6e40 con 0x55df44050520 2023-10-06T21:15:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4014003680 con 0x55df44050520 2023-10-06T21:15:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4014003860 con 0x55df44050520 2023-10-06T21:15:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4042761b80 10 client.26246 fetch_fsmap learned FSMap version 38 2023-10-06T21:15:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55df43dcd250 con 0x55df44050520 2023-10-06T21:15:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.767+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c0d6e40 con 0x55df44050520 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f401402c070 con 0x55df44050520 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 20 client.26246 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df440beaa0 0x55df440c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 4 client.26246 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.769+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df43f9cad0 con 0x55df440beaa0 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.770+0000 7f402cb9b700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df440beaa0 0x55df440c0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.771+0000 7f402cb9b700 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df440beaa0 0x55df440c0ec0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e391839340ad57 server_cookie=aab3f7ad6fc61cf1 in_seq=0 out_seq=0 2023-10-06T21:15:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.772+0000 7f4007fff700 10 client.26246 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:56 smithi167 ceph-mon[134494]: pgmap v172: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 39 MiB/s wr, 2.06k op/s 2023-10-06T21:15:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.852+0000 7f4007fff700 1 -- 172.21.15.26:0/1178628228 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df43f9cad0 con 0x55df440beaa0 2023-10-06T21:15:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.852+0000 7f4007fff700 10 client.26246 handle_command_reply: tid=0 2023-10-06T21:15:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.852+0000 7f4042761b80 1 client.26246 shutdown 2023-10-06T21:15:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 msgr2=0x55df4405da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405da00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55df4406d340 tx=0x7f401401dd30 comp rx=0 tx=0).stop 2023-10-06T21:15:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 shutdown_connections 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55df4405b3d0 0x55df4405df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df4404fc00 0x55df44060750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55df440beaa0 0x55df440c0ec0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 --2- 172.21.15.26:0/1178628228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55df44050520 0x55df4405da00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 >> 172.21.15.26:0/1178628228 conn(0x55df43fc5200 msgr2=0x55df44055780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4007fff700 0 client.26246 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 shutdown_connections 2023-10-06T21:15:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.853+0000 7f4042761b80 1 -- 172.21.15.26:0/1178628228 wait complete. 2023-10-06T21:15:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.854+0000 7f4042761b80 20 client.26246 trim_cache size 0 max 16384 2023-10-06T21:15:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:15:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:15:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:15:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:15:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.854+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 msgr2=0x7f40180a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.854+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 0x7f40180a3c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4020009420 tx=0x7f4020009040 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.854+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 msgr2=0x7f40281a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.854+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f401c005750 tx=0x7f401c005ef0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 shutdown_connections 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40180a17c0 0x7f40180a3c80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402810caa0 0x7f402810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40281010c0 0x7f40281a3590 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 --2- 172.21.15.26:0/1881701172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40281061e0 0x7f40281a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 >> 172.21.15.26:0/1881701172 conn(0x7f40280fc940 msgr2=0x7f4028102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 shutdown_connections 2023-10-06T21:15:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:15:56.855+0000 7f402edff700 1 -- 172.21.15.26:0/1881701172 wait complete. 2023-10-06T21:15:56.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:56 smithi078 ceph-mon[134896]: pgmap v172: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 39 MiB/s wr, 2.06k op/s 2023-10-06T21:15:56.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/2538533935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 msgr2=0x7f7bd81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/2538533935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd81036c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b0a0 tx=0x7f7bc800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/2538533935 shutdown_connections 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/2538533935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 0x7f7bd810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/2538533935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd81036c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/2538533935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd8103c90 0x7f7bd8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.921+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/2538533935 >> 172.21.15.26:0/2538533935 conn(0x7f7bd80feb60 msgr2=0x7f7bd8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.922+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/2538533935 shutdown_connections 2023-10-06T21:15:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.922+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/2538533935 wait complete. 2023-10-06T21:15:56.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.923+0000 7f7bdd7ff700 1 Processor -- start 2023-10-06T21:15:56.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 -- start start 2023-10-06T21:15:56.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 0x7f7bd810ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 0x7f7bd810cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd810d7b0 con 0x7f7bd81100d0 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81a9e50 con 0x7f7bd81032e0 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.924+0000 7f7bdd7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81a9f90 con 0x7f7bd8108400 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bce7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 0x7f7bd810ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56264/0 (socket says 172.21.15.26:56264) 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/746228869 learned_addr learned my addr 172.21.15.26:0/746228869 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 0x7f7bd810cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/746228869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 msgr2=0x7f7bd810ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 0x7f7bd810ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 msgr2=0x7f7bd810cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 0x7f7bd810cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/746228869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009d00 con 0x7f7bd81032e0 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 0x7f7bd810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:56.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b070 tx=0x7f7bc80077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd4ff9700 1 -- 172.21.15.26:0/746228869 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e040 con 0x7f7bd81032e0 2023-10-06T21:15:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.925+0000 7f7bd4ff9700 1 -- 172.21.15.26:0/746228869 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc8003c10 con 0x7f7bd81032e0 2023-10-06T21:15:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.926+0000 7f7bd4ff9700 1 -- 172.21.15.26:0/746228869 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801b6c0 con 0x7f7bd81032e0 2023-10-06T21:15:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.926+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd81aa190 con 0x7f7bd81032e0 2023-10-06T21:15:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.926+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd81aa560 con 0x7f7bd81032e0 2023-10-06T21:15:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.932+0000 7f7bd4ff9700 1 -- 172.21.15.26:0/746228869 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7bc801d470 con 0x7f7bd81032e0 2023-10-06T21:15:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.933+0000 7f7bf115eb80 1 Processor -- start 2023-10-06T21:15:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 -- start start 2023-10-06T21:15:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bd4ff9700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 0x7f7bc40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f69154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6922c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f687e460 con 0x5561f6920890 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f686fc20 con 0x5561f69150c0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.934+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f686d540 con 0x5561f69159e0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bd4ff9700 1 -- 172.21.15.26:0/746228869 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7bc80ce3c0 con 0x7f7bd81032e0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bce7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 0x7f7bc40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bce7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:58666/0 (socket says 172.21.15.26:58666) 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bce7fc700 1 -- 172.21.15.26:0/933642549 learned_addr learned my addr 172.21.15.26:0/933642549 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.935+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6922c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 -- 172.21.15.26:0/933642549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 msgr2=0x5561f69154a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f69154a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 -- 172.21.15.26:0/933642549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 msgr2=0x5561f6922c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6922c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 -- 172.21.15.26:0/933642549 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f6693de0 con 0x5561f69159e0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000bcb0 tx=0x7f7bc00103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5e2c3299d73d78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 0x7f7bc40a3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009fd0 tx=0x7f7bd0009aa0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bcd7fa700 1 -- 172.21.15.26:0/933642549 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0010d10 con 0x5561f69159e0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.936+0000 7f7bcd7fa700 1 -- 172.21.15.26:0/933642549 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc0015070 con 0x5561f69159e0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bcd7fa700 1 -- 172.21.15.26:0/933642549 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0005020 con 0x5561f69159e0 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 -- 172.21.15.26:0/933642549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 msgr2=0x5561f6920350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000bcb0 tx=0x7f7bc00103d0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 -- 172.21.15.26:0/933642549 shutdown_connections 2023-10-06T21:15:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6922c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f69154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/933642549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f6920350 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.937+0000 7f7bf115eb80 1 -- 172.21.15.26:0/933642549 >> 172.21.15.26:0/933642549 conn(0x5561f688a260 msgr2=0x5561f6911450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.938+0000 7f7bf115eb80 1 -- 172.21.15.26:0/933642549 shutdown_connections 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.938+0000 7f7bf115eb80 1 -- 172.21.15.26:0/933642549 wait complete. 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.938+0000 7f7bf115eb80 1 -- start start 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.939+0000 7f7bf115eb80 1 Processor -- start 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69150c0 0x5561f691d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 0x5561f691db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f6868e30 con 0x5561f6920890 2023-10-06T21:15:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f6805710 con 0x5561f69159e0 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f68053b0 con 0x5561f69150c0 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.940+0000 7f7bf115eb80 10 client.0 _ll_register_callbacks cb 0x5561f6662800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58096/0 (socket says 172.21.15.26:58096) 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 -- 172.21.15.26:0/3309114366 learned_addr learned my addr 172.21.15.26:0/3309114366 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 0x5561f691db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 -- 172.21.15.26:0/3309114366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69150c0 msgr2=0x5561f691d650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69150c0 0x5561f691d650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 -- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 msgr2=0x5561f691db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 0x5561f691db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bd77fe700 1 -- 172.21.15.26:0/3309114366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc000b8e0 con 0x5561f6920890 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.941+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 0x5561f691db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0011d00 tx=0x7f7bd0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9059ab4f9c9a23a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004220 con 0x5561f6920890 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7bd0004380 con 0x5561f6920890 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561f6908f10 con 0x5561f6920890 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bd00142e0 con 0x5561f6920890 2023-10-06T21:15:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bf115eb80 10 client.16032 fetch_fsmap learned FSMap version 38 2023-10-06T21:15:56.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.942+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5561f667cc60 con 0x5561f6920890 2023-10-06T21:15:56.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561f6908f10 con 0x5561f6920890 2023-10-06T21:15:56.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7bd000f960 con 0x5561f6920890 2023-10-06T21:15:56.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 10 client.16032 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 20 client.16032 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69414b0 0x5561f69438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 4 client.16032 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.943+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561f6861e10 con 0x5561f69414b0 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.944+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69414b0 0x5561f69438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.945+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69414b0 0x5561f69438d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331f574e50bd1fee server_cookie=7f5f13d5f8a408c1 in_seq=0 out_seq=0 2023-10-06T21:15:56.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.945+0000 7f7bbe7fc700 10 client.16032 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.946+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/3309114366 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561f6861e10 con 0x5561f69414b0 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.946+0000 7f7bbe7fc700 10 client.16032 handle_command_reply: tid=0 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.946+0000 7f7bf115eb80 1 client.16032 shutdown 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 msgr2=0x5561f691e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0011d00 tx=0x7f7bd0009260 comp rx=0 tx=0).stop 2023-10-06T21:15:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 shutdown_connections 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691e0d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69150c0 0x5561f691d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69414b0 0x5561f69438d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/3309114366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69159e0 0x5561f691db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 >> 172.21.15.26:0/3309114366 conn(0x5561f688a260 msgr2=0x5561f6910fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bbe7fc700 0 client.16032 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.947+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 shutdown_connections 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.948+0000 7f7bf115eb80 1 -- 172.21.15.26:0/3309114366 wait complete. 2023-10-06T21:15:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.948+0000 7f7bf115eb80 20 client.16032 trim_cache size 0 max 16384 2023-10-06T21:15:56.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 Processor -- start 2023-10-06T21:15:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 -- start start 2023-10-06T21:15:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f691df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f66e9920 con 0x5561f6920890 2023-10-06T21:15:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f6920730 con 0x5561f69150c0 2023-10-06T21:15:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.978+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f693eaf0 con 0x5561f69159e0 2023-10-06T21:15:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56298/0 (socket says 172.21.15.26:56298) 2023-10-06T21:15:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/698729279 learned_addr learned my addr 172.21.15.26:0/698729279 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f691df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd77fe700 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/698729279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 msgr2=0x5561f691df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f691df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/698729279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 msgr2=0x5561f691a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.979+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/698729279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f66ba250 con 0x5561f69150c0 2023-10-06T21:15:56.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.980+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80cc120 tx=0x7f7bc8009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44fc751820e66e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.981+0000 7f7bcdffb700 1 -- 172.21.15.26:0/698729279 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc809b440 con 0x5561f69150c0 2023-10-06T21:15:56.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.981+0000 7f7bcdffb700 1 -- 172.21.15.26:0/698729279 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc809f590 con 0x5561f69150c0 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.981+0000 7f7bcdffb700 1 -- 172.21.15.26:0/698729279 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc809f170 con 0x5561f69150c0 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 -- 172.21.15.26:0/698729279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 msgr2=0x5561f6917f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80cc120 tx=0x7f7bc8009a70 comp rx=0 tx=0).stop 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 -- 172.21.15.26:0/698729279 shutdown_connections 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f691a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6917f60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.982+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/698729279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f691df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.983+0000 7f7bf115eb80 1 -- 172.21.15.26:0/698729279 >> 172.21.15.26:0/698729279 conn(0x5561f688a260 msgr2=0x5561f691c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:56.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.983+0000 7f7bf115eb80 1 -- 172.21.15.26:0/698729279 shutdown_connections 2023-10-06T21:15:56.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.983+0000 7f7bf115eb80 1 -- 172.21.15.26:0/698729279 wait complete. 2023-10-06T21:15:56.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.983+0000 7f7bf115eb80 1 -- start start 2023-10-06T21:15:56.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.984+0000 7f7bf115eb80 1 Processor -- start 2023-10-06T21:15:56.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f69256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6925c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f69262d0 con 0x5561f6920890 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f6926440 con 0x5561f69150c0 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f69952f0 con 0x5561f69159e0 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bf115eb80 10 client.0 _ll_register_callbacks cb 0x7f7bd00118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.985+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:56.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56302/0 (socket says 172.21.15.26:56302) 2023-10-06T21:15:56.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/874935859 learned_addr learned my addr 172.21.15.26:0/874935859 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:15:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/874935859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 msgr2=0x5561f69256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f69256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/874935859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 msgr2=0x5561f6925c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:15:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6925c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:56.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 -- 172.21.15.26:0/874935859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc80b3e70 con 0x5561f69150c0 2023-10-06T21:15:56.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bd6ffd700 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80a3210 tx=0x7f7bc80a3860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb1297e7d75906d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:15:56.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.986+0000 7f7bbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:15:56.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.987+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8097b50 con 0x5561f69150c0 2023-10-06T21:15:56.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.987+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7bc80b0070 con 0x5561f69150c0 2023-10-06T21:15:56.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:56.987+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bc0015070 con 0x5561f69150c0 2023-10-06T21:15:57.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.003+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc8097cb0 con 0x5561f69150c0 2023-10-06T21:15:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.005+0000 7f7bf115eb80 10 client.26258 fetch_fsmap learned FSMap version 38 2023-10-06T21:15:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.005+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5561f66922b0 con 0x5561f69150c0 2023-10-06T21:15:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.008+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc0015070 con 0x5561f69150c0 2023-10-06T21:15:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.008+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 38) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7bc809a930 con 0x5561f69150c0 2023-10-06T21:15:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 10 client.26258 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T21:15:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:15:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 20 client.26258 populate_metadata read hostname 'smithi026' 2023-10-06T21:15:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69964d0 0x5561f69988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:15:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 4 client.26258 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:15:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.009+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561f6861b30 con 0x5561f69964d0 2023-10-06T21:15:57.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.014+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69964d0 0x5561f69988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:15:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.017+0000 7f7bce7fc700 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69964d0 0x5561f69988f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46974123ce41d29 server_cookie=98eedd50f04ed4d4 in_seq=0 out_seq=0 2023-10-06T21:15:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.017+0000 7f7bbe7fc700 10 client.26258 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.017+0000 7f7bbe7fc700 1 -- 172.21.15.26:0/874935859 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5561f6861b30 con 0x5561f69964d0 2023-10-06T21:15:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.017+0000 7f7bbe7fc700 10 client.26258 handle_command_reply: tid=0 2023-10-06T21:15:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.018+0000 7f7bf115eb80 1 client.26258 shutdown 2023-10-06T21:15:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.019+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 msgr2=0x5561f6927390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.019+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80a3210 tx=0x7f7bc80a3860 comp rx=0 tx=0).stop 2023-10-06T21:15:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 shutdown_connections 2023-10-06T21:15:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5561f6920890 0x5561f6925c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5561f69964d0 0x5561f69988f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561f69150c0 0x5561f6927390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 --2- 172.21.15.26:0/874935859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561f69159e0 0x5561f69256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 >> 172.21.15.26:0/874935859 conn(0x5561f688a260 msgr2=0x5561f691de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bbe7fc700 0 client.26258 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:15:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.022+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 shutdown_connections 2023-10-06T21:15:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.023+0000 7f7bf115eb80 1 -- 172.21.15.26:0/874935859 wait complete. 2023-10-06T21:15:57.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.023+0000 7f7bf115eb80 20 client.26258 trim_cache size 0 max 16384 2023-10-06T21:15:57.026 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "840be7b8-0467-4f3d-8ff0-c22ef96e3429": { 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "840be7b8-0467-4f3d-8ff0-c22ef96e3429", 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:15:57.027 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 msgr2=0x7f7bc40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 0x7f7bc40a3b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009fd0 tx=0x7f7bd0009aa0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 msgr2=0x7f7bd810fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b070 tx=0x7f7bc80077f0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 shutdown_connections 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7bc40a1670 0x7f7bc40a3b30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd81100d0 0x7f7bd810cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bd81032e0 0x7f7bd810fa80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 --2- 172.21.15.26:0/746228869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bd8108400 0x7f7bd810ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:15:57.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.024+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 >> 172.21.15.26:0/746228869 conn(0x7f7bd80feb60 msgr2=0x7f7bd8106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:15:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.028+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 shutdown_connections 2023-10-06T21:15:57.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:15:57.028+0000 7f7bdd7ff700 1 -- 172.21.15.26:0/746228869 wait complete. 2023-10-06T21:15:57.065 INFO:tasks.cephfs.filesystem:scrub status for tag:840be7b8-0467-4f3d-8ff0-c22ef96e3429 - {'path': '/', 'tag': '840be7b8-0467-4f3d-8ff0-c22ef96e3429', 'options': 'recursive,force'} 2023-10-06T21:15:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 315.0 s, 2171.3 tps, lat 46.443 ms stddev 50.723 2023-10-06T21:15:58.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:58 smithi167 ceph-mon[134494]: pgmap v173: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2023-10-06T21:15:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:58 smithi026 ceph-mon[98490]: pgmap v173: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2023-10-06T21:15:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:58 smithi078 ceph-mon[134896]: pgmap v173: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2023-10-06T21:15:59.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:15:59 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:59.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:15:59 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:15:59.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:15:59 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:00.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:00 smithi167 ceph-mon[134494]: pgmap v174: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2023-10-06T21:16:00.830 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:00 smithi026 ceph-mon[98490]: pgmap v174: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2023-10-06T21:16:00.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:00 smithi078 ceph-mon[134896]: pgmap v174: 97 pgs: 97 active+clean; 6.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2023-10-06T21:16:01.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:01 smithi167 ceph-mon[134494]: pgmap v175: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 38 MiB/s wr, 2.09k op/s 2023-10-06T21:16:01.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:01 smithi026 ceph-mon[98490]: pgmap v175: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 38 MiB/s wr, 2.09k op/s 2023-10-06T21:16:01.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:01 smithi078 ceph-mon[134896]: pgmap v175: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 38 MiB/s wr, 2.09k op/s 2023-10-06T21:16:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 320.0 s, 1645.2 tps, lat 60.100 ms stddev 87.695 2023-10-06T21:16:03.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:03 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:03.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:03 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:03 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:03 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:03.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:03 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:03.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:03 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:04 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:04 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:04 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:04 smithi078 ceph-mon[134896]: pgmap v176: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 569 op/s 2023-10-06T21:16:04.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:04 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:04 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:04 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:04 smithi167 ceph-mon[134494]: pgmap v176: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 569 op/s 2023-10-06T21:16:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:04 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:04 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:04 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:04 smithi026 ceph-mon[98490]: pgmap v176: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 569 op/s 2023-10-06T21:16:06.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:06 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:06.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:06 smithi078 ceph-mon[134896]: pgmap v177: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2023-10-06T21:16:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:06 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:06 smithi167 ceph-mon[134494]: pgmap v177: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2023-10-06T21:16:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:06 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:06 smithi026 ceph-mon[98490]: pgmap v177: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.29k op/s 2023-10-06T21:16:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 325.0 s, 2174.8 tps, lat 46.065 ms stddev 51.247 2023-10-06T21:16:08.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:08 smithi167 ceph-mon[134494]: pgmap v178: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-10-06T21:16:08.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:08 smithi026 ceph-mon[98490]: pgmap v178: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-10-06T21:16:08.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:08 smithi078 ceph-mon[134896]: pgmap v178: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-10-06T21:16:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.074+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/2186948621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a08073ba0 msgr2=0x7f9a0810d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.074+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/2186948621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a08073ba0 0x7f9a0810d1b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0098f0 tx=0x7f99fc009c00 comp rx=0 tx=0).stop 2023-10-06T21:16:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/2186948621 shutdown_connections 2023-10-06T21:16:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/2186948621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a0810d6f0 0x7f9a0810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/2186948621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a08073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/2186948621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a08073ba0 0x7f9a0810d1b0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0098f0 tx=0x7f99fc009c00 comp rx=0 tx=0).stop 2023-10-06T21:16:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/2186948621 >> 172.21.15.26:0/2186948621 conn(0x7f9a080fc940 msgr2=0x7f9a080fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/2186948621 shutdown_connections 2023-10-06T21:16:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.075+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/2186948621 wait complete. 2023-10-06T21:16:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 Processor -- start 2023-10-06T21:16:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 -- start start 2023-10-06T21:16:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a0810d6f0 0x7f9a0810bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 0x7f9a081a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081a2fb0 con 0x7f9a0810d6f0 2023-10-06T21:16:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081a3120 con 0x7f9a08073280 2023-10-06T21:16:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.077+0000 7f9a0e7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081a32e0 con 0x7f9a081a2530 2023-10-06T21:16:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.078+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.078+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36886/0 (socket says 172.21.15.26:36886) 2023-10-06T21:16:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.078+0000 7f9a07fff700 1 -- 172.21.15.26:0/587287335 learned_addr learned my addr 172.21.15.26:0/587287335 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.078+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 0x7f9a081a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.079+0000 7f9a07fff700 1 -- 172.21.15.26:0/587287335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 msgr2=0x7f9a081a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.079+0000 7f9a07fff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 0x7f9a081a29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.079+0000 7f9a07fff700 1 -- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a0810d6f0 msgr2=0x7f9a0810bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.079+0000 7f9a07fff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a0810d6f0 0x7f9a0810bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.079+0000 7f9a07fff700 1 -- 172.21.15.26:0/587287335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc009520 con 0x7f9a08073280 2023-10-06T21:16:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.080+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 0x7f9a081a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.080+0000 7f9a07fff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99f800e9c0 tx=0x7f99f800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.080+0000 7f9a057fa700 1 -- 172.21.15.26:0/587287335 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800cbd0 con 0x7f9a08073280 2023-10-06T21:16:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.080+0000 7f9a057fa700 1 -- 172.21.15.26:0/587287335 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f800eeb0 con 0x7f9a08073280 2023-10-06T21:16:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.080+0000 7f9a057fa700 1 -- 172.21.15.26:0/587287335 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8005580 con 0x7f9a08073280 2023-10-06T21:16:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.081+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a081aa760 con 0x7f9a08073280 2023-10-06T21:16:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.081+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a081aabf0 con 0x7f9a08073280 2023-10-06T21:16:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.088+0000 7f9a221fdb80 1 Processor -- start 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.088+0000 7f9a221fdb80 1 -- start start 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d615400 con 0x55ed2d6abc00 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d606bc0 con 0x55ed2d6b73d0 2023-10-06T21:16:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.089+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6044e0 con 0x55ed2d6ac520 2023-10-06T21:16:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.090+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.090+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38280/0 (socket says 172.21.15.26:38280) 2023-10-06T21:16:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.090+0000 7f9a0cd9c700 1 -- 172.21.15.26:0/2769237197 learned_addr learned my addr 172.21.15.26:0/2769237197 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.090+0000 7f9a077fe700 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 -- 172.21.15.26:0/2769237197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 msgr2=0x55ed2d6b6e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 -- 172.21.15.26:0/2769237197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 msgr2=0x55ed2d6b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 -- 172.21.15.26:0/2769237197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2d42ad80 con 0x55ed2d6abc00 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a057fa700 1 -- 172.21.15.26:0/587287335 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f99f8004750 con 0x7f9a08073280 2023-10-06T21:16:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.091+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9a00007680 tx=0x7f9a00007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee92a0957e92b291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.092+0000 7f99ee7fc700 1 -- 172.21.15.26:0/2769237197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00015850 con 0x55ed2d6abc00 2023-10-06T21:16:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.093+0000 7f99ee7fc700 1 -- 172.21.15.26:0/2769237197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0000f040 con 0x55ed2d6abc00 2023-10-06T21:16:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.093+0000 7f99ee7fc700 1 -- 172.21.15.26:0/2769237197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00014d60 con 0x55ed2d6abc00 2023-10-06T21:16:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.093+0000 7f9a057fa700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 0x7f99f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.093+0000 7f9a057fa700 1 -- 172.21.15.26:0/587287335 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99f8014070 con 0x7f9a08073280 2023-10-06T21:16:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.093+0000 7f9a07fff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 0x7f99f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2769237197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 msgr2=0x55ed2d6abfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9a00007680 tx=0x7f9a00007990 comp rx=0 tx=0).stop 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2769237197 shutdown_connections 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6abfe0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2769237197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2769237197 >> 172.21.15.26:0/2769237197 conn(0x55ed2d621200 msgr2=0x55ed2d6a8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.094+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2769237197 shutdown_connections 2023-10-06T21:16:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.095+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2769237197 wait complete. 2023-10-06T21:16:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.095+0000 7f9a07fff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 0x7f99f00a3ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99f80a2330 tx=0x7f99f8085050 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.095+0000 7f9a221fdb80 1 -- start start 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 Processor -- start 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6abc00 0x55ed2d6b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d5ffdd0 con 0x55ed2d6ac520 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d59c6b0 con 0x55ed2d6abc00 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.096+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d59c350 con 0x55ed2d6b73d0 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.097+0000 7f9a221fdb80 10 client.0 _ll_register_callbacks cb 0x55ed2d3f97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.097+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38292/0 (socket says 172.21.15.26:38292) 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 -- 172.21.15.26:0/2121940632 learned_addr learned my addr 172.21.15.26:0/2121940632 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6abc00 0x55ed2d6b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 -- 172.21.15.26:0/2121940632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 msgr2=0x55ed2d6b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 -- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6abc00 msgr2=0x55ed2d6b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6abc00 0x55ed2d6b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.098+0000 7f9a077fe700 1 -- 172.21.15.26:0/2121940632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a00009dd0 con 0x55ed2d6ac520 2023-10-06T21:16:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.099+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.099+0000 7f9a077fe700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005c10 tx=0x7f99fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d4ba7374e96625 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.099+0000 7f99e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.100+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01e070 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.100+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99fc005040 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.100+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed2d69fd50 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.101+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99fc00f980 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.101+0000 7f9a221fdb80 10 client.16044 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.101+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ed2d413c00 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.101+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed2d69fd50 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f99fc015020 con 0x55ed2d6ac520 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 10 client.16044 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 20 client.16044 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6d7fe0 0x55ed2d6da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 4 client.16044 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.102+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed2d5f8db0 con 0x55ed2d6d7fe0 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.103+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6d7fe0 0x55ed2d6da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.104+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6d7fe0 0x55ed2d6da400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a143979eb422a1ec server_cookie=ca2a98a9bc723d5b in_seq=0 out_seq=0 2023-10-06T21:16:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.104+0000 7f99e77fe700 10 client.16044 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.105+0000 7f99e77fe700 1 -- 172.21.15.26:0/2121940632 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed2d5f8db0 con 0x55ed2d6d7fe0 2023-10-06T21:16:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.105+0000 7f99e77fe700 10 client.16044 handle_command_reply: tid=0 2023-10-06T21:16:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.105+0000 7f9a221fdb80 1 client.16044 shutdown 2023-10-06T21:16:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.105+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 msgr2=0x55ed2d6b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.105+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005c10 tx=0x7f99fc00b840 comp rx=0 tx=0).stop 2023-10-06T21:16:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 shutdown_connections 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6d7fe0 0x55ed2d6da400 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6abc00 0x55ed2d6b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/2121940632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6ac520 0x55ed2d6b4660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 >> 172.21.15.26:0/2121940632 conn(0x55ed2d621200 msgr2=0x55ed2d6a7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.106+0000 7f99e77fe700 0 client.16044 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.107+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 shutdown_connections 2023-10-06T21:16:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.107+0000 7f9a221fdb80 1 -- 172.21.15.26:0/2121940632 wait complete. 2023-10-06T21:16:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.108+0000 7f9a221fdb80 20 client.16044 trim_cache size 0 max 16384 2023-10-06T21:16:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.136+0000 7f9a221fdb80 1 Processor -- start 2023-10-06T21:16:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.136+0000 7f9a221fdb80 1 -- start start 2023-10-06T21:16:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d4808c0 con 0x55ed2d6abc00 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6d55f0 con 0x55ed2d6b73d0 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.137+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6ace20 con 0x55ed2d6ac520 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.142+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.142+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.142+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38302/0 (socket says 172.21.15.26:38302) 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.142+0000 7f9a0cd9c700 1 -- 172.21.15.26:0/3456232962 learned_addr learned my addr 172.21.15.26:0/3456232962 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.142+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.144+0000 7f9a077fe700 1 -- 172.21.15.26:0/3456232962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 msgr2=0x55ed2d6af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.144+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.145+0000 7f9a077fe700 1 -- 172.21.15.26:0/3456232962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 msgr2=0x55ed2d6aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.145+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.145+0000 7f9a077fe700 1 -- 172.21.15.26:0/3456232962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed2d4511f0 con 0x55ed2d6ac520 2023-10-06T21:16:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.150+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bdf80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f99fc003a50 tx=0x7f99fc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9f88ed7f396c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.154+0000 7f99eeffd700 1 -- 172.21.15.26:0/3456232962 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01e040 con 0x55ed2d6ac520 2023-10-06T21:16:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.154+0000 7f99eeffd700 1 -- 172.21.15.26:0/3456232962 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99fc051070 con 0x55ed2d6ac520 2023-10-06T21:16:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.154+0000 7f99eeffd700 1 -- 172.21.15.26:0/3456232962 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0058e0 con 0x55ed2d6ac520 2023-10-06T21:16:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.155+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3456232962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 msgr2=0x55ed2d6bdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.155+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bdf80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f99fc003a50 tx=0x7f99fc000bc0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3456232962 shutdown_connections 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3456232962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bdf80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3456232962 >> 172.21.15.26:0/3456232962 conn(0x55ed2d621200 msgr2=0x55ed2d6d9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3456232962 shutdown_connections 2023-10-06T21:16:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.156+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3456232962 wait complete. 2023-10-06T21:16:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.157+0000 7f9a221fdb80 1 -- start start 2023-10-06T21:16:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.157+0000 7f9a221fdb80 1 Processor -- start 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6bb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6bc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6cc120 con 0x55ed2d6abc00 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6cc290 con 0x55ed2d6b73d0 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2d6cc450 con 0x55ed2d6ac520 2023-10-06T21:16:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.158+0000 7f9a221fdb80 10 client.0 _ll_register_callbacks cb 0x7f99fc00f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6bc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a077fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33558/0 (socket says 172.21.15.26:33558) 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a0cd9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6bc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36922/0 (socket says 172.21.15.26:36922) 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a077fe700 1 -- 172.21.15.26:0/3888411596 learned_addr learned my addr 172.21.15.26:0/3888411596 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.159+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6bb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.160+0000 7f9a077fe700 1 -- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 msgr2=0x55ed2d6bc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.160+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6bc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.160+0000 7f9a077fe700 1 -- 172.21.15.26:0/3888411596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 msgr2=0x55ed2d6bb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.160+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6bb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.160+0000 7f9a077fe700 1 -- 172.21.15.26:0/3888411596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed2d413c00 con 0x55ed2d6ac520 2023-10-06T21:16:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.162+0000 7f9a077fe700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b880 tx=0x7f99fc00b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be51e2cfeabed089 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.162+0000 7f99e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.162+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01e070 con 0x55ed2d6ac520 2023-10-06T21:16:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.162+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f99fc000e40 con 0x55ed2d6ac520 2023-10-06T21:16:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.162+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a0000f040 con 0x55ed2d6ac520 2023-10-06T21:16:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.163+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99fc003880 con 0x55ed2d6ac520 2023-10-06T21:16:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.163+0000 7f9a221fdb80 10 client.26245 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.163+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ed2d429250 con 0x55ed2d6ac520 2023-10-06T21:16:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.163+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0000f040 con 0x55ed2d6ac520 2023-10-06T21:16:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f99fc015070 con 0x55ed2d6ac520 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 10 client.26245 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 20 client.26245 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6cd7a0 0x55ed2d6cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 4 client.26245 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed2d5f8ad0 con 0x55ed2d6cd7a0 2023-10-06T21:16:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.164+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6cd7a0 0x55ed2d6cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.165+0000 7f9a0cd9c700 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6cd7a0 0x55ed2d6cfbc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788523bcf7c7cf80 server_cookie=aa30364f5dbd1d3e in_seq=0 out_seq=0 2023-10-06T21:16:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.165+0000 7f99e77fe700 10 client.26245 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.185+0000 7f99e77fe700 1 -- 172.21.15.26:0/3888411596 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed2d5f8ad0 con 0x55ed2d6cd7a0 2023-10-06T21:16:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.185+0000 7f99e77fe700 10 client.26245 handle_command_reply: tid=0 2023-10-06T21:16:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.185+0000 7f9a221fdb80 1 client.26245 shutdown 2023-10-06T21:16:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.185+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 msgr2=0x55ed2d6bbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.185+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b880 tx=0x7f99fc00b350 comp rx=0 tx=0).stop 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 shutdown_connections 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed2d6cd7a0 0x55ed2d6cfbc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed2d6b73d0 0x55ed2d6bc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed2d6abc00 0x55ed2d6bb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 --2- 172.21.15.26:0/3888411596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed2d6ac520 0x55ed2d6bbc60 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00b880 tx=0x7f99fc00b350 comp rx=0 tx=0).stop 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 >> 172.21.15.26:0/3888411596 conn(0x55ed2d621200 msgr2=0x55ed2d6af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f99e77fe700 0 client.26245 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 shutdown_connections 2023-10-06T21:16:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.186+0000 7f9a221fdb80 1 -- 172.21.15.26:0/3888411596 wait complete. 2023-10-06T21:16:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a221fdb80 20 client.26245 trim_cache size 0 max 16384 2023-10-06T21:16:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:16:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:16:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:16:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 msgr2=0x7f99f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 0x7f99f00a3ab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99f80a2330 tx=0x7f99f8085050 comp rx=0 tx=0).stop 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 msgr2=0x7f9a0810bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99f800e9c0 tx=0x7f99f800ecd0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 shutdown_connections 2023-10-06T21:16:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a081a2530 0x7f9a081a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a08073280 0x7f9a0810bab0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99f00a15f0 0x7f99f00a3ab0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 --2- 172.21.15.26:0/587287335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a0810d6f0 0x7f9a0810bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 >> 172.21.15.26:0/587287335 conn(0x7f9a080fc940 msgr2=0x7f9a080fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 shutdown_connections 2023-10-06T21:16:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:10.187+0000 7f9a0e7ff700 1 -- 172.21.15.26:0/587287335 wait complete. 2023-10-06T21:16:10.538 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:10 smithi167 ceph-mon[134494]: pgmap v179: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-10-06T21:16:10.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:10 smithi026 ceph-mon[98490]: pgmap v179: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-10-06T21:16:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:10 smithi078 ceph-mon[134896]: pgmap v179: 97 pgs: 97 active+clean; 6.6 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-10-06T21:16:11.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:16:11.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:16:11.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:16:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:16:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:16:11.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:16:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:16:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:16:11.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:16:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 330.0 s, 2105.4 tps, lat 47.921 ms stddev 52.494 2023-10-06T21:16:12.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:12 smithi167 ceph-mon[134494]: pgmap v180: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:16:12.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:12 smithi026 ceph-mon[98490]: pgmap v180: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:16:12.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:12 smithi078 ceph-mon[134896]: pgmap v180: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:16:14.326 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:14 smithi026 ceph-mon[98490]: pgmap v181: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2023-10-06T21:16:14.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:14 smithi078 ceph-mon[134896]: pgmap v181: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2023-10-06T21:16:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:14 smithi167 ceph-mon[134494]: pgmap v181: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2023-10-06T21:16:16.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:16 smithi167 ceph-mon[134494]: pgmap v182: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-10-06T21:16:16.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:16 smithi026 ceph-mon[98490]: pgmap v182: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-10-06T21:16:16.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:16 smithi078 ceph-mon[134896]: pgmap v182: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-10-06T21:16:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 335.0 s, 2158.1 tps, lat 45.768 ms stddev 51.905 2023-10-06T21:16:18.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:18 smithi167 ceph-mon[134494]: pgmap v183: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-10-06T21:16:18.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:18 smithi026 ceph-mon[98490]: pgmap v183: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-10-06T21:16:18.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:18 smithi078 ceph-mon[134896]: pgmap v183: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-10-06T21:16:20.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:20 smithi167 ceph-mon[134494]: pgmap v184: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 637 op/s 2023-10-06T21:16:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:20 smithi026 ceph-mon[98490]: pgmap v184: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 637 op/s 2023-10-06T21:16:20.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:20 smithi078 ceph-mon[134896]: pgmap v184: 97 pgs: 97 active+clean; 6.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.6 MiB/s wr, 637 op/s 2023-10-06T21:16:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 340.0 s, 1981.8 tps, lat 50.791 ms stddev 58.397 2023-10-06T21:16:22.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:22 smithi167 ceph-mon[134494]: pgmap v185: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 913 op/s 2023-10-06T21:16:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:22 smithi026 ceph-mon[98490]: pgmap v185: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 913 op/s 2023-10-06T21:16:22.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:22 smithi078 ceph-mon[134896]: pgmap v185: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 913 op/s 2023-10-06T21:16:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.463+0000 7f25a9fff700 1 -- 172.21.15.26:0/859207442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 msgr2=0x7f25a4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.463+0000 7f25a9fff700 1 --2- 172.21.15.26:0/859207442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a4107eb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b0a0 tx=0x7f259c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 -- 172.21.15.26:0/859207442 shutdown_connections 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 --2- 172.21.15.26:0/859207442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 --2- 172.21.15.26:0/859207442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a4107eb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 --2- 172.21.15.26:0/859207442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 -- 172.21.15.26:0/859207442 >> 172.21.15.26:0/859207442 conn(0x7f25a406cfa0 msgr2=0x7f25a406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 -- 172.21.15.26:0/859207442 shutdown_connections 2023-10-06T21:16:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.464+0000 7f25a9fff700 1 -- 172.21.15.26:0/859207442 wait complete. 2023-10-06T21:16:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 Processor -- start 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 -- start start 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a41a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41ab020 con 0x7f25a410f620 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41ab190 con 0x7f25a410fa90 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.466+0000 7f25a9fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41ab350 con 0x7f25a411a690 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.467+0000 7f25a3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.467+0000 7f25a3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47002/0 (socket says 172.21.15.26:47002) 2023-10-06T21:16:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.467+0000 7f25a3fff700 1 -- 172.21.15.26:0/2818234135 learned_addr learned my addr 172.21.15.26:0/2818234135 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.467+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.467+0000 7f25a37fe700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a41a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a3fff700 1 -- 172.21.15.26:0/2818234135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 msgr2=0x7f25a41a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a3fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a3fff700 1 -- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 msgr2=0x7f25a41a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a3fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a41a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a3fff700 1 -- 172.21.15.26:0/2818234135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c009d00 con 0x7f25a411a690 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.468+0000 7f25a37fe700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a41a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a3fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f259400b480 tx=0x7f259400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a0ff9700 1 -- 172.21.15.26:0/2818234135 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594011830 con 0x7f25a411a690 2023-10-06T21:16:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a0ff9700 1 -- 172.21.15.26:0/2818234135 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2594011e50 con 0x7f25a411a690 2023-10-06T21:16:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a0ff9700 1 -- 172.21.15.26:0/2818234135 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400f2e0 con 0x7f25a411a690 2023-10-06T21:16:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a41ab630 con 0x7f25a411a690 2023-10-06T21:16:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.469+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41b2e70 con 0x7f25a411a690 2023-10-06T21:16:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.476+0000 7f25bd98cb80 1 Processor -- start 2023-10-06T21:16:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.478+0000 7f25a0ff9700 1 -- 172.21.15.26:0/2818234135 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f25940105a0 con 0x7f25a411a690 2023-10-06T21:16:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.478+0000 7f25bd98cb80 1 -- start start 2023-10-06T21:16:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0406fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e04147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0370400 con 0x5583e04123d0 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0361bc0 con 0x5583e0407520 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.480+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e035f4e0 con 0x5583e0406c00 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36612/0 (socket says 172.21.15.26:36612) 2023-10-06T21:16:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a37fe700 1 -- 172.21.15.26:0/76561281 learned_addr learned my addr 172.21.15.26:0/76561281 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0406fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a0ff9700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 0x7f258c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.481+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e04147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 -- 172.21.15.26:0/76561281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 msgr2=0x5583e0411e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a0ff9700 1 -- 172.21.15.26:0/2818234135 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25940cc970 con 0x7f25a411a690 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 -- 172.21.15.26:0/76561281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 msgr2=0x5583e04147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e04147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 -- 172.21.15.26:0/76561281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e0185d80 con 0x5583e0406c00 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a37fe700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a3fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 0x7f258c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0406fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f25a41a7650 tx=0x7f259800ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d22657105feadb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f2589ffb700 1 -- 172.21.15.26:0/76561281 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598004020 con 0x5583e0406c00 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f2589ffb700 1 -- 172.21.15.26:0/76561281 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2598003680 con 0x5583e0406c00 2023-10-06T21:16:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.482+0000 7f2589ffb700 1 -- 172.21.15.26:0/76561281 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800f680 con 0x5583e0406c00 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25a3fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 0x7f258c0a3b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f259400b450 tx=0x7f25940109a0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 -- 172.21.15.26:0/76561281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 msgr2=0x5583e0406fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0406fe0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f25a41a7650 tx=0x7f259800ab70 comp rx=0 tx=0).stop 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 -- 172.21.15.26:0/76561281 shutdown_connections 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e04147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0406fe0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/76561281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0411e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 -- 172.21.15.26:0/76561281 >> 172.21.15.26:0/76561281 conn(0x5583e037c200 msgr2=0x5583e0403080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 -- 172.21.15.26:0/76561281 shutdown_connections 2023-10-06T21:16:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.483+0000 7f25bd98cb80 1 -- 172.21.15.26:0/76561281 wait complete. 2023-10-06T21:16:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.484+0000 7f25bd98cb80 1 -- start start 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.484+0000 7f25bd98cb80 1 Processor -- start 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e040f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e035add0 con 0x5583e0406c00 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e02f76b0 con 0x5583e0407520 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e02f7350 con 0x5583e04123d0 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.485+0000 7f25bd98cb80 10 client.0 _ll_register_callbacks cb 0x5583e01547a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e040f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36628/0 (socket says 172.21.15.26:36628) 2023-10-06T21:16:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47030/0 (socket says 172.21.15.26:47030) 2023-10-06T21:16:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.486+0000 7f25a37fe700 1 -- 172.21.15.26:0/3430007226 learned_addr learned my addr 172.21.15.26:0/3430007226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 -- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 msgr2=0x5583e040f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 -- 172.21.15.26:0/3430007226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 msgr2=0x5583e040f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e040f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 -- 172.21.15.26:0/3430007226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a411d3b0 con 0x5583e04123d0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2598005950 tx=0x7f259800afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=962f3d659614923 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.488+0000 7f2582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.489+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598004350 con 0x5583e04123d0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.489+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25980044b0 con 0x5583e04123d0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.489+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583e03fad50 con 0x5583e04123d0 2023-10-06T21:16:23.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.489+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259802b450 con 0x5583e04123d0 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.490+0000 7f25bd98cb80 10 client.26254 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.490+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f25980114c0 con 0x5583e04123d0 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583e03fad50 con 0x5583e04123d0 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2598035420 con 0x5583e04123d0 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 10 client.26254 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:23.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 20 client.26254 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0421ec0 0x5583e04242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 4 client.26254 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.491+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583e0353db0 con 0x5583e0421ec0 2023-10-06T21:16:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.492+0000 7f25a37fe700 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0421ec0 0x5583e04242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.492+0000 7f2582ffd700 10 client.26254 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.492+0000 7f25a37fe700 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0421ec0 0x5583e04242e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93fb8dfc8092e06 server_cookie=8da706416a7db047 in_seq=0 out_seq=0 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.493+0000 7f2582ffd700 1 -- 172.21.15.26:0/3430007226 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583e0353db0 con 0x5583e0421ec0 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.493+0000 7f2582ffd700 10 client.26254 handle_command_reply: tid=0 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.493+0000 7f25bd98cb80 1 client.26254 shutdown 2023-10-06T21:16:23.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.493+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 msgr2=0x5583e040fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.493+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2598005950 tx=0x7f259800afd0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 shutdown_connections 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e040fbd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e040f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0421ec0 0x5583e04242e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3430007226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f2582ffd700 0 client.26254 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 >> 172.21.15.26:0/3430007226 conn(0x5583e037c200 msgr2=0x5583e0402c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 shutdown_connections 2023-10-06T21:16:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3430007226 wait complete. 2023-10-06T21:16:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.494+0000 7f25bd98cb80 20 client.26254 trim_cache size 0 max 16384 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 Processor -- start 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 -- start start 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e0418f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e04091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e01db8c0 con 0x5583e0406c00 2023-10-06T21:16:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0419480 con 0x5583e0407520 2023-10-06T21:16:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.519+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0424940 con 0x5583e04123d0 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36640/0 (socket says 172.21.15.26:36640) 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 -- 172.21.15.26:0/3911348576 learned_addr learned my addr 172.21.15.26:0/3911348576 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 -- 172.21.15.26:0/3911348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 msgr2=0x5583e04091a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e04091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e04091a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 -- 172.21.15.26:0/3911348576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 msgr2=0x5583e0418f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e0418f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a37fe700 1 -- 172.21.15.26:0/3911348576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583e016ec00 con 0x5583e0407520 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.520+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e04091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.521+0000 7f25a37fe700 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f259c015000 tx=0x7f259c007e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ca6beec5be6f760 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.521+0000 7f258a7fc700 1 -- 172.21.15.26:0/3911348576 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00e040 con 0x5583e0407520 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.521+0000 7f258a7fc700 1 -- 172.21.15.26:0/3911348576 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f259c026560 con 0x5583e0407520 2023-10-06T21:16:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.521+0000 7f258a7fc700 1 -- 172.21.15.26:0/3911348576 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0052a0 con 0x5583e0407520 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.522+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3911348576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 msgr2=0x5583e040fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.522+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f259c015000 tx=0x7f259c007e50 comp rx=0 tx=0).stop 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.523+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3911348576 shutdown_connections 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.523+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e04123d0 0x5583e04091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.523+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e0406c00 0x5583e0418f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.523+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/3911348576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e040fa00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.523+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3911348576 >> 172.21.15.26:0/3911348576 conn(0x5583e037c200 msgr2=0x5583e0408a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.524+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3911348576 shutdown_connections 2023-10-06T21:16:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.524+0000 7f25bd98cb80 1 -- 172.21.15.26:0/3911348576 wait complete. 2023-10-06T21:16:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.524+0000 7f25bd98cb80 1 -- start start 2023-10-06T21:16:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.525+0000 7f25bd98cb80 1 Processor -- start 2023-10-06T21:16:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0418a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e0415460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e04159a0 con 0x5583e04123d0 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0415b10 con 0x5583e0407520 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583e0415cd0 con 0x5583e0406c00 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25bd98cb80 10 client.0 _ll_register_callbacks cb 0x7f259c00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36654/0 (socket says 172.21.15.26:36654) 2023-10-06T21:16:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25a37fe700 1 -- 172.21.15.26:0/1923886046 learned_addr learned my addr 172.21.15.26:0/1923886046 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.526+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0418a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e0415460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 -- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 msgr2=0x5583e0414f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 -- 172.21.15.26:0/1923886046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 msgr2=0x5583e0415460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e0415460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 -- 172.21.15.26:0/1923886046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c0090d0 con 0x5583e0406c00 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a37fe700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.527+0000 7f25a2ffd700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0418a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f259800ac10 tx=0x7f2598005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a10ace4114c08e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f2582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980044e0 con 0x5583e0406c00 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2598000e40 con 0x5583e0406c00 2023-10-06T21:16:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2598003680 con 0x5583e0406c00 2023-10-06T21:16:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2598005070 con 0x5583e0406c00 2023-10-06T21:16:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f25bd98cb80 10 client.26257 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.528+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5583e01ac1f0 con 0x5583e0406c00 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2598003680 con 0x5583e0406c00 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2598004a40 con 0x5583e0406c00 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 10 client.26257 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 20 client.26257 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0476c70 0x5583e0479090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 4 client.26257 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:16:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.529+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583e0353ad0 con 0x5583e0476c70 2023-10-06T21:16:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.530+0000 7f25a37fe700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0476c70 0x5583e0479090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.532+0000 7f25a37fe700 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0476c70 0x5583e0479090 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b421caeb5d09e7 server_cookie=51263542824dc68f in_seq=0 out_seq=0 2023-10-06T21:16:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.532+0000 7f2582ffd700 10 client.26257 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.549+0000 7f2582ffd700 1 -- 172.21.15.26:0/1923886046 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583e0353ad0 con 0x5583e0476c70 2023-10-06T21:16:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.549+0000 7f2582ffd700 10 client.26257 handle_command_reply: tid=0 2023-10-06T21:16:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.549+0000 7f25bd98cb80 1 client.26257 shutdown 2023-10-06T21:16:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.552+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 msgr2=0x5583e0418a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.552+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0418a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f259800ac10 tx=0x7f2598005ea0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 shutdown_connections 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5583e04123d0 0x5583e0415460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5583e0406c00 0x5583e0418a90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5583e0476c70 0x5583e0479090 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 --2- 172.21.15.26:0/1923886046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583e0407520 0x5583e0414f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f2582ffd700 0 client.26257 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 >> 172.21.15.26:0/1923886046 conn(0x5583e037c200 msgr2=0x5583e040df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 shutdown_connections 2023-10-06T21:16:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 1 -- 172.21.15.26:0/1923886046 wait complete. 2023-10-06T21:16:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.553+0000 7f25bd98cb80 20 client.26257 trim_cache size 0 max 16384 2023-10-06T21:16:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:16:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:16:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:16:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 msgr2=0x7f258c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 0x7f258c0a3b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f259400b450 tx=0x7f25940109a0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 msgr2=0x7f25a41aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f259400b480 tx=0x7f259400b790 comp rx=0 tx=0).stop 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 shutdown_connections 2023-10-06T21:16:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a411a690 0x7f25a41aa9e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f258c0a16b0 0x7f258c0a3b70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a410f620 0x7f25a41a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 --2- 172.21.15.26:0/2818234135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25a410fa90 0x7f25a41a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 >> 172.21.15.26:0/2818234135 conn(0x7f25a406cfa0 msgr2=0x7f25a4119110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 shutdown_connections 2023-10-06T21:16:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:23.555+0000 7f25a9fff700 1 -- 172.21.15.26:0/2818234135 wait complete. 2023-10-06T21:16:24.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:24 smithi026 ceph-mon[98490]: pgmap v186: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 631 op/s 2023-10-06T21:16:24.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:24 smithi078 ceph-mon[134896]: pgmap v186: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 631 op/s 2023-10-06T21:16:24.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:24 smithi167 ceph-mon[134494]: pgmap v186: 97 pgs: 97 active+clean; 6.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.0 MiB/s wr, 631 op/s 2023-10-06T21:16:26.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:26 smithi167 ceph-mon[134494]: pgmap v187: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 816 op/s 2023-10-06T21:16:26.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:26 smithi026 ceph-mon[98490]: pgmap v187: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 816 op/s 2023-10-06T21:16:26.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:26 smithi078 ceph-mon[134896]: pgmap v187: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 816 op/s 2023-10-06T21:16:27.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:16:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 345.0 s, 1650.2 tps, lat 60.610 ms stddev 64.371 2023-10-06T21:16:27.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.458+0000 7f9e70fff700 1 -- 172.21.15.26:0/940085233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 msgr2=0x7f9e6c1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.458+0000 7f9e70fff700 1 --2- 172.21.15.26:0/940085233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1001c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009f90 tx=0x7f9e6400b330 comp rx=0 tx=0).stop 2023-10-06T21:16:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.459+0000 7f9e70fff700 1 -- 172.21.15.26:0/940085233 shutdown_connections 2023-10-06T21:16:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.459+0000 7f9e70fff700 1 --2- 172.21.15.26:0/940085233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.459+0000 7f9e70fff700 1 --2- 172.21.15.26:0/940085233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1001c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.459+0000 7f9e70fff700 1 --2- 172.21.15.26:0/940085233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.459+0000 7f9e70fff700 1 -- 172.21.15.26:0/940085233 >> 172.21.15.26:0/940085233 conn(0x7f9e6c0fb830 msgr2=0x7f9e6c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.460+0000 7f9e70fff700 1 -- 172.21.15.26:0/940085233 shutdown_connections 2023-10-06T21:16:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.460+0000 7f9e70fff700 1 -- 172.21.15.26:0/940085233 wait complete. 2023-10-06T21:16:27.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.461+0000 7f9e70fff700 1 Processor -- start 2023-10-06T21:16:27.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.462+0000 7f9e70fff700 1 -- start start 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.463+0000 7f9e70fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.463+0000 7f9e70fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e70fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e70fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19dc20 con 0x7f9e6c0ffde0 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e70fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19dd90 con 0x7f9e6c1051e0 2023-10-06T21:16:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e70fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c19df50 con 0x7f9e6c100700 2023-10-06T21:16:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e6a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47062/0 (socket says 172.21.15.26:47062) 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36670/0 (socket says 172.21.15.26:36670) 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 -- 172.21.15.26:0/225361062 learned_addr learned my addr 172.21.15.26:0/225361062 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 -- 172.21.15.26:0/225361062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 msgr2=0x7f9e6c19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 -- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 msgr2=0x7f9e6c1a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.464+0000 7f9e69d9b700 1 -- 172.21.15.26:0/225361062 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64009c40 con 0x7f9e6c100700 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d980 tx=0x7f9e5c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e5b7fe700 1 -- 172.21.15.26:0/225361062 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009930 con 0x7f9e6c100700 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e5b7fe700 1 -- 172.21.15.26:0/225361062 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5c010460 con 0x7f9e6c100700 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c19e230 con 0x7f9e6c100700 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c1aa530 con 0x7f9e6c100700 2023-10-06T21:16:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.465+0000 7f9e5b7fe700 1 -- 172.21.15.26:0/225361062 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00f620 con 0x7f9e6c100700 2023-10-06T21:16:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.471+0000 7f9e5b7fe700 1 -- 172.21.15.26:0/225361062 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9e5c009a90 con 0x7f9e6c100700 2023-10-06T21:16:27.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.473+0000 7f9e8488fb80 1 Processor -- start 2023-10-06T21:16:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 -- start start 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd17f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd0dd460 con 0x5609bd1749e0 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd0cec20 con 0x5609bd1740c0 2023-10-06T21:16:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.474+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd0cc540 con 0x5609bd17f890 2023-10-06T21:16:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.475+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.475+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47078/0 (socket says 172.21.15.26:47078) 2023-10-06T21:16:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.475+0000 7f9e69d9b700 1 -- 172.21.15.26:0/2733087252 learned_addr learned my addr 172.21.15.26:0/2733087252 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.476+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.476+0000 7f9e5b7fe700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 0x7f9e540a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.476+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd17f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.476+0000 7f9e5b7fe700 1 -- 172.21.15.26:0/225361062 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9e5c0cc8f0 con 0x7f9e6c100700 2023-10-06T21:16:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6a59c700 1 -- 172.21.15.26:0/2733087252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 msgr2=0x5609bd181c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6a59c700 1 -- 172.21.15.26:0/2733087252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 msgr2=0x5609bd1744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6a59c700 1 -- 172.21.15.26:0/2733087252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609bcef2de0 con 0x5609bd1749e0 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 0x7f9e540a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.477+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.478+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd17f350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f80 tx=0x7f9e640093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cae1fd944ac53a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.478+0000 7f9e58ff9700 1 -- 172.21.15.26:0/2733087252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400e070 con 0x5609bd1749e0 2023-10-06T21:16:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.479+0000 7f9e58ff9700 1 -- 172.21.15.26:0/2733087252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e64007b50 con 0x5609bd1749e0 2023-10-06T21:16:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.479+0000 7f9e58ff9700 1 -- 172.21.15.26:0/2733087252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400bc50 con 0x5609bd1749e0 2023-10-06T21:16:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.479+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 0x7f9e540a3c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097a0 tx=0x7f9e600047c0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 -- 172.21.15.26:0/2733087252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 msgr2=0x5609bd17f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd17f350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9e64000f80 tx=0x7f9e640093e0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 -- 172.21.15.26:0/2733087252 shutdown_connections 2023-10-06T21:16:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd181c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/2733087252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd17f350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.480+0000 7f9e8488fb80 1 -- 172.21.15.26:0/2733087252 >> 172.21.15.26:0/2733087252 conn(0x5609bd0e9260 msgr2=0x5609bd170450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.482+0000 7f9e8488fb80 1 -- 172.21.15.26:0/2733087252 shutdown_connections 2023-10-06T21:16:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.483+0000 7f9e8488fb80 1 -- 172.21.15.26:0/2733087252 wait complete. 2023-10-06T21:16:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.484+0000 7f9e8488fb80 1 -- start start 2023-10-06T21:16:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.484+0000 7f9e8488fb80 1 Processor -- start 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd17f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd1749e0 0x5609bd17c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd0c7e30 con 0x5609bd1740c0 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd064710 con 0x5609bd17f890 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd0643b0 con 0x5609bd1749e0 2023-10-06T21:16:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.485+0000 7f9e8488fb80 10 client.0 _ll_register_callbacks cb 0x5609bcec1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36700/0 (socket says 172.21.15.26:36700) 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 -- 172.21.15.26:0/120524215 learned_addr learned my addr 172.21.15.26:0/120524215 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd1749e0 0x5609bd17c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd17f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 -- 172.21.15.26:0/120524215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd1749e0 msgr2=0x5609bd17c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd1749e0 0x5609bd17c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 -- 172.21.15.26:0/120524215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 msgr2=0x5609bd17f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd17f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6ad9d700 1 -- 172.21.15.26:0/120524215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c1aa530 con 0x5609bd17f890 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.486+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd17f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000f790 tx=0x7f9e60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee34961bc43d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60003e90 con 0x5609bd17f890 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609bd167f10 con 0x5609bd17f890 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9e60004540 con 0x5609bd17f890 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e60010640 con 0x5609bd17f890 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e8488fb80 10 client.26303 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.487+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f9e6c114b90 con 0x5609bd17f890 2023-10-06T21:16:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609bd167f10 con 0x5609bd17f890 2023-10-06T21:16:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e600229f0 con 0x5609bd17f890 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 10 client.26303 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 20 client.26303 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd18f430 0x5609bd191850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 4 client.26303 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609bd0c0e10 con 0x5609bd18f430 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.488+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd18f430 0x5609bd191850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.489+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd18f430 0x5609bd191850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a1f991998851d8 server_cookie=f675c73202fdf965 in_seq=0 out_seq=0 2023-10-06T21:16:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.489+0000 7f9e51ffb700 10 client.26303 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.491+0000 7f9e51ffb700 1 -- 172.21.15.26:0/120524215 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609bd0c0e10 con 0x5609bd18f430 2023-10-06T21:16:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.491+0000 7f9e51ffb700 10 client.26303 handle_command_reply: tid=0 2023-10-06T21:16:27.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.491+0000 7f9e8488fb80 1 client.26303 shutdown 2023-10-06T21:16:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.494+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 msgr2=0x5609bd17cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.494+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000f790 tx=0x7f9e60004750 comp rx=0 tx=0).stop 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 shutdown_connections 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd18f430 0x5609bd191850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd17f890 0x5609bd17cbd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd17f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/120524215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd1749e0 0x5609bd17c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 >> 172.21.15.26:0/120524215 conn(0x5609bd0e9260 msgr2=0x5609bd170040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e51ffb700 0 client.26303 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 shutdown_connections 2023-10-06T21:16:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.495+0000 7f9e8488fb80 1 -- 172.21.15.26:0/120524215 wait complete. 2023-10-06T21:16:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.496+0000 7f9e8488fb80 20 client.26303 trim_cache size 0 max 16384 2023-10-06T21:16:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.531+0000 7f9e8488fb80 1 Processor -- start 2023-10-06T21:16:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.534+0000 7f9e8488fb80 1 -- start start 2023-10-06T21:16:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd191440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bcf48920 con 0x5609bd1749e0 2023-10-06T21:16:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd186920 con 0x5609bd1740c0 2023-10-06T21:16:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.535+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd17aa90 con 0x5609bd17f890 2023-10-06T21:16:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.539+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.539+0000 7f9e69d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59654/0 (socket says 172.21.15.26:59654) 2023-10-06T21:16:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.539+0000 7f9e69d9b700 1 -- 172.21.15.26:0/1709757364 learned_addr learned my addr 172.21.15.26:0/1709757364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.546+0000 7f9e6ad9d700 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd191440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.546+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.548+0000 7f9e69d9b700 1 -- 172.21.15.26:0/1709757364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 msgr2=0x5609bd191440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.548+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd191440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.548+0000 7f9e69d9b700 1 -- 172.21.15.26:0/1709757364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 msgr2=0x5609bd1863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.548+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.548+0000 7f9e69d9b700 1 -- 172.21.15.26:0/1709757364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609bcedbc60 con 0x5609bd1749e0 2023-10-06T21:16:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.549+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0962f0 tx=0x7f9e5c0d2530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a44239cd6044b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.550+0000 7f9e597fa700 1 -- 172.21.15.26:0/1709757364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0993c0 con 0x5609bd1749e0 2023-10-06T21:16:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.550+0000 7f9e597fa700 1 -- 172.21.15.26:0/1709757364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5c0991e0 con 0x5609bd1749e0 2023-10-06T21:16:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.550+0000 7f9e597fa700 1 -- 172.21.15.26:0/1709757364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c095f90 con 0x5609bd1749e0 2023-10-06T21:16:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.552+0000 7f9e8488fb80 1 -- 172.21.15.26:0/1709757364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 msgr2=0x5609bd176050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.552+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0962f0 tx=0x7f9e5c0d2530 comp rx=0 tx=0).stop 2023-10-06T21:16:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.555+0000 7f9e8488fb80 1 -- 172.21.15.26:0/1709757364 shutdown_connections 2023-10-06T21:16:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.555+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd191440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.556+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.556+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/1709757364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1749e0 0x5609bd176050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.556+0000 7f9e8488fb80 1 -- 172.21.15.26:0/1709757364 >> 172.21.15.26:0/1709757364 conn(0x5609bd0e9260 msgr2=0x5609bd18b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.557+0000 7f9e8488fb80 1 -- 172.21.15.26:0/1709757364 shutdown_connections 2023-10-06T21:16:27.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.558+0000 7f9e8488fb80 1 -- 172.21.15.26:0/1709757364 wait complete. 2023-10-06T21:16:27.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.558+0000 7f9e8488fb80 1 -- start start 2023-10-06T21:16:27.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 Processor -- start 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1749e0 0x5609bd1de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd1db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd1db7a0 con 0x5609bd1740c0 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd1db910 con 0x5609bd1749e0 2023-10-06T21:16:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bd1dbad0 con 0x5609bd17f890 2023-10-06T21:16:27.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.559+0000 7f9e8488fb80 10 client.0 _ll_register_callbacks cb 0x5609bcec1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.561+0000 7f9e6a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.561+0000 7f9e6a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59656/0 (socket says 172.21.15.26:59656) 2023-10-06T21:16:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.561+0000 7f9e6a59c700 1 -- 172.21.15.26:0/3460269297 learned_addr learned my addr 172.21.15.26:0/3460269297 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.562+0000 7f9e6a59c700 1 -- 172.21.15.26:0/3460269297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 msgr2=0x5609bd1db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.562+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd1db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.562+0000 7f9e6a59c700 1 -- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1749e0 msgr2=0x5609bd1de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.562+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1749e0 0x5609bd1de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.562+0000 7f9e6a59c700 1 -- 172.21.15.26:0/3460269297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c19e230 con 0x5609bd1740c0 2023-10-06T21:16:27.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.566+0000 7f9e6a59c700 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b370 tx=0x7f9e6400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7806fb9644c4805e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.566+0000 7f9e51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:16:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.567+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6402f070 con 0x5609bd1740c0 2023-10-06T21:16:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.567+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9e6402a4a0 con 0x5609bd1740c0 2023-10-06T21:16:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.567+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e64007b50 con 0x5609bd1740c0 2023-10-06T21:16:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.567+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e6400e040 con 0x5609bd1740c0 2023-10-06T21:16:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.570+0000 7f9e8488fb80 10 client.16080 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.570+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609bcf19250 con 0x5609bd1740c0 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e64007b50 con 0x5609bd1740c0 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e64032020 con 0x5609bd1740c0 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 10 client.16080 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 20 client.16080 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd1f5030 0x5609bd1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 4 client.16080 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:16:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.571+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609bd0c0b30 con 0x5609bd1f5030 2023-10-06T21:16:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.573+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd1f5030 0x5609bd1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.576+0000 7f9e69d9b700 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd1f5030 0x5609bd1f7450 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b1cea916290177 server_cookie=7e95a6a5ab8ae813 in_seq=0 out_seq=0 2023-10-06T21:16:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.576+0000 7f9e51ffb700 10 client.16080 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.576+0000 7f9e51ffb700 1 -- 172.21.15.26:0/3460269297 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5609bd0c0b30 con 0x5609bd1f5030 2023-10-06T21:16:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.576+0000 7f9e51ffb700 10 client.16080 handle_command_reply: tid=0 2023-10-06T21:16:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.576+0000 7f9e8488fb80 1 client.16080 shutdown 2023-10-06T21:16:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 msgr2=0x5609bd1863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400b370 tx=0x7f9e6400b680 comp rx=0 tx=0).stop 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 shutdown_connections 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609bd17f890 0x5609bd1db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5609bd1740c0 0x5609bd1863e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5609bd1f5030 0x5609bd1f7450 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 --2- 172.21.15.26:0/3460269297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609bd1749e0 0x5609bd1de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 >> 172.21.15.26:0/3460269297 conn(0x5609bd0e9260 msgr2=0x5609bd176050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.577+0000 7f9e51ffb700 0 client.16080 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.578+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 shutdown_connections 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.578+0000 7f9e8488fb80 1 -- 172.21.15.26:0/3460269297 wait complete. 2023-10-06T21:16:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.578+0000 7f9e8488fb80 20 client.16080 trim_cache size 0 max 16384 2023-10-06T21:16:27.582 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:16:27.582 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:16:27.582 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:16:27.582 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:16:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 msgr2=0x7f9e540a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 0x7f9e540a3c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e600097a0 tx=0x7f9e600047c0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 msgr2=0x7f9e6c1a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d980 tx=0x7f9e5c00dc90 comp rx=0 tx=0).stop 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 shutdown_connections 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e6c1051e0 0x7f9e6c19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9e540a17c0 0x7f9e540a3c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e6c0ffde0 0x7f9e6c1a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 --2- 172.21.15.26:0/225361062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e6c100700 0x7f9e6c1a3a80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.580+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 >> 172.21.15.26:0/225361062 conn(0x7f9e6c0fb830 msgr2=0x7f9e6c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.585+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 shutdown_connections 2023-10-06T21:16:27.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.585+0000 7f9e70fff700 1 -- 172.21.15.26:0/225361062 wait complete. 2023-10-06T21:16:27.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:16:27.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:16:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.977+0000 7f81735ff700 1 -- 172.21.15.26:0/3702999279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 msgr2=0x7f816c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.977+0000 7f81735ff700 1 --2- 172.21.15.26:0/3702999279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c10dd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8164009f90 tx=0x7f816400b330 comp rx=0 tx=0).stop 2023-10-06T21:16:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 -- 172.21.15.26:0/3702999279 shutdown_connections 2023-10-06T21:16:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 --2- 172.21.15.26:0/3702999279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c10dd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 --2- 172.21.15.26:0/3702999279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c1010b0 0x7f816c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 --2- 172.21.15.26:0/3702999279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c101a60 0x7f816c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 -- 172.21.15.26:0/3702999279 >> 172.21.15.26:0/3702999279 conn(0x7f816c0fc970 msgr2=0x7f816c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 -- 172.21.15.26:0/3702999279 shutdown_connections 2023-10-06T21:16:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.978+0000 7f81735ff700 1 -- 172.21.15.26:0/3702999279 wait complete. 2023-10-06T21:16:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 Processor -- start 2023-10-06T21:16:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 -- start start 2023-10-06T21:16:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1010b0 0x7f816c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19e470 con 0x7f816c1010b0 2023-10-06T21:16:27.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19e5e0 con 0x7f816c1061d0 2023-10-06T21:16:27.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.980+0000 7f81735ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19e7a0 con 0x7f816c19d9f0 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.981+0000 7f8171b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.981+0000 7f8171b9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47106/0 (socket says 172.21.15.26:47106) 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.981+0000 7f8171b9c700 1 -- 172.21.15.26:0/589160956 learned_addr learned my addr 172.21.15.26:0/589160956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.981+0000 7f8170b9a700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 -- 172.21.15.26:0/589160956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 msgr2=0x7f816c199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 -- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1010b0 msgr2=0x7f816c199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1010b0 0x7f816c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 -- 172.21.15.26:0/589160956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009520 con 0x7f816c19d9f0 2023-10-06T21:16:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8170b9a700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f8171b9c700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8164004c80 tx=0x7f8164004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.982+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400e040 con 0x7f816c19d9f0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.983+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8164009c40 con 0x7f816c19d9f0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.983+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c106f40 con 0x7f816c19d9f0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.983+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81640078e0 con 0x7f816c19d9f0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.983+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640052a0 con 0x7f816c19d9f0 2023-10-06T21:16:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.984+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8150005320 con 0x7f816c19d9f0 2023-10-06T21:16:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.991+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8164005400 con 0x7f816c19d9f0 2023-10-06T21:16:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.994+0000 7f81627fc700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 0x7f81580a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.994+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f81640ce6c0 con 0x7f816c19d9f0 2023-10-06T21:16:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.995+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f81640ceaa0 con 0x7f816c19d9f0 2023-10-06T21:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.996+0000 7f817139b700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 0x7f81580a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:27.997+0000 7f817139b700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 0x7f81580a3bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8168005950 tx=0x7f81680058e0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.419+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8150005190 con 0x7f816c19d9f0 2023-10-06T21:16:28.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.420+0000 7f81627fc700 1 -- 172.21.15.26:0/589160956 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11315 (secure 0 0 0) 0x7f8164097ab0 con 0x7f816c19d9f0 2023-10-06T21:16:28.423 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:16:28.423 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:16:04.424105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:16:28.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.424+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 msgr2=0x7f81580a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.424+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 0x7f81580a3bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8168005950 tx=0x7f81680058e0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.424+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 msgr2=0x7f816c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.424+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8164004c80 tx=0x7f8164004cb0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 shutdown_connections 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f816c19d9f0 0x7f816c19de70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f81580a1730 0x7f81580a3bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1010b0 0x7f816c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 --2- 172.21.15.26:0/589160956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1061d0 0x7f816c199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 >> 172.21.15.26:0/589160956 conn(0x7f816c0fc970 msgr2=0x7f816c10d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 shutdown_connections 2023-10-06T21:16:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.425+0000 7f81735ff700 1 -- 172.21.15.26:0/589160956 wait complete. 2023-10-06T21:16:28.429 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 40 2023-10-06T21:16:28.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:16:28.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:28 smithi167 ceph-mon[134494]: pgmap v188: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:16:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:28 smithi026 ceph-mon[98490]: pgmap v188: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:16:28.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:28 smithi078 ceph-mon[134896]: pgmap v188: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:16:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 -- 172.21.15.26:0/1127270958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338101060 msgr2=0x7f2338101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 --2- 172.21.15.26:0/1127270958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338101060 0x7f2338101440 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f23280098f0 tx=0x7f2328009c00 comp rx=0 tx=0).stop 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 -- 172.21.15.26:0/1127270958 shutdown_connections 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 --2- 172.21.15.26:0/1127270958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2338106180 0x7f233810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 --2- 172.21.15.26:0/1127270958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338101060 0x7f2338101440 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 --2- 172.21.15.26:0/1127270958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101a10 0x7f2338105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 -- 172.21.15.26:0/1127270958 >> 172.21.15.26:0/1127270958 conn(0x7f23380fc900 msgr2=0x7f23380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 -- 172.21.15.26:0/1127270958 shutdown_connections 2023-10-06T21:16:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.834+0000 7f233d1ff700 1 -- 172.21.15.26:0/1127270958 wait complete. 2023-10-06T21:16:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.835+0000 7f233d1ff700 1 Processor -- start 2023-10-06T21:16:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.836+0000 7f233d1ff700 1 -- start start 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338106180 0x7f233819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f233819dc30 0x7f233819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819e5f0 con 0x7f2338106180 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819e760 con 0x7f2338101060 2023-10-06T21:16:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f233d1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819e920 con 0x7f233819dc30 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f2336d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f2336d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36774/0 (socket says 172.21.15.26:36774) 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.838+0000 7f2336d9d700 1 -- 172.21.15.26:0/2333485661 learned_addr learned my addr 172.21.15.26:0/2333485661 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f233659c700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338106180 0x7f233819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f233759e700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f233819dc30 0x7f233819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f2336d9d700 1 -- 172.21.15.26:0/2333485661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f233819dc30 msgr2=0x7f233819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f2336d9d700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f233819dc30 0x7f233819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f2336d9d700 1 -- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338106180 msgr2=0x7f233819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f2336d9d700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338106180 0x7f233819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.839+0000 7f2336d9d700 1 -- 172.21.15.26:0/2333485661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328009520 con 0x7f2338101060 2023-10-06T21:16:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f2336d9d700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2328005d90 tx=0x7f232800baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f231ffff700 1 -- 172.21.15.26:0/2333485661 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328005520 con 0x7f2338101060 2023-10-06T21:16:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f231ffff700 1 -- 172.21.15.26:0/2333485661 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2328005680 con 0x7f2338101060 2023-10-06T21:16:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f231ffff700 1 -- 172.21.15.26:0/2333485661 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328017840 con 0x7f2338101060 2023-10-06T21:16:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2338110ad0 con 0x7f2338101060 2023-10-06T21:16:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.840+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2338110f30 con 0x7f2338101060 2023-10-06T21:16:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f2350b6cb80 1 Processor -- start 2023-10-06T21:16:28.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f231ffff700 1 -- 172.21.15.26:0/2333485661 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f23280179c0 con 0x7f2338101060 2023-10-06T21:16:28.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f2350b6cb80 1 -- start start 2023-10-06T21:16:28.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 0x5617a4254350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.847+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a41b2460 con 0x5617a42490c0 2023-10-06T21:16:28.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a41a3c20 con 0x5617a4254890 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a41a1540 con 0x5617a42499e0 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36778/0 (socket says 172.21.15.26:36778) 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f233659c700 1 -- 172.21.15.26:0/692567627 learned_addr learned my addr 172.21.15.26:0/692567627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f233759e700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f2336d9d700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 0x5617a4254350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.848+0000 7f233659c700 1 -- 172.21.15.26:0/692567627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 msgr2=0x5617a4254350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233659c700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 0x5617a4254350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233659c700 1 -- 172.21.15.26:0/692567627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 msgr2=0x5617a42494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233659c700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233659c700 1 -- 172.21.15.26:0/692567627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617a3fc7de0 con 0x5617a4254890 2023-10-06T21:16:28.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233759e700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:28.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f2336d9d700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 0x5617a4254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:28.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f233659c700 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f232c004660 tx=0x7f232c00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc344a35a94121 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f231d7fa700 1 -- 172.21.15.26:0/692567627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c004a30 con 0x5617a4254890 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f231d7fa700 1 -- 172.21.15.26:0/692567627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f232c0054e0 con 0x5617a4254890 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f231d7fa700 1 -- 172.21.15.26:0/692567627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c003c60 con 0x5617a4254890 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.849+0000 7f231ffff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 0x7f23200a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.850+0000 7f231ffff700 1 -- 172.21.15.26:0/2333485661 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f23280ce2b0 con 0x7f2338101060 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.850+0000 7f2350b6cb80 1 -- 172.21.15.26:0/692567627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 msgr2=0x5617a4256c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.850+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f232c004660 tx=0x7f232c00ba10 comp rx=0 tx=0).stop 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.850+0000 7f233759e700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 0x7f23200a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.851+0000 7f2350b6cb80 1 -- 172.21.15.26:0/692567627 shutdown_connections 2023-10-06T21:16:28.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.851+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a4254890 0x5617a4256c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.851+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.851+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/692567627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42499e0 0x5617a4254350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.851+0000 7f2350b6cb80 1 -- 172.21.15.26:0/692567627 >> 172.21.15.26:0/692567627 conn(0x5617a41be260 msgr2=0x5617a4245450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.852+0000 7f2350b6cb80 1 -- 172.21.15.26:0/692567627 shutdown_connections 2023-10-06T21:16:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.852+0000 7f2350b6cb80 1 -- 172.21.15.26:0/692567627 wait complete. 2023-10-06T21:16:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.853+0000 7f2350b6cb80 1 -- start start 2023-10-06T21:16:28.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.854+0000 7f2350b6cb80 1 Processor -- start 2023-10-06T21:16:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.854+0000 7f233759e700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 0x7f23200a3b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23300094c0 tx=0x7f2330009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.854+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a42542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.854+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4251550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.854+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a419ce30 con 0x5617a4254890 2023-10-06T21:16:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a4139710 con 0x5617a42499e0 2023-10-06T21:16:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a41393b0 con 0x5617a42490c0 2023-10-06T21:16:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f2350b6cb80 10 client.0 _ll_register_callbacks cb 0x5617a3f96800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f233759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f233759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59714/0 (socket says 172.21.15.26:59714) 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f233759e700 1 -- 172.21.15.26:0/3070845682 learned_addr learned my addr 172.21.15.26:0/3070845682 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f233659c700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4251550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.855+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a42542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 -- 172.21.15.26:0/3070845682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 msgr2=0x5617a42542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a42542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 -- 172.21.15.26:0/3070845682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 msgr2=0x5617a4251c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 -- 172.21.15.26:0/3070845682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c004290 con 0x5617a42499e0 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233759e700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f233659c700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4251550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f232c004630 tx=0x7f232c004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=822ead4204292c28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f231aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c01d070 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f232c004750 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.856+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617a423cf10 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.857+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f232c004910 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.857+0000 7f2350b6cb80 10 client.26324 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.857+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5617a3fb0c60 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617a423cf10 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f232c02a050 con 0x5617a42499e0 2023-10-06T21:16:28.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 10 client.26324 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 10 client.26324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 20 client.26324 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a4275110 0x5617a4277530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 4 client.26324 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.858+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617a4195e10 con 0x5617a4275110 2023-10-06T21:16:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.859+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a4275110 0x5617a4277530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.860+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a4275110 0x5617a4277530 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9152f3777495783 server_cookie=496a57aba39b6bfa in_seq=0 out_seq=0 2023-10-06T21:16:28.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.860+0000 7f231aff5700 10 client.26324 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:16:28.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f231aff5700 1 -- 172.21.15.26:0/3070845682 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617a4195e10 con 0x5617a4275110 2023-10-06T21:16:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f231aff5700 10 client.26324 handle_command_reply: tid=0 2023-10-06T21:16:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 client.26324 shutdown 2023-10-06T21:16:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 msgr2=0x5617a4251550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4251550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f232c004630 tx=0x7f232c004e90 comp rx=0 tx=0).stop 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 shutdown_connections 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a4275110 0x5617a4277530 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a4251c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a42542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3070845682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4251550 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 >> 172.21.15.26:0/3070845682 conn(0x5617a41be260 msgr2=0x5617a4245040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.861+0000 7f231aff5700 0 client.26324 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:16:28.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.862+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 shutdown_connections 2023-10-06T21:16:28.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.862+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3070845682 wait complete. 2023-10-06T21:16:28.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.862+0000 7f2350b6cb80 20 client.26324 trim_cache size 0 max 16384 2023-10-06T21:16:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 Processor -- start 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 -- start start 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a424bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a424c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a401d920 con 0x5617a4254890 2023-10-06T21:16:28.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a423cae0 con 0x5617a42499e0 2023-10-06T21:16:28.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.883+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a424d3e0 con 0x5617a42490c0 2023-10-06T21:16:28.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36806/0 (socket says 172.21.15.26:36806) 2023-10-06T21:16:28.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 -- 172.21.15.26:0/1389804194 learned_addr learned my addr 172.21.15.26:0/1389804194 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f2336d9d700 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a424bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 -- 172.21.15.26:0/1389804194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 msgr2=0x5617a424bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a424bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 -- 172.21.15.26:0/1389804194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 msgr2=0x5617a424c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a424c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.884+0000 7f233659c700 1 -- 172.21.15.26:0/1389804194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617a3fee250 con 0x5617a42499e0 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.885+0000 7f2336d9d700 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a424bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.885+0000 7f233659c700 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b470 tx=0x7f232c02d940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba26bbe0a1593e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.885+0000 7f231dffb700 1 -- 172.21.15.26:0/1389804194 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c01d070 con 0x5617a42499e0 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.885+0000 7f231dffb700 1 -- 172.21.15.26:0/1389804194 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f232c01e040 con 0x5617a42499e0 2023-10-06T21:16:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.885+0000 7f231dffb700 1 -- 172.21.15.26:0/1389804194 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c003a00 con 0x5617a42499e0 2023-10-06T21:16:28.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 -- 172.21.15.26:0/1389804194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 msgr2=0x5617a4253e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b470 tx=0x7f232c02d940 comp rx=0 tx=0).stop 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 -- 172.21.15.26:0/1389804194 shutdown_connections 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a4254890 0x5617a424c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a42490c0 0x5617a424bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/1389804194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4253e00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 -- 172.21.15.26:0/1389804194 >> 172.21.15.26:0/1389804194 conn(0x5617a41be260 msgr2=0x5617a425fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.886+0000 7f2350b6cb80 1 -- 172.21.15.26:0/1389804194 shutdown_connections 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.887+0000 7f2350b6cb80 1 -- 172.21.15.26:0/1389804194 wait complete. 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.887+0000 7f2350b6cb80 1 -- start start 2023-10-06T21:16:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.887+0000 7f2350b6cb80 1 Processor -- start 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a4254890 0x5617a42c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a42c8480 con 0x5617a42490c0 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a42c85c0 con 0x5617a42499e0 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617a42c8700 con 0x5617a4254890 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f2350b6cb80 10 client.0 _ll_register_callbacks cb 0x7f232c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f233659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36822/0 (socket says 172.21.15.26:36822) 2023-10-06T21:16:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.888+0000 7f233659c700 1 -- 172.21.15.26:0/3138114930 learned_addr learned my addr 172.21.15.26:0/3138114930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233759e700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a4254890 0x5617a42c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233659c700 1 -- 172.21.15.26:0/3138114930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a4254890 msgr2=0x5617a42c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233659c700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a4254890 0x5617a42c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233659c700 1 -- 172.21.15.26:0/3138114930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 msgr2=0x5617a42595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233659c700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f233659c700 1 -- 172.21.15.26:0/3138114930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617a3fb0c60 con 0x5617a42499e0 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.890+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.891+0000 7f233659c700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f232c02d4c0 tx=0x7f232c004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f41984479e36fca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.892+0000 7f231aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:28.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.892+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c03d070 con 0x5617a42499e0 2023-10-06T21:16:28.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.892+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f232c000e40 con 0x5617a42499e0 2023-10-06T21:16:28.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.893+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f232c0054e0 con 0x5617a42499e0 2023-10-06T21:16:28.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.894+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f232c01d040 con 0x5617a42499e0 2023-10-06T21:16:28.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.894+0000 7f2350b6cb80 10 client.26330 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:28.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.894+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5617a3fc62b0 con 0x5617a42499e0 2023-10-06T21:16:28.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.895+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f232c046030 con 0x5617a42499e0 2023-10-06T21:16:28.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.895+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f232c035070 con 0x5617a42499e0 2023-10-06T21:16:28.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 10 client.26330 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:28.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 10 client.26330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:16:28.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 20 client.26330 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:28.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a42c99b0 0x5617a42cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:28.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 4 client.26330 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:16:28.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5617a4195b30 con 0x5617a42c99b0 2023-10-06T21:16:28.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.896+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a42c99b0 0x5617a42cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f231aff5700 10 client.26330 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:16:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f2336d9d700 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a42c99b0 0x5617a42cbdd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7693926bca975918 server_cookie=a28c7efbe19a1b1f in_seq=0 out_seq=0 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f231aff5700 1 -- 172.21.15.26:0/3138114930 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5617a4195b30 con 0x5617a42c99b0 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f231aff5700 10 client.26330 handle_command_reply: tid=0 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f2350b6cb80 1 client.26330 shutdown 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 msgr2=0x5617a4259af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.897+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f232c02d4c0 tx=0x7f232c004c00 comp rx=0 tx=0).stop 2023-10-06T21:16:28.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 shutdown_connections 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x5617a42c99b0 0x5617a42cbdd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617a4254890 0x5617a42c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5617a42490c0 0x5617a42595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 --2- 172.21.15.26:0/3138114930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617a42499e0 0x5617a4259af0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f231aff5700 0 client.26330 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:16:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 >> 172.21.15.26:0/3138114930 conn(0x5617a41be260 msgr2=0x5617a424c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 shutdown_connections 2023-10-06T21:16:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 1 -- 172.21.15.26:0/3138114930 wait complete. 2023-10-06T21:16:28.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.898+0000 7f2350b6cb80 20 client.26330 trim_cache size 0 max 16384 2023-10-06T21:16:28.904 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:16:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 msgr2=0x7f23200a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 0x7f23200a3b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23300094c0 tx=0x7f2330009040 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 msgr2=0x7f233819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2328005d90 tx=0x7f232800baa0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 shutdown_connections 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f233819dc30 0x7f233819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2338101060 0x7f233819ee90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f23200a16b0 0x7f23200a3b70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 --2- 172.21.15.26:0/2333485661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2338106180 0x7f233819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 >> 172.21.15.26:0/2333485661 conn(0x7f23380fc900 msgr2=0x7f23380fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 shutdown_connections 2023-10-06T21:16:28.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:28.899+0000 7f233d1ff700 1 -- 172.21.15.26:0/2333485661 wait complete. 2023-10-06T21:16:28.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:16:29.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.292+0000 7fa227dff700 1 -- 172.21.15.26:0/867589484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 msgr2=0x7fa220101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.292+0000 7fa227dff700 1 --2- 172.21.15.26:0/867589484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220101490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009990 tx=0x7fa21c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 -- 172.21.15.26:0/867589484 shutdown_connections 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 --2- 172.21.15.26:0/867589484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 0x7fa22010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 --2- 172.21.15.26:0/867589484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220101490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 --2- 172.21.15.26:0/867589484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220101a60 0x7fa220105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 -- 172.21.15.26:0/867589484 >> 172.21.15.26:0/867589484 conn(0x7fa2200fc930 msgr2=0x7fa2200fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.293+0000 7fa227dff700 1 -- 172.21.15.26:0/867589484 shutdown_connections 2023-10-06T21:16:29.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.294+0000 7fa227dff700 1 -- 172.21.15.26:0/867589484 wait complete. 2023-10-06T21:16:29.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.295+0000 7fa227dff700 1 Processor -- start 2023-10-06T21:16:29.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.296+0000 7fa227dff700 1 -- start start 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 0x7fa220107210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a9cf0 con 0x7fa220107750 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a9e30 con 0x7fa2201061d0 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa227dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201a9fa0 con 0x7fa2201010b0 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa22639c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa22639c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59740/0 (socket says 172.21.15.26:59740) 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa22639c700 1 -- 172.21.15.26:0/2698877551 learned_addr learned my addr 172.21.15.26:0/2698877551 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.297+0000 7fa22539a700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 0x7fa220107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.298+0000 7fa22639c700 1 -- 172.21.15.26:0/2698877551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 msgr2=0x7fa220106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.298+0000 7fa22639c700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220106cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.298+0000 7fa22639c700 1 -- 172.21.15.26:0/2698877551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 msgr2=0x7fa220107210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.298+0000 7fa22639c700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 0x7fa220107210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.298+0000 7fa22639c700 1 -- 172.21.15.26:0/2698877551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa220107750 2023-10-06T21:16:29.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.300+0000 7fa22639c700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa218007ee0 tx=0x7fa21800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.300+0000 7fa216ffd700 1 -- 172.21.15.26:0/2698877551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218017070 con 0x7fa220107750 2023-10-06T21:16:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.300+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2201aa280 con 0x7fa220107750 2023-10-06T21:16:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.300+0000 7fa216ffd700 1 -- 172.21.15.26:0/2698877551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21800f040 con 0x7fa220107750 2023-10-06T21:16:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.301+0000 7fa216ffd700 1 -- 172.21.15.26:0/2698877551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218005580 con 0x7fa220107750 2023-10-06T21:16:29.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.301+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2201aa790 con 0x7fa220107750 2023-10-06T21:16:29.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.304+0000 7fa216ffd700 1 -- 172.21.15.26:0/2698877551 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa2180057a0 con 0x7fa220107750 2023-10-06T21:16:29.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.305+0000 7fa23b812b80 1 Processor -- start 2023-10-06T21:16:29.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 -- start start 2023-10-06T21:16:29.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b9e0 0x563c3ea76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea78c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e9d4460 con 0x563c3ea6b9e0 2023-10-06T21:16:29.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.306+0000 7fa23b812b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e9c5c20 con 0x563c3ea6b0c0 2023-10-06T21:16:29.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.307+0000 7fa23b812b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e9c3540 con 0x563c3ea76890 2023-10-06T21:16:29.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36834/0 (socket says 172.21.15.26:36834) 2023-10-06T21:16:29.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 -- 172.21.15.26:0/2205493252 learned_addr learned my addr 172.21.15.26:0/2205493252 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b9e0 0x563c3ea76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea78c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 -- 172.21.15.26:0/2205493252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 msgr2=0x563c3ea78c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea78c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 -- 172.21.15.26:0/2205493252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b9e0 msgr2=0x563c3ea76350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b9e0 0x563c3ea76350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.308+0000 7fa22539a700 1 -- 172.21.15.26:0/2205493252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c3e7e9de0 con 0x563c3ea6b0c0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.309+0000 7fa22539a700 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa2100125a0 tx=0x7fa21000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de1142ab063f0dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.309+0000 7fa20bfff700 1 -- 172.21.15.26:0/2205493252 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x563c3ea6b0c0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.309+0000 7fa20bfff700 1 -- 172.21.15.26:0/2205493252 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210012e30 con 0x563c3ea6b0c0 2023-10-06T21:16:29.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.309+0000 7fa20bfff700 1 -- 172.21.15.26:0/2205493252 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210005020 con 0x563c3ea6b0c0 2023-10-06T21:16:29.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.310+0000 7fa216ffd700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 0x7fa20c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.310+0000 7fa216ffd700 1 -- 172.21.15.26:0/2698877551 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa2180cc7c0 con 0x7fa220107750 2023-10-06T21:16:29.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.310+0000 7fa23b812b80 1 -- 172.21.15.26:0/2205493252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 msgr2=0x563c3ea6b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.310+0000 7fa23b812b80 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa2100125a0 tx=0x7fa21000b650 comp rx=0 tx=0).stop 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa23b812b80 1 -- 172.21.15.26:0/2205493252 shutdown_connections 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa23b812b80 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea78c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa23b812b80 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6b4a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa23b812b80 1 --2- 172.21.15.26:0/2205493252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b9e0 0x563c3ea76350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa23b812b80 1 -- 172.21.15.26:0/2205493252 >> 172.21.15.26:0/2205493252 conn(0x563c3e9e0260 msgr2=0x563c3ea67450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.311+0000 7fa225b9b700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 0x7fa20c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.312+0000 7fa23b812b80 1 -- 172.21.15.26:0/2205493252 shutdown_connections 2023-10-06T21:16:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.312+0000 7fa23b812b80 1 -- 172.21.15.26:0/2205493252 wait complete. 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.312+0000 7fa23b812b80 1 -- start start 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.313+0000 7fa23b812b80 1 Processor -- start 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.313+0000 7fa225b9b700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 0x7fa20c0a3b30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b5c0 tx=0x7fa21c005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.313+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 0x563c3ea76310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.313+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea73c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa23b812b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e9bee30 con 0x563c3ea76890 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa23b812b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e95b710 con 0x563c3ea6b9e0 2023-10-06T21:16:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa23b812b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e95b3b0 con 0x563c3ea6b0c0 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa23b812b80 10 client.0 _ll_register_callbacks cb 0x563c3e7b8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 0x563c3ea76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 0x563c3ea76310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47192/0 (socket says 172.21.15.26:47192) 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36840/0 (socket says 172.21.15.26:36840) 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa225b9b700 1 -- 172.21.15.26:0/1547000820 learned_addr learned my addr 172.21.15.26:0/1547000820 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22639c700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea73c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 -- 172.21.15.26:0/1547000820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 msgr2=0x563c3ea76310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 0x563c3ea76310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 -- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 msgr2=0x563c3ea73c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea73c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.314+0000 7fa22539a700 1 -- 172.21.15.26:0/1547000820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21000be50 con 0x563c3ea6b9e0 2023-10-06T21:16:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.317+0000 7fa22539a700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa210005950 tx=0x7fa21000b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632bc56510ab6c2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.317+0000 7fa2097fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.317+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x563c3ea6b9e0 2023-10-06T21:16:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.317+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa210004440 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.317+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c3ea5ef10 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa210003680 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 10 client.26342 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563c3e7d2c60 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c3ea5ef10 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa21001e860 con 0x563c3ea6b9e0 2023-10-06T21:16:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 10 client.26342 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:29.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 10 client.26342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:16:29.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 20 client.26342 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:29.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3ea97310 0x563c3ea99730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 4 client.26342 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:29.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.318+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c3e9b7e10 con 0x563c3ea97310 2023-10-06T21:16:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.319+0000 7fa225b9b700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3ea97310 0x563c3ea99730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.320+0000 7fa225b9b700 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3ea97310 0x563c3ea99730 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3580004df765c31 server_cookie=6702f0a6480033e4 in_seq=0 out_seq=0 2023-10-06T21:16:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.321+0000 7fa2097fa700 10 client.26342 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa2097fa700 1 -- 172.21.15.26:0/1547000820 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c3e9b7e10 con 0x563c3ea97310 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa2097fa700 10 client.26342 handle_command_reply: tid=0 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 client.26342 shutdown 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 msgr2=0x563c3ea73590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa210005950 tx=0x7fa21000b780 comp rx=0 tx=0).stop 2023-10-06T21:16:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 shutdown_connections 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea73c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3ea97310 0x563c3ea99730 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b0c0 0x563c3ea76310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 --2- 172.21.15.26:0/1547000820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea73590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 >> 172.21.15.26:0/1547000820 conn(0x563c3e9e0260 msgr2=0x563c3ea67040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa2097fa700 0 client.26342 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 shutdown_connections 2023-10-06T21:16:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.322+0000 7fa23b812b80 1 -- 172.21.15.26:0/1547000820 wait complete. 2023-10-06T21:16:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.323+0000 7fa23b812b80 20 client.26342 trim_cache size 0 max 16384 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 Processor -- start 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 -- start start 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea72d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3e83f920 con 0x563c3ea6b0c0 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3ea74b70 con 0x563c3ea6b9e0 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa23b812b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3ea6c8b0 con 0x563c3ea76890 2023-10-06T21:16:29.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa22639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea72d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa22539a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36844/0 (socket says 172.21.15.26:36844) 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa22639c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea72d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47208/0 (socket says 172.21.15.26:47208) 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa22539a700 1 -- 172.21.15.26:0/3694578251 learned_addr learned my addr 172.21.15.26:0/3694578251 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.342+0000 7fa225b9b700 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 -- 172.21.15.26:0/3694578251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 msgr2=0x563c3ea72d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea72d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 -- 172.21.15.26:0/3694578251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 msgr2=0x563c3ea6e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 -- 172.21.15.26:0/3694578251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c3e810250 con 0x563c3ea6b9e0 2023-10-06T21:16:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa225b9b700 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa22539a700 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa210004000 tx=0x7fa210004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88703e94e07aad91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa214bff700 1 -- 172.21.15.26:0/3694578251 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x563c3ea6b9e0 2023-10-06T21:16:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa214bff700 1 -- 172.21.15.26:0/3694578251 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21000b910 con 0x563c3ea6b9e0 2023-10-06T21:16:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.343+0000 7fa214bff700 1 -- 172.21.15.26:0/3694578251 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100038e0 con 0x563c3ea6b9e0 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.344+0000 7fa23b812b80 1 -- 172.21.15.26:0/3694578251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 msgr2=0x563c3ea8e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.344+0000 7fa23b812b80 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa210004000 tx=0x7fa210004960 comp rx=0 tx=0).stop 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.345+0000 7fa23b812b80 1 -- 172.21.15.26:0/3694578251 shutdown_connections 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.345+0000 7fa23b812b80 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea76890 0x563c3ea72d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.345+0000 7fa23b812b80 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea6b0c0 0x563c3ea6e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.345+0000 7fa23b812b80 1 --2- 172.21.15.26:0/3694578251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8e410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.345+0000 7fa23b812b80 1 -- 172.21.15.26:0/3694578251 >> 172.21.15.26:0/3694578251 conn(0x563c3e9e0260 msgr2=0x563c3ea936b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.346+0000 7fa23b812b80 1 -- 172.21.15.26:0/3694578251 shutdown_connections 2023-10-06T21:16:29.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.346+0000 7fa23b812b80 1 -- 172.21.15.26:0/3694578251 wait complete. 2023-10-06T21:16:29.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.346+0000 7fa23b812b80 1 -- start start 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 Processor -- start 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea8a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3eaea240 con 0x563c3ea76890 2023-10-06T21:16:29.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3eaea380 con 0x563c3ea6b0c0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3eaea540 con 0x563c3ea6b9e0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.347+0000 7fa23b812b80 10 client.0 _ll_register_callbacks cb 0x7fa21000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea8a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47210/0 (socket says 172.21.15.26:47210) 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa225b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea8a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36850/0 (socket says 172.21.15.26:36850) 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 -- 172.21.15.26:0/832230469 learned_addr learned my addr 172.21.15.26:0/832230469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22639c700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 -- 172.21.15.26:0/832230469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 msgr2=0x563c3ea8a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea8a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 -- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 msgr2=0x563c3ea8ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea8ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.348+0000 7fa22539a700 1 -- 172.21.15.26:0/832230469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c3e7d2c60 con 0x563c3ea6b9e0 2023-10-06T21:16:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.349+0000 7fa22539a700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2100048c0 tx=0x7fa21000bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3071a04ba5dfad50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.349+0000 7fa2097fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.350+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210017040 con 0x563c3ea6b9e0 2023-10-06T21:16:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.350+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa210000e40 con 0x563c3ea6b9e0 2023-10-06T21:16:29.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.350+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa210012e30 con 0x563c3ea6b9e0 2023-10-06T21:16:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.361+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa21000b410 con 0x563c3ea6b9e0 2023-10-06T21:16:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.366+0000 7fa23b812b80 10 client.26308 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.366+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563c3e7e82b0 con 0x563c3ea6b9e0 2023-10-06T21:16:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210012e30 con 0x563c3ea6b9e0 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa210041050 con 0x563c3ea6b9e0 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 10 client.26308 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 10 client.26308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 20 client.26308 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3eaeb8b0 0x563c3eaedcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 4 client.26308 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:16:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.372+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c3e9b7b30 con 0x563c3eaeb8b0 2023-10-06T21:16:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.375+0000 7fa225b9b700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3eaeb8b0 0x563c3eaedcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.379+0000 7fa225b9b700 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3eaeb8b0 0x563c3eaedcd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28504143b89914a9 server_cookie=d178ba9b5466eace in_seq=0 out_seq=0 2023-10-06T21:16:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.380+0000 7fa2097fa700 10 client.26308 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:16:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.382+0000 7fa2097fa700 1 -- 172.21.15.26:0/832230469 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c3e9b7b30 con 0x563c3eaeb8b0 2023-10-06T21:16:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.382+0000 7fa2097fa700 10 client.26308 handle_command_reply: tid=0 2023-10-06T21:16:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.382+0000 7fa23b812b80 1 client.26308 shutdown 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 msgr2=0x563c3ea8a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2100048c0 tx=0x7fa21000bb90 comp rx=0 tx=0).stop 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 shutdown_connections 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c3ea76890 0x563c3ea8ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563c3ea6b0c0 0x563c3ea8a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x563c3eaeb8b0 0x563c3eaedcd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 --2- 172.21.15.26:0/832230469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c3ea6b9e0 0x563c3ea8a930 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 >> 172.21.15.26:0/832230469 conn(0x563c3e9e0260 msgr2=0x563c3ea92b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa2097fa700 0 client.26308 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:16:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 shutdown_connections 2023-10-06T21:16:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.386+0000 7fa23b812b80 1 -- 172.21.15.26:0/832230469 wait complete. 2023-10-06T21:16:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.387+0000 7fa23b812b80 20 client.26308 trim_cache size 0 max 16384 2023-10-06T21:16:29.390 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:16:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.388+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 msgr2=0x7fa20c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.388+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 0x7fa20c0a3b30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b5c0 tx=0x7fa21c005fd0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.388+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 msgr2=0x7fa2201a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.388+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa218007ee0 tx=0x7fa21800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 shutdown_connections 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa20c0a1670 0x7fa20c0a3b30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa220107750 0x7fa2201a97b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2201010b0 0x7fa220106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 --2- 172.21.15.26:0/2698877551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2201061d0 0x7fa220107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.389+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 >> 172.21.15.26:0/2698877551 conn(0x7fa2200fc930 msgr2=0x7fa22010d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.391+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 shutdown_connections 2023-10-06T21:16:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.391+0000 7fa227dff700 1 -- 172.21.15.26:0/2698877551 wait complete. 2023-10-06T21:16:29.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:16:29.523 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:29 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/589160956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:16:29.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:29 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/589160956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:16:29.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:29 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/589160956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 -- 172.21.15.26:0/3049169372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c100ff2f0 msgr2=0x7f1c100ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 --2- 172.21.15.26:0/3049169372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c100ff2f0 0x7f1c100ff6d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c0401a9a0 tx=0x7f1c0401acb0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 -- 172.21.15.26:0/3049169372 shutdown_connections 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 --2- 172.21.15.26:0/3049169372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 0x7f1c1010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 --2- 172.21.15.26:0/3049169372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c100ff2f0 0x7f1c100ff6d0 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c0401a9a0 tx=0x7f1c0401acb0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 --2- 172.21.15.26:0/3049169372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c100ffca0 0x7f1c10103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.794+0000 7f1c14bff700 1 -- 172.21.15.26:0/3049169372 >> 172.21.15.26:0/3049169372 conn(0x7f1c10076ed0 msgr2=0x7f1c100fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.800+0000 7f1c14bff700 1 -- 172.21.15.26:0/3049169372 shutdown_connections 2023-10-06T21:16:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.800+0000 7f1c14bff700 1 -- 172.21.15.26:0/3049169372 wait complete. 2023-10-06T21:16:29.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.801+0000 7f1c14bff700 1 Processor -- start 2023-10-06T21:16:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.801+0000 7f1c14bff700 1 -- start start 2023-10-06T21:16:29.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 0x7f1c10107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c1019efa0 0x7f1c101a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a1a30 con 0x7f1c10104410 2023-10-06T21:16:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a1ba0 con 0x7f1c1019efa0 2023-10-06T21:16:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c14bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a1d60 con 0x7f1c101078d0 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c0dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c0dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47238/0 (socket says 172.21.15.26:47238) 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/2531480823 learned_addr learned my addr 172.21.15.26:0/2531480823 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.802+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 0x7f1c10107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c1019efa0 0x7f1c101a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/2531480823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c1019efa0 msgr2=0x7f1c101a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c1019efa0 0x7f1c101a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 msgr2=0x7f1c10107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 0x7f1c10107390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/2531480823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0401a600 con 0x7f1c101078d0 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.803+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e970 tx=0x7f1c0000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.804+0000 7f1bff7fe700 1 -- 172.21.15.26:0/2531480823 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00009790 con 0x7f1c101078d0 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.804+0000 7f1bff7fe700 1 -- 172.21.15.26:0/2531480823 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c00004cf0 con 0x7f1c101078d0 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.804+0000 7f1bff7fe700 1 -- 172.21.15.26:0/2531480823 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00005680 con 0x7f1c101078d0 2023-10-06T21:16:29.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.805+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c101917c0 con 0x7f1c101078d0 2023-10-06T21:16:29.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.806+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10191c50 con 0x7f1c101078d0 2023-10-06T21:16:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.812+0000 7f1c28421b80 1 Processor -- start 2023-10-06T21:16:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.813+0000 7f1bff7fe700 1 -- 172.21.15.26:0/2531480823 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1c00004750 con 0x7f1c101078d0 2023-10-06T21:16:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.814+0000 7f1bff7fe700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 0x7f1bf80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.815+0000 7f1bff7fe700 1 -- 172.21.15.26:0/2531480823 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c00014070 con 0x7f1c101078d0 2023-10-06T21:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.815+0000 7f1c28421b80 1 -- start start 2023-10-06T21:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 0x7f1bf80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a3399460 con 0x55e7a34300c0 2023-10-06T21:16:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a338ac20 con 0x55e7a343b890 2023-10-06T21:16:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.816+0000 7f1c28421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a3388540 con 0x55e7a34309e0 2023-10-06T21:16:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36878/0 (socket says 172.21.15.26:36878) 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 -- 172.21.15.26:0/3087609627 learned_addr learned my addr 172.21.15.26:0/3087609627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 -- 172.21.15.26:0/3087609627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 msgr2=0x55e7a343b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 -- 172.21.15.26:0/3087609627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 msgr2=0x55e7a34304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 -- 172.21.15.26:0/3087609627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7a31aede0 con 0x55e7a343b890 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.817+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 0x7f1bf80b4cc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c080076f0 tx=0x7f1c08007ab0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.818+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c08007eb0 tx=0x7f1c0801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae1ff61cb061747 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.818+0000 7f1bfcff9700 1 -- 172.21.15.26:0/3087609627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080137b0 con 0x55e7a343b890 2023-10-06T21:16:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.818+0000 7f1bfcff9700 1 -- 172.21.15.26:0/3087609627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c08013dd0 con 0x55e7a343b890 2023-10-06T21:16:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.818+0000 7f1bfcff9700 1 -- 172.21.15.26:0/3087609627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08023cb0 con 0x55e7a343b890 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 -- 172.21.15.26:0/3087609627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 msgr2=0x55e7a343dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c08007eb0 tx=0x7f1c0801d000 comp rx=0 tx=0).stop 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 -- 172.21.15.26:0/3087609627 shutdown_connections 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a343dc80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3087609627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 -- 172.21.15.26:0/3087609627 >> 172.21.15.26:0/3087609627 conn(0x55e7a33a5260 msgr2=0x55e7a342c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 -- 172.21.15.26:0/3087609627 shutdown_connections 2023-10-06T21:16:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.820+0000 7f1c28421b80 1 -- 172.21.15.26:0/3087609627 wait complete. 2023-10-06T21:16:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.821+0000 7f1c28421b80 1 -- start start 2023-10-06T21:16:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.821+0000 7f1c28421b80 1 Processor -- start 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 0x55e7a3438510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 0x55e7a3438b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a3383e30 con 0x55e7a34309e0 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a3320710 con 0x55e7a34300c0 2023-10-06T21:16:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.822+0000 7f1c28421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a33203b0 con 0x55e7a343b890 2023-10-06T21:16:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c28421b80 10 client.0 _ll_register_callbacks cb 0x55e7a317d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36890/0 (socket says 172.21.15.26:36890) 2023-10-06T21:16:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 -- 172.21.15.26:0/3768845402 learned_addr learned my addr 172.21.15.26:0/3768845402 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 0x55e7a3438510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 0x55e7a3438b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 -- 172.21.15.26:0/3768845402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 msgr2=0x55e7a3438b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 0x55e7a3438b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 -- 172.21.15.26:0/3768845402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 msgr2=0x55e7a3438510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 0x55e7a3438510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0e59c700 1 -- 172.21.15.26:0/3768845402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c08019400 con 0x55e7a34300c0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.823+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 0x55e7a3438b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 0x55e7a3438510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006a5b0 tx=0x7f1c0401a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d6e8a9c72eaaaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1bf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040052b0 con 0x55e7a34300c0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c04005410 con 0x55e7a34300c0 2023-10-06T21:16:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.824+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7a3423f10 con 0x55e7a34300c0 2023-10-06T21:16:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.825+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c040055d0 con 0x55e7a34300c0 2023-10-06T21:16:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.825+0000 7f1c28421b80 10 client.26360 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.825+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7a3197c60 con 0x55e7a34300c0 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.825+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c04040020 con 0x55e7a34300c0 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1c04003800 con 0x55e7a34300c0 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 20 client.26360 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a344b230 0x55e7a344d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 4 client.26360 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.826+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7a337ce10 con 0x55e7a344b230 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.827+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a344b230 0x55e7a344d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.827+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a344b230 0x55e7a344d650 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9314ae3b9925321c server_cookie=2dd2405dd5b76779 in_seq=0 out_seq=0 2023-10-06T21:16:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.827+0000 7f1bf5ffb700 10 client.26360 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:29.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/3768845402 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7a337ce10 con 0x55e7a344b230 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1bf5ffb700 10 client.26360 handle_command_reply: tid=0 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 client.26360 shutdown 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 msgr2=0x55e7a343b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006a5b0 tx=0x7f1c0401a180 comp rx=0 tx=0).stop 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 shutdown_connections 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a343b890 0x55e7a3438b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a344b230 0x55e7a344d650 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a34300c0 0x55e7a343b290 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 --2- 172.21.15.26:0/3768845402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34309e0 0x55e7a3438510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 >> 172.21.15.26:0/3768845402 conn(0x55e7a33a5260 msgr2=0x55e7a342bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.828+0000 7f1bf5ffb700 0 client.26360 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.829+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 shutdown_connections 2023-10-06T21:16:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.829+0000 7f1c28421b80 1 -- 172.21.15.26:0/3768845402 wait complete. 2023-10-06T21:16:29.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.829+0000 7f1c28421b80 20 client.26360 trim_cache size 0 max 16384 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 Processor -- start 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 -- start start 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a3437d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a34338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a3204920 con 0x55e7a34300c0 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a342d0e0 con 0x55e7a343b890 2023-10-06T21:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.847+0000 7f1c28421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a34382b0 con 0x55e7a34309e0 2023-10-06T21:16:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a3437d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47264/0 (socket says 172.21.15.26:47264) 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/4230158361 learned_addr learned my addr 172.21.15.26:0/4230158361 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a34338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/4230158361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 msgr2=0x55e7a34338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a34338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/4230158361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 msgr2=0x55e7a3437d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a3437d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/4230158361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7a31d5250 con 0x55e7a34309e0 2023-10-06T21:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.848+0000 7f1c0ed9d700 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a34338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.850+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c5b0 tx=0x7f1c00005120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e382dae7b9fedc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.850+0000 7f1bfd7fa700 1 -- 172.21.15.26:0/4230158361 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0009a360 con 0x55e7a34309e0 2023-10-06T21:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.850+0000 7f1bfd7fa700 1 -- 172.21.15.26:0/4230158361 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c000af040 con 0x55e7a34309e0 2023-10-06T21:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.850+0000 7f1bfd7fa700 1 -- 172.21.15.26:0/4230158361 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000ccd0 con 0x55e7a34309e0 2023-10-06T21:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 -- 172.21.15.26:0/4230158361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 msgr2=0x55e7a343ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c5b0 tx=0x7f1c00005120 comp rx=0 tx=0).stop 2023-10-06T21:16:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 -- 172.21.15.26:0/4230158361 shutdown_connections 2023-10-06T21:16:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a34338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a3437d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 --2- 172.21.15.26:0/4230158361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a343ae30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.851+0000 7f1c28421b80 1 -- 172.21.15.26:0/4230158361 >> 172.21.15.26:0/4230158361 conn(0x55e7a33a5260 msgr2=0x55e7a344d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.852+0000 7f1c28421b80 1 -- 172.21.15.26:0/4230158361 shutdown_connections 2023-10-06T21:16:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.852+0000 7f1c28421b80 1 -- 172.21.15.26:0/4230158361 wait complete. 2023-10-06T21:16:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.852+0000 7f1c28421b80 1 -- start start 2023-10-06T21:16:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.852+0000 7f1c28421b80 1 Processor -- start 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a3440a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a349e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a34411a0 con 0x55e7a34300c0 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a349e5c0 con 0x55e7a343b890 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7a349e780 con 0x55e7a34309e0 2023-10-06T21:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c28421b80 10 client.0 _ll_register_callbacks cb 0x7f1c04005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:29.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a349e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.853+0000 7f1c0dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a3440a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a349e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:36914/0 (socket says 172.21.15.26:36914) 2023-10-06T21:16:29.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0ed9d700 1 -- 172.21.15.26:0/604911436 learned_addr learned my addr 172.21.15.26:0/604911436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:29.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 msgr2=0x55e7a349e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a349e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/604911436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 msgr2=0x55e7a34404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 -- 172.21.15.26:0/604911436 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10191c50 con 0x55e7a34309e0 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1c0dd9b700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a3440a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c00004f20 tx=0x7f1c00096750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9e0a1b02dd6db9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1bf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:29.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.854+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000a26f0 con 0x55e7a34309e0 2023-10-06T21:16:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1c00020e70 con 0x55e7a34309e0 2023-10-06T21:16:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c08013dd0 con 0x55e7a34309e0 2023-10-06T21:16:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c0000ccd0 con 0x55e7a34309e0 2023-10-06T21:16:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1c28421b80 10 client.26326 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:29.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e7a31ad2b0 con 0x55e7a34309e0 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.855+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c08013dd0 con 0x55e7a34309e0 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1c000a9060 con 0x55e7a34309e0 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 20 client.26326 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a349fac0 0x55e7a34a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 4 client.26326 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.856+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7a337cb30 con 0x55e7a349fac0 2023-10-06T21:16:29.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.857+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a349fac0 0x55e7a34a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.857+0000 7f1c0e59c700 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a349fac0 0x55e7a34a1eb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618bad812c2ad697 server_cookie=2100d545f760dbbd in_seq=0 out_seq=0 2023-10-06T21:16:29.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1bf5ffb700 10 client.26326 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1bf5ffb700 1 -- 172.21.15.26:0/604911436 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7a337cb30 con 0x55e7a349fac0 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1bf5ffb700 10 client.26326 handle_command_reply: tid=0 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 client.26326 shutdown 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 msgr2=0x55e7a3440a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a3440a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c00004f20 tx=0x7f1c00096750 comp rx=0 tx=0).stop 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 shutdown_connections 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e7a349fac0 0x55e7a34a1eb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7a343b890 0x55e7a349e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7a34300c0 0x55e7a34404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1bf5ffb700 0 client.26326 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 --2- 172.21.15.26:0/604911436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e7a34309e0 0x55e7a3440a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.858+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 >> 172.21.15.26:0/604911436 conn(0x55e7a33a5260 msgr2=0x55e7a34338b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 shutdown_connections 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c28421b80 1 -- 172.21.15.26:0/604911436 wait complete. 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c28421b80 20 client.26326 trim_cache size 0 max 16384 2023-10-06T21:16:29.863 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:16:29.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 msgr2=0x7f1bf80b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 0x7f1bf80b4cc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c080076f0 tx=0x7f1c08007ab0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 msgr2=0x7f1c10107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:29.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.859+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e970 tx=0x7f1c0000ed30 comp rx=0 tx=0).stop 2023-10-06T21:16:29.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 shutdown_connections 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c1019efa0 0x7f1c101a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1bf80b2800 0x7f1bf80b4cc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c10104410 0x7f1c10107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 --2- 172.21.15.26:0/2531480823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c101078d0 0x7f1c10107d50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 >> 172.21.15.26:0/2531480823 conn(0x7f1c10076ed0 msgr2=0x7f1c10103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 shutdown_connections 2023-10-06T21:16:29.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:29.860+0000 7f1c14bff700 1 -- 172.21.15.26:0/2531480823 wait complete. 2023-10-06T21:16:30.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:30 smithi167 ceph-mon[134494]: pgmap v189: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.5 MiB/s wr, 670 op/s 2023-10-06T21:16:30.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:30 smithi026 ceph-mon[98490]: pgmap v189: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.5 MiB/s wr, 670 op/s 2023-10-06T21:16:30.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:30 smithi078 ceph-mon[134896]: pgmap v189: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.5 MiB/s wr, 670 op/s 2023-10-06T21:16:30.888 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:16:30.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:16:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 -- 172.21.15.26:0/235773591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e8102110 msgr2=0x7fd8e81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 --2- 172.21.15.26:0/235773591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e8102110 0x7fd8e81024f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b0a0 tx=0x7fd8e400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 -- 172.21.15.26:0/235773591 shutdown_connections 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 --2- 172.21.15.26:0/235773591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e810b4e0 0x7fd8e810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 --2- 172.21.15.26:0/235773591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e8102110 0x7fd8e81024f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 --2- 172.21.15.26:0/235773591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 0x7fd8e810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.215+0000 7fd8efdff700 1 -- 172.21.15.26:0/235773591 >> 172.21.15.26:0/235773591 conn(0x7fd8e80fb890 msgr2=0x7fd8e80fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.216+0000 7fd8efdff700 1 -- 172.21.15.26:0/235773591 shutdown_connections 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.216+0000 7fd8efdff700 1 -- 172.21.15.26:0/235773591 wait complete. 2023-10-06T21:16:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.217+0000 7fd8efdff700 1 Processor -- start 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.217+0000 7fd8efdff700 1 -- start start 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 0x7fd8e81002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 0x7fd8e81a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8100ad0 con 0x7fd8e8102a30 2023-10-06T21:16:31.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e81a5870 con 0x7fd8e810b4e0 2023-10-06T21:16:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8efdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e81a59b0 con 0x7fd8e8102110 2023-10-06T21:16:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 0x7fd8e81002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44940/0 (socket says 172.21.15.26:44940) 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1730533806 learned_addr learned my addr 172.21.15.26:0/1730533806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 0x7fd8e81a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1730533806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 msgr2=0x7fd8e81a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 0x7fd8e81a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 msgr2=0x7fd8e81002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 0x7fd8e81002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.218+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1730533806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4009d00 con 0x7fd8e8102110 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 0x7fd8e81a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:31.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4006070 tx=0x7fd8e400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8deffd700 1 -- 172.21.15.26:0/1730533806 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400e040 con 0x7fd8e8102110 2023-10-06T21:16:31.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8deffd700 1 -- 172.21.15.26:0/1730533806 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e4004e20 con 0x7fd8e8102110 2023-10-06T21:16:31.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8deffd700 1 -- 172.21.15.26:0/1730533806 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e401b450 con 0x7fd8e8102110 2023-10-06T21:16:31.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.219+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e81a5bb0 con 0x7fd8e8102110 2023-10-06T21:16:31.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.220+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e81a6040 con 0x7fd8e8102110 2023-10-06T21:16:31.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.223+0000 7fd8deffd700 1 -- 172.21.15.26:0/1730533806 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd8e4004120 con 0x7fd8e8102110 2023-10-06T21:16:31.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.224+0000 7fd9037f0b80 1 Processor -- start 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.224+0000 7fd9037f0b80 1 -- start start 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.225+0000 7fd8deffd700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 0x7fd8d40a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.225+0000 7fd8deffd700 1 -- 172.21.15.26:0/1730533806 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8e40ce4d0 con 0x7fd8e8102110 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3140 0x55cbf50b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf501c4e0 con 0x55cbf50b3140 2023-10-06T21:16:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf500dca0 con 0x55cbf50b3a60 2023-10-06T21:16:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf500b5c0 con 0x55cbf50c7120 2023-10-06T21:16:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ee39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51520/0 (socket says 172.21.15.26:51520) 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ed39a700 1 -- 172.21.15.26:0/1403853620 learned_addr learned my addr 172.21.15.26:0/1403853620 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ee39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44942/0 (socket says 172.21.15.26:44942) 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.226+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 0x7fd8d40a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 -- 172.21.15.26:0/1403853620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 msgr2=0x55cbf50c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 -- 172.21.15.26:0/1403853620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3140 msgr2=0x55cbf50b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3140 0x55cbf50b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 -- 172.21.15.26:0/1403853620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbf4e31e60 con 0x55cbf50b3a60 2023-10-06T21:16:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800bd30 tx=0x7fd8d80106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a2356bb439a782b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 0x7fd8d40a3c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8101e70 tx=0x7fd8e0009110 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8d3fff700 1 -- 172.21.15.26:0/1403853620 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8010e10 con 0x55cbf50b3a60 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8d3fff700 1 -- 172.21.15.26:0/1403853620 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8d8015070 con 0x55cbf50b3a60 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.227+0000 7fd8d3fff700 1 -- 172.21.15.26:0/1403853620 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8003e60 con 0x55cbf50b3a60 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1403853620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 msgr2=0x55cbf50c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800bd30 tx=0x7fd8d80106b0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1403853620 shutdown_connections 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3140 0x55cbf50b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1403853620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c6be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1403853620 >> 172.21.15.26:0/1403853620 conn(0x55cbf50282e0 msgr2=0x55cbf50af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1403853620 shutdown_connections 2023-10-06T21:16:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.228+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1403853620 wait complete. 2023-10-06T21:16:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 -- start start 2023-10-06T21:16:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 Processor -- start 2023-10-06T21:16:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3140 0x55cbf50c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50c4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf5006eb0 con 0x55cbf50c7120 2023-10-06T21:16:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.229+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf4fa3790 con 0x55cbf50b3a60 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf4fa3430 con 0x55cbf50b3140 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd9037f0b80 10 client.0 _ll_register_callbacks cb 0x55cbf4e00880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51526/0 (socket says 172.21.15.26:51526) 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 -- 172.21.15.26:0/2740810268 learned_addr learned my addr 172.21.15.26:0/2740810268 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 -- 172.21.15.26:0/2740810268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3140 msgr2=0x55cbf50c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3140 0x55cbf50c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 -- 172.21.15.26:0/2740810268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 msgr2=0x55cbf50c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 -- 172.21.15.26:0/2740810268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d800b960 con 0x55cbf50b3a60 2023-10-06T21:16:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8010a00 tx=0x7fd8d8010ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d278d14aaa1ed39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.230+0000 7fd8d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8005450 con 0x55cbf50b3a60 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbf50a6f90 con 0x55cbf50b3a60 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8d8017070 con 0x55cbf50b3a60 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8d80055b0 con 0x55cbf50b3a60 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd9037f0b80 10 client.26378 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cbf4e1ace0 con 0x55cbf50b3a60 2023-10-06T21:16:31.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.231+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8d8031020 con 0x55cbf50b3a60 2023-10-06T21:16:31.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd8d8011740 con 0x55cbf50b3a60 2023-10-06T21:16:31.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 10 client.26378 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:31.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 20 client.26378 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf50e7c80 0x55cbf50ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 4 client.26378 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbf4fffe90 con 0x55cbf50e7c80 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.234+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf50e7c80 0x55cbf50ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.235+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf50e7c80 0x55cbf50ea0a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7dfeaf21bf67957 server_cookie=38f5763b92755d6c in_seq=0 out_seq=0 2023-10-06T21:16:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.235+0000 7fd8d17fa700 10 client.26378 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.236+0000 7fd8d17fa700 1 -- 172.21.15.26:0/2740810268 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbf4fffe90 con 0x55cbf50e7c80 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.236+0000 7fd8d17fa700 10 client.26378 handle_command_reply: tid=0 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.236+0000 7fd9037f0b80 1 client.26378 shutdown 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 msgr2=0x55cbf50c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8010a00 tx=0x7fd8d8010ef0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 shutdown_connections 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3140 0x55cbf50c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf50e7c80 0x55cbf50ea0a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2740810268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3a60 0x55cbf50c43e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd8d17fa700 0 client.26378 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 >> 172.21.15.26:0/2740810268 conn(0x55cbf50282e0 msgr2=0x55cbf50af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 shutdown_connections 2023-10-06T21:16:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2740810268 wait complete. 2023-10-06T21:16:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.237+0000 7fd9037f0b80 20 client.26378 trim_cache size 0 max 16384 2023-10-06T21:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.262+0000 7fd9037f0b80 1 Processor -- start 2023-10-06T21:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 -- start start 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3a60 0x55cbf50b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf4e879a0 con 0x55cbf50b3a60 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf4fffe90 con 0x55cbf50b3140 2023-10-06T21:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf50bfd30 con 0x55cbf50c7120 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd8ee39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3a60 0x55cbf50b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51532/0 (socket says 172.21.15.26:51532) 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.263+0000 7fd8edb9b700 1 -- 172.21.15.26:0/2777803461 learned_addr learned my addr 172.21.15.26:0/2777803461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 -- 172.21.15.26:0/2777803461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 msgr2=0x55cbf50c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 -- 172.21.15.26:0/2777803461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3a60 msgr2=0x55cbf50b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3a60 0x55cbf50b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 -- 172.21.15.26:0/2777803461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbf50eec90 con 0x55cbf50c7120 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.264+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50b5c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fd0 tx=0x7fd8e000b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9e75dafa1de6dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.265+0000 7fd8dcbff700 1 -- 172.21.15.26:0/2777803461 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0022580 con 0x55cbf50c7120 2023-10-06T21:16:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.265+0000 7fd8dcbff700 1 -- 172.21.15.26:0/2777803461 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e001e050 con 0x55cbf50c7120 2023-10-06T21:16:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.265+0000 7fd8dcbff700 1 -- 172.21.15.26:0/2777803461 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0023770 con 0x55cbf50c7120 2023-10-06T21:16:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.266+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2777803461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 msgr2=0x55cbf50b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.266+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50b5c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fd0 tx=0x7fd8e000b000 comp rx=0 tx=0).stop 2023-10-06T21:16:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.267+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2777803461 shutdown_connections 2023-10-06T21:16:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.267+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50c7120 0x55cbf50b5c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.267+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.267+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/2777803461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50b3a60 0x55cbf50b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.267+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2777803461 >> 172.21.15.26:0/2777803461 conn(0x55cbf50282e0 msgr2=0x55cbf50b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.268+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2777803461 shutdown_connections 2023-10-06T21:16:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.268+0000 7fd9037f0b80 1 -- 172.21.15.26:0/2777803461 wait complete. 2023-10-06T21:16:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.269+0000 7fd9037f0b80 1 -- start start 2023-10-06T21:16:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.270+0000 7fd9037f0b80 1 Processor -- start 2023-10-06T21:16:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf513ba80 con 0x55cbf50c7120 2023-10-06T21:16:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf513bbc0 con 0x55cbf50b3140 2023-10-06T21:16:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbf513bd80 con 0x55cbf50b3a60 2023-10-06T21:16:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd9037f0b80 10 client.0 _ll_register_callbacks cb 0x7fd8e0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd8edb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51546/0 (socket says 172.21.15.26:51546) 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1882414273 learned_addr learned my addr 172.21.15.26:0/1882414273 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.271+0000 7fd8ed39a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44964/0 (socket says 172.21.15.26:44964) 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8ee39c700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1882414273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 msgr2=0x55cbf50cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1882414273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 msgr2=0x55cbf50cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 -- 172.21.15.26:0/1882414273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0018480 con 0x55cbf50b3140 2023-10-06T21:16:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8edb9b700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40ce440 tx=0x7fd8e40044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd74a203f2a1f26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.272+0000 7fd8d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.277+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e409b4f0 con 0x55cbf50b3140 2023-10-06T21:16:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.277+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd8e409f640 con 0x55cbf50b3140 2023-10-06T21:16:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.278+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8d8015070 con 0x55cbf50b3140 2023-10-06T21:16:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.278+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e40a37f0 con 0x55cbf50b3140 2023-10-06T21:16:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.278+0000 7fd9037f0b80 10 client.26384 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.278+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55cbf4e582d0 con 0x55cbf50b3140 2023-10-06T21:16:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.278+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8d8015070 con 0x55cbf50b3140 2023-10-06T21:16:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd8e40b1070 con 0x55cbf50b3140 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 10 client.26384 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 20 client.26384 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf513d320 0x55cbf513f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 4 client.26384 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cbf4fffbb0 con 0x55cbf513d320 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.279+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf513d320 0x55cbf513f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.280+0000 7fd8d17fa700 10 client.26384 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.280+0000 7fd8ed39a700 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf513d320 0x55cbf513f740 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5db44b72590c9c server_cookie=93cb33cab4403cd3 in_seq=0 out_seq=0 2023-10-06T21:16:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd8d17fa700 1 -- 172.21.15.26:0/1882414273 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbf4fffbb0 con 0x55cbf513d320 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd8d17fa700 10 client.26384 handle_command_reply: tid=0 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 client.26384 shutdown 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 msgr2=0x55cbf50cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40ce440 tx=0x7fd8e40044c0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 shutdown_connections 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbf50c7120 0x55cbf50cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cbf513d320 0x55cbf513f740 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbf50b3140 0x55cbf50cc380 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 --2- 172.21.15.26:0/1882414273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbf50b3a60 0x55cbf50cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 >> 172.21.15.26:0/1882414273 conn(0x55cbf50282e0 msgr2=0x55cbf50b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd8d17fa700 0 client.26384 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 shutdown_connections 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.281+0000 7fd9037f0b80 1 -- 172.21.15.26:0/1882414273 wait complete. 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd9037f0b80 20 client.26384 trim_cache size 0 max 16384 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:16:31.286 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:16:31.287 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:16:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 msgr2=0x7fd8d40a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 0x7fd8d40a3c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8101e70 tx=0x7fd8e0009110 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 msgr2=0x7fd8e80ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4006070 tx=0x7fd8e400cef0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 shutdown_connections 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8e810b4e0 0x7fd8e81a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8e8102110 0x7fd8e80ffd60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd8d40a1740 0x7fd8d40a3c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 --2- 172.21.15.26:0/1730533806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e8102a30 0x7fd8e81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 >> 172.21.15.26:0/1730533806 conn(0x7fd8e80fb890 msgr2=0x7fd8e8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 shutdown_connections 2023-10-06T21:16:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.282+0000 7fd8efdff700 1 -- 172.21.15.26:0/1730533806 wait complete. 2023-10-06T21:16:31.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1642498795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc64107f90 msgr2=0x7efc64114330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1642498795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc64107f90 0x7efc64114330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1642498795 shutdown_connections 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1642498795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc64107f90 0x7efc64114330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1642498795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6410d570 0x7efc6410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1642498795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc641075d0 0x7efc64107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1642498795 >> 172.21.15.26:0/1642498795 conn(0x7efc64076d50 msgr2=0x7efc64105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1642498795 shutdown_connections 2023-10-06T21:16:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.664+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1642498795 wait complete. 2023-10-06T21:16:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.665+0000 7efc6b3ff700 1 Processor -- start 2023-10-06T21:16:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.665+0000 7efc6b3ff700 1 -- start start 2023-10-06T21:16:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc641075d0 0x7efc6419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6410d570 0x7efc641a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a2420 con 0x7efc641075d0 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a2590 con 0x7efc64107f90 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6b3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641a2750 con 0x7efc6410d570 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6899a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6899a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51564/0 (socket says 172.21.15.26:51564) 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6899a700 1 -- 172.21.15.26:0/1988656078 learned_addr learned my addr 172.21.15.26:0/1988656078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.666+0000 7efc6999c700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6410d570 0x7efc641a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6919b700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc641075d0 0x7efc6419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 -- 172.21.15.26:0/1988656078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6410d570 msgr2=0x7efc641a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6410d570 0x7efc641a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 -- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc641075d0 msgr2=0x7efc6419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc641075d0 0x7efc6419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 -- 172.21.15.26:0/1988656078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0095f0 con 0x7efc64107f90 2023-10-06T21:16:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6899a700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc58007e20 tx=0x7efc5800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc627fc700 1 -- 172.21.15.26:0/1988656078 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5800ad40 con 0x7efc64107f90 2023-10-06T21:16:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc627fc700 1 -- 172.21.15.26:0/1988656078 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc580128e0 con 0x7efc64107f90 2023-10-06T21:16:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc641a2a30 con 0x7efc64107f90 2023-10-06T21:16:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.667+0000 7efc627fc700 1 -- 172.21.15.26:0/1988656078 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580117a0 con 0x7efc64107f90 2023-10-06T21:16:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.668+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc641aa270 con 0x7efc64107f90 2023-10-06T21:16:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.675+0000 7efc7ec94b80 1 Processor -- start 2023-10-06T21:16:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.676+0000 7efc7ec94b80 1 -- start start 2023-10-06T21:16:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c610f890 0x5636c6111c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c606d460 con 0x5636c61040c0 2023-10-06T21:16:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c605ec20 con 0x5636c61049e0 2023-10-06T21:16:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c605c540 con 0x5636c610f890 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc6919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc6919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51578/0 (socket says 172.21.15.26:51578) 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc6919b700 1 -- 172.21.15.26:0/3483304842 learned_addr learned my addr 172.21.15.26:0/3483304842 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.677+0000 7efc627fc700 1 -- 172.21.15.26:0/1988656078 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efc58012a50 con 0x7efc64107f90 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 -- 172.21.15.26:0/3483304842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c610f890 msgr2=0x5636c6111c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c610f890 0x5636c6111c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 -- 172.21.15.26:0/3483304842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 msgr2=0x5636c61044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 -- 172.21.15.26:0/3483304842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c5e82de0 con 0x5636c61049e0 2023-10-06T21:16:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc6919b700 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc5400bd70 tx=0x7efc5400f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a895dc9b6b381191 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.678+0000 7efc4b7fe700 1 -- 172.21.15.26:0/3483304842 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400fe50 con 0x5636c61049e0 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.679+0000 7efc4b7fe700 1 -- 172.21.15.26:0/3483304842 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5401a920 con 0x5636c61049e0 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.679+0000 7efc4b7fe700 1 -- 172.21.15.26:0/3483304842 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540117a0 con 0x5636c61049e0 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.679+0000 7efc627fc700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 0x7efc4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.679+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3483304842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 msgr2=0x5636c610f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.679+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc5400bd70 tx=0x7efc5400f640 comp rx=0 tx=0).stop 2023-10-06T21:16:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc627fc700 1 -- 172.21.15.26:0/1988656078 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efc580cc910 con 0x7efc64107f90 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3483304842 shutdown_connections 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c610f890 0x5636c6111c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3483304842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c61049e0 0x5636c610f350 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3483304842 >> 172.21.15.26:0/3483304842 conn(0x5636c6079260 msgr2=0x5636c6100450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc6999c700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 0x7efc4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3483304842 shutdown_connections 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.680+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3483304842 wait complete. 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.681+0000 7efc7ec94b80 1 -- start start 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.681+0000 7efc6999c700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 0x7efc4c0a3b30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c00bee0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.681+0000 7efc7ec94b80 1 Processor -- start 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c610c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c610cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c6057e30 con 0x5636c61040c0 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c5ff4710 con 0x5636c610f890 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c5ff43b0 con 0x5636c61049e0 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc7ec94b80 10 client.0 _ll_register_callbacks cb 0x5636c5e51800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc6999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc6999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51590/0 (socket says 172.21.15.26:51590) 2023-10-06T21:16:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc6999c700 1 -- 172.21.15.26:0/4148334250 learned_addr learned my addr 172.21.15.26:0/4148334250 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.682+0000 7efc6919b700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c610c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6899a700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c610cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 -- 172.21.15.26:0/4148334250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 msgr2=0x5636c610cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c610cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 -- 172.21.15.26:0/4148334250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 msgr2=0x5636c610c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c610c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 -- 172.21.15.26:0/4148334250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5400b9a0 con 0x5636c610f890 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6919b700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c610c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.683+0000 7efc6999c700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbf0e1b6975ea67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c023660 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636c60f7f10 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efc5c0237c0 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5c003680 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc7ec94b80 10 client.26393 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636c5e6bc60 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.684+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5c03b020 con 0x5636c610f890 2023-10-06T21:16:31.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc5c0109d0 con 0x5636c610f890 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 20 client.26393 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c61303b0 0x5636c61327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 4 client.26393 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.685+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636c6050e10 con 0x5636c61303b0 2023-10-06T21:16:31.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.689+0000 7efc6919b700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c61303b0 0x5636c61327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.690+0000 7efc6919b700 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c61303b0 0x5636c61327d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59aff058f50f3af server_cookie=cb2cb2821a478079 in_seq=0 out_seq=0 2023-10-06T21:16:31.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.690+0000 7efc48ff9700 10 client.26393 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc48ff9700 1 -- 172.21.15.26:0/4148334250 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5636c6050e10 con 0x5636c61303b0 2023-10-06T21:16:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc48ff9700 10 client.26393 handle_command_reply: tid=0 2023-10-06T21:16:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 client.26393 shutdown 2023-10-06T21:16:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 msgr2=0x5636c610d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c005ab0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 shutdown_connections 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c61303b0 0x5636c61327d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c610d110 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c610c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/4148334250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c610cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 >> 172.21.15.26:0/4148334250 conn(0x5636c6079260 msgr2=0x5636c60fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.691+0000 7efc48ff9700 0 client.26393 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.692+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 shutdown_connections 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.692+0000 7efc7ec94b80 1 -- 172.21.15.26:0/4148334250 wait complete. 2023-10-06T21:16:31.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.692+0000 7efc7ec94b80 20 client.26393 trim_cache size 0 max 16384 2023-10-06T21:16:31.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.716+0000 7efc7ec94b80 1 Processor -- start 2023-10-06T21:16:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 -- start start 2023-10-06T21:16:31.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61040c0 0x5636c61163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61049e0 0x5636c610cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c5ed8920 con 0x5636c61049e0 2023-10-06T21:16:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c6101280 con 0x5636c610f890 2023-10-06T21:16:31.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.719+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c6116910 con 0x5636c61040c0 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51594/0 (socket says 172.21.15.26:51594) 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 -- 172.21.15.26:0/3219290467 learned_addr learned my addr 172.21.15.26:0/3219290467 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6919b700 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61040c0 0x5636c61163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 -- 172.21.15.26:0/3219290467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61040c0 msgr2=0x5636c61163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61040c0 0x5636c61163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 -- 172.21.15.26:0/3219290467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61049e0 msgr2=0x5636c610cf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:16:31.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61049e0 0x5636c610cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 -- 172.21.15.26:0/3219290467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c60fc140 con 0x5636c610f890 2023-10-06T21:16:31.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.720+0000 7efc6999c700 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc5c005b40 tx=0x7efc5c00be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52d77c2036f9ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.721+0000 7efc4bfff700 1 -- 172.21.15.26:0/3219290467 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c010670 con 0x5636c610f890 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.721+0000 7efc4bfff700 1 -- 172.21.15.26:0/3219290467 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5c03b030 con 0x5636c610f890 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.721+0000 7efc4bfff700 1 -- 172.21.15.26:0/3219290467 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c023a50 con 0x5636c610f890 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3219290467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 msgr2=0x5636c6109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc5c005b40 tx=0x7efc5c00be00 comp rx=0 tx=0).stop 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3219290467 shutdown_connections 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6109920 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61040c0 0x5636c61163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/3219290467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61049e0 0x5636c610cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3219290467 >> 172.21.15.26:0/3219290467 conn(0x5636c6079260 msgr2=0x5636c61079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3219290467 shutdown_connections 2023-10-06T21:16:31.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.722+0000 7efc7ec94b80 1 -- 172.21.15.26:0/3219290467 wait complete. 2023-10-06T21:16:31.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.723+0000 7efc7ec94b80 1 -- start start 2023-10-06T21:16:31.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.723+0000 7efc7ec94b80 1 Processor -- start 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c6114900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6114e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c61832d0 con 0x5636c61040c0 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c6183440 con 0x5636c610f890 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c6194610 con 0x5636c61049e0 2023-10-06T21:16:31.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc7ec94b80 10 client.0 _ll_register_callbacks cb 0x7efc5c00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.724+0000 7efc6919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45960/0 (socket says 172.21.15.26:45960) 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 -- 172.21.15.26:0/2674208653 learned_addr learned my addr 172.21.15.26:0/2674208653 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6999c700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6899a700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c6114900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 -- 172.21.15.26:0/2674208653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 msgr2=0x5636c6114900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c6114900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 -- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 msgr2=0x5636c6114e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6114e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 -- 172.21.15.26:0/2674208653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636c5e6bc60 con 0x5636c61040c0 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6999c700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6899a700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c6114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc6919b700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7523b64124cf33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.725+0000 7efc48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5401a660 con 0x5636c61040c0 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7efc540289c0 con 0x5636c61040c0 2023-10-06T21:16:31.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc5401a920 con 0x5636c61040c0 2023-10-06T21:16:31.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5401a7c0 con 0x5636c61040c0 2023-10-06T21:16:31.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc7ec94b80 10 client.16155 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:31.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636c5ea9250 con 0x5636c61040c0 2023-10-06T21:16:31.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.726+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc5401a920 con 0x5636c61040c0 2023-10-06T21:16:31.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc54028050 con 0x5636c61040c0 2023-10-06T21:16:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 20 client.16155 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c6195d40 0x5636c6198160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 4 client.16155 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.727+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636c6050b30 con 0x5636c6195d40 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc6899a700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c6195d40 0x5636c6198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc6899a700 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c6195d40 0x5636c6198160 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965e031312c20246 server_cookie=d8dc1a92513c14c8 in_seq=0 out_seq=0 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc48ff9700 10 client.16155 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc48ff9700 1 -- 172.21.15.26:0/2674208653 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5636c6050b30 con 0x5636c6195d40 2023-10-06T21:16:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc48ff9700 10 client.16155 handle_command_reply: tid=0 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.728+0000 7efc7ec94b80 1 client.16155 shutdown 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 msgr2=0x5636c61143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc54000c00 tx=0x7efc5400fb80 comp rx=0 tx=0).stop 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 shutdown_connections 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5636c6195d40 0x5636c6198160 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636c610f890 0x5636c6114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc48ff9700 0 client.16155 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636c61040c0 0x5636c61143c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 --2- 172.21.15.26:0/2674208653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c61049e0 0x5636c6114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 >> 172.21.15.26:0/2674208653 conn(0x5636c6079260 msgr2=0x5636c610ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.729+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 shutdown_connections 2023-10-06T21:16:31.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.733+0000 7efc7ec94b80 1 -- 172.21.15.26:0/2674208653 wait complete. 2023-10-06T21:16:31.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.733+0000 7efc7ec94b80 20 client.16155 trim_cache size 0 max 16384 2023-10-06T21:16:31.736 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:16:31.736 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:16:31.737 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:16:31.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.734+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 msgr2=0x7efc4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.734+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 0x7efc4c0a3b30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006010 tx=0x7efc5c00bee0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.734+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 msgr2=0x7efc6419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.734+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc58007e20 tx=0x7efc5800a340 comp rx=0 tx=0).stop 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 shutdown_connections 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6410d570 0x7efc641a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efc4c0a1670 0x7efc4c0a3b30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc641075d0 0x7efc6419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 --2- 172.21.15.26:0/1988656078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc64107f90 0x7efc6419d980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 >> 172.21.15.26:0/1988656078 conn(0x7efc64076d50 msgr2=0x7efc64077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 shutdown_connections 2023-10-06T21:16:31.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:16:31.735+0000 7efc6b3ff700 1 -- 172.21.15.26:0/1988656078 wait complete. 2023-10-06T21:16:31.764 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:16:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 350.0 s, 1497.6 tps, lat 66.980 ms stddev 66.110 2023-10-06T21:16:32.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: pgmap v190: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:16:32.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:32 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: pgmap v190: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-10-06T21:16:32.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:16:32.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:32 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: pgmap v190: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:32.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:32.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:32.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:16:32.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:32 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:33.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:33 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:33.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:33 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:33.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:33 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:16:34.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:34 smithi026 ceph-mon[98490]: pgmap v191: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-06T21:16:34.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:34 smithi078 ceph-mon[134896]: pgmap v191: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-06T21:16:34.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:34 smithi167 ceph-mon[134494]: pgmap v191: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 766 op/s 2023-10-06T21:16:36.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:36 smithi026 ceph-mon[98490]: pgmap v192: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 986 op/s 2023-10-06T21:16:36.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:36 smithi078 ceph-mon[134896]: pgmap v192: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 986 op/s 2023-10-06T21:16:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:36 smithi167 ceph-mon[134494]: pgmap v192: 97 pgs: 97 active+clean; 6.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 986 op/s 2023-10-06T21:16:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 -- 172.21.15.26:0/1193767852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb74101ab0 msgr2=0x7fbb74105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 --2- 172.21.15.26:0/1193767852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb74101ab0 0x7fbb74105af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009990 tx=0x7fbb68009ca0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 -- 172.21.15.26:0/1193767852 shutdown_connections 2023-10-06T21:16:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 --2- 172.21.15.26:0/1193767852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74106220 0x7fbb7410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 --2- 172.21.15.26:0/1193767852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb74101100 0x7fbb741014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 --2- 172.21.15.26:0/1193767852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb74101ab0 0x7fbb74105af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.798+0000 7fbb797ff700 1 -- 172.21.15.26:0/1193767852 >> 172.21.15.26:0/1193767852 conn(0x7fbb740fc9a0 msgr2=0x7fbb740fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.800+0000 7fbb797ff700 1 -- 172.21.15.26:0/1193767852 shutdown_connections 2023-10-06T21:16:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.800+0000 7fbb797ff700 1 -- 172.21.15.26:0/1193767852 wait complete. 2023-10-06T21:16:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.801+0000 7fbb797ff700 1 Processor -- start 2023-10-06T21:16:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 -- start start 2023-10-06T21:16:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74101100 0x7fbb74072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb74106220 0x7fbb740729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7406bec0 con 0x7fbb74106220 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7406c030 con 0x7fbb74101100 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.802+0000 7fbb797ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7406ffd0 con 0x7fbb7406b240 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.803+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.803+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45056/0 (socket says 172.21.15.26:45056) 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.803+0000 7fbb737fe700 1 -- 172.21.15.26:0/3781846871 learned_addr learned my addr 172.21.15.26:0/3781846871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.803+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74101100 0x7fbb74072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 -- 172.21.15.26:0/3781846871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74101100 msgr2=0x7fbb74072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74101100 0x7fbb74072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 -- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb74106220 msgr2=0x7fbb740729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb74106220 0x7fbb740729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 -- 172.21.15.26:0/3781846871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680095f0 con 0x7fbb7406b240 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb737fe700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00ef30 tx=0x7fbb6c00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.804+0000 7fbb5bfff700 1 -- 172.21.15.26:0/3781846871 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c018070 con 0x7fbb7406b240 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.805+0000 7fbb5bfff700 1 -- 172.21.15.26:0/3781846871 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb6c00f040 con 0x7fbb7406b240 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.805+0000 7fbb5bfff700 1 -- 172.21.15.26:0/3781846871 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0078c0 con 0x7fbb7406b240 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.805+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb740701f0 con 0x7fbb7406b240 2023-10-06T21:16:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.805+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740706b0 con 0x7fbb7406b240 2023-10-06T21:16:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.810+0000 7fbb8d17db80 1 Processor -- start 2023-10-06T21:16:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.810+0000 7fbb5bfff700 1 -- 172.21.15.26:0/3781846871 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbb6c00caa0 con 0x7fbb7406b240 2023-10-06T21:16:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.810+0000 7fbb8d17db80 1 -- start start 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc4c00 0x560533fc4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533f2e400 con 0x560533fc4c00 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533f1fbc0 con 0x560533fd03d0 2023-10-06T21:16:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533f1d4e0 con 0x560533fc5520 2023-10-06T21:16:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45072/0 (socket says 172.21.15.26:45072) 2023-10-06T21:16:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb72ffd700 1 -- 172.21.15.26:0/3151803901 learned_addr learned my addr 172.21.15.26:0/3151803901 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.811+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc4c00 0x560533fc4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 -- 172.21.15.26:0/3151803901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 msgr2=0x560533fd27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 -- 172.21.15.26:0/3151803901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc4c00 msgr2=0x560533fc4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc4c00 0x560533fc4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 -- 172.21.15.26:0/3151803901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560533d43d80 con 0x560533fc5520 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb5bfff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 0x7fbb5c0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009d10 tx=0x7fbb640116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f46dd09add28a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.812+0000 7fbb5bfff700 1 -- 172.21.15.26:0/3781846871 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb6c0cce30 con 0x7fbb7406b240 2023-10-06T21:16:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.813+0000 7fbb597fa700 1 -- 172.21.15.26:0/3151803901 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640097e0 con 0x560533fc5520 2023-10-06T21:16:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.813+0000 7fbb597fa700 1 -- 172.21.15.26:0/3151803901 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb64011ed0 con 0x560533fc5520 2023-10-06T21:16:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.813+0000 7fbb597fa700 1 -- 172.21.15.26:0/3151803901 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400f8e0 con 0x560533fc5520 2023-10-06T21:16:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.814+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3151803901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 msgr2=0x560533fcfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.814+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009d10 tx=0x7fbb640116c0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3151803901 shutdown_connections 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc4c00 0x560533fc4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 0x7fbb5c0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3151803901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc5520 0x560533fcfe90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3151803901 >> 172.21.15.26:0/3151803901 conn(0x560533f3a200 msgr2=0x560533fc1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.815+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3151803901 shutdown_connections 2023-10-06T21:16:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.816+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3151803901 wait complete. 2023-10-06T21:16:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.816+0000 7fbb8d17db80 1 -- start start 2023-10-06T21:16:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.817+0000 7fbb8d17db80 1 Processor -- start 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fcd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fcd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533f18dd0 con 0x560533fc5520 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533eb56b0 con 0x560533fd03d0 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533eb5350 con 0x560533fc4c00 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb8d17db80 10 client.0 _ll_register_callbacks cb 0x560533d127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45084/0 (socket says 172.21.15.26:45084) 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb72ffd700 1 -- 172.21.15.26:0/45059480 learned_addr learned my addr 172.21.15.26:0/45059480 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.818+0000 7fbb737fe700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fcd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb727fc700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fcd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb72ffd700 1 -- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 msgr2=0x560533fcd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fcd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb72ffd700 1 -- 172.21.15.26:0/45059480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 msgr2=0x560533fcd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fcd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 0x7fbb5c0a3b10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006010 tx=0x7fbb68009510 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb72ffd700 1 -- 172.21.15.26:0/45059480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb64009940 con 0x560533fc4c00 2023-10-06T21:16:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.819+0000 7fbb737fe700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fcd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:16:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.820+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbb640059d0 tx=0x7fbb64012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478640fae01047df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.820+0000 7fbb56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.820+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64005ce0 con 0x560533fc4c00 2023-10-06T21:16:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.820+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb64004540 con 0x560533fc4c00 2023-10-06T21:16:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.821+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560533fb8d50 con 0x560533fc4c00 2023-10-06T21:16:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.822+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb640097e0 con 0x560533fc4c00 2023-10-06T21:16:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.822+0000 7fbb8d17db80 10 client.26362 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.822+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560533d2cc00 con 0x560533fc4c00 2023-10-06T21:16:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560533fb8d50 con 0x560533fc4c00 2023-10-06T21:16:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbb6400f4e0 con 0x560533fc4c00 2023-10-06T21:16:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 10 client.26362 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 10 client.26362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 20 client.26362 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560533ff0e00 0x560533ff3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 4 client.26362 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560533f11db0 con 0x560533ff0e00 2023-10-06T21:16:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.823+0000 7fbb727fc700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560533ff0e00 0x560533ff3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.824+0000 7fbb56ff5700 10 client.26362 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.824+0000 7fbb727fc700 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560533ff0e00 0x560533ff3220 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7879b5171b06381 server_cookie=6a9917e0a77a7dfb in_seq=0 out_seq=0 2023-10-06T21:16:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.825+0000 7fbb56ff5700 1 -- 172.21.15.26:0/45059480 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560533f11db0 con 0x560533ff0e00 2023-10-06T21:16:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.825+0000 7fbb56ff5700 10 client.26362 handle_command_reply: tid=0 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.825+0000 7fbb8d17db80 1 client.26362 shutdown 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 msgr2=0x560533fcfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbb640059d0 tx=0x7fbb64012680 comp rx=0 tx=0).stop 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 shutdown_connections 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560533ff0e00 0x560533ff3220 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fcd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fcfe50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/45059480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fcd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 >> 172.21.15.26:0/45059480 conn(0x560533f3a200 msgr2=0x560533fc0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.827+0000 7fbb56ff5700 0 client.26362 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.829+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 shutdown_connections 2023-10-06T21:16:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.829+0000 7fbb8d17db80 1 -- 172.21.15.26:0/45059480 wait complete. 2023-10-06T21:16:36.832 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.829+0000 7fbb8d17db80 20 client.26362 trim_cache size 0 max 16384 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.852+0000 7fbb8d17db80 1 Processor -- start 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.852+0000 7fbb8d17db80 1 -- start start 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533ff31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533d998c0 con 0x560533fc5520 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533fc5e20 con 0x560533fd03d0 2023-10-06T21:16:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.853+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533fbd890 con 0x560533fc4c00 2023-10-06T21:16:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533ff31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533ff31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45094/0 (socket says 172.21.15.26:45094) 2023-10-06T21:16:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb72ffd700 1 -- 172.21.15.26:0/2179680248 learned_addr learned my addr 172.21.15.26:0/2179680248 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb727fc700 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 -- 172.21.15.26:0/2179680248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 msgr2=0x560533ff31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533ff31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 -- 172.21.15.26:0/2179680248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 msgr2=0x560533fc7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fc7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.854+0000 7fbb737fe700 1 -- 172.21.15.26:0/2179680248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560533d6a1f0 con 0x560533fd03d0 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.855+0000 7fbb737fe700 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd6f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0ca560 tx=0x7fbb6c099d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a8f4989600df153 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.855+0000 7fbb59ffb700 1 -- 172.21.15.26:0/2179680248 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c09a520 con 0x560533fd03d0 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.855+0000 7fbb59ffb700 1 -- 172.21.15.26:0/2179680248 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb6c099e20 con 0x560533fd03d0 2023-10-06T21:16:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.855+0000 7fbb59ffb700 1 -- 172.21.15.26:0/2179680248 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c09a080 con 0x560533fd03d0 2023-10-06T21:16:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 -- 172.21.15.26:0/2179680248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 msgr2=0x560533fd6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd6f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0ca560 tx=0x7fbb6c099d10 comp rx=0 tx=0).stop 2023-10-06T21:16:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 -- 172.21.15.26:0/2179680248 shutdown_connections 2023-10-06T21:16:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fd03d0 0x560533fd6f50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533ff31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/2179680248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fc5520 0x560533fc7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 -- 172.21.15.26:0/2179680248 >> 172.21.15.26:0/2179680248 conn(0x560533f3a200 msgr2=0x560533fed1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 -- 172.21.15.26:0/2179680248 shutdown_connections 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.856+0000 7fbb8d17db80 1 -- 172.21.15.26:0/2179680248 wait complete. 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.857+0000 7fbb8d17db80 1 -- start start 2023-10-06T21:16:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.857+0000 7fbb8d17db80 1 Processor -- start 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fd4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 0x560533fd50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 0x56053403dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053403e100 con 0x560533fd03d0 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053403e270 con 0x560533fc5520 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053403e430 con 0x560533fc4c00 2023-10-06T21:16:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.858+0000 7fbb8d17db80 10 client.0 _ll_register_callbacks cb 0x7fbb64004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.859+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 0x56053403dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.859+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 0x56053403dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46012/0 (socket says 172.21.15.26:46012) 2023-10-06T21:16:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.859+0000 7fbb737fe700 1 -- 172.21.15.26:0/3732563020 learned_addr learned my addr 172.21.15.26:0/3732563020 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.859+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 0x560533fd50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.859+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fd4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 -- 172.21.15.26:0/3732563020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 msgr2=0x560533fd4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fd4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 -- 172.21.15.26:0/3732563020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 msgr2=0x56053403dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 0x56053403dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 -- 172.21.15.26:0/3732563020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740706b0 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.860+0000 7fbb727fc700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 0x560533fd50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b2e0 tx=0x7fbb68000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8615239e9358bc65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800b9f0 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fbb6800bb50 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb64011ed0 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb6800bd10 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb8d17db80 10 client.26414 fetch_fsmap learned FSMap version 40 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.861+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560533d42250 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb64011ed0 con 0x560533fc5520 2023-10-06T21:16:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 40) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbb6800f460 con 0x560533fc5520 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 10 client.26414 fetch_fsmap finished waiting for FSMap version 40 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 20 client.26414 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560534045d00 0x560534048120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 4 client.26414 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560533f11ad0 con 0x560534045d00 2023-10-06T21:16:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.862+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560534045d00 0x560534048120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.863+0000 7fbb72ffd700 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560534045d00 0x560534048120 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5154f484fbb01778 server_cookie=e90f225e1cf9ca5e in_seq=0 out_seq=0 2023-10-06T21:16:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.863+0000 7fbb56ff5700 10 client.26414 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.951+0000 7fbb56ff5700 1 -- 172.21.15.26:0/3732563020 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560533f11ad0 con 0x560534045d00 2023-10-06T21:16:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.951+0000 7fbb56ff5700 10 client.26414 handle_command_reply: tid=0 2023-10-06T21:16:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.951+0000 7fbb8d17db80 1 client.26414 shutdown 2023-10-06T21:16:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.951+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 msgr2=0x560533fd50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.951+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 0x560533fd50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b2e0 tx=0x7fbb68000bc0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 shutdown_connections 2023-10-06T21:16:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560533fd03d0 0x56053403dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560533fc4c00 0x560533fd4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560534045d00 0x560534048120 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 --2- 172.21.15.26:0/3732563020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560533fc5520 0x560533fd50f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 >> 172.21.15.26:0/3732563020 conn(0x560533f3a200 msgr2=0x560533fc8430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb56ff5700 0 client.26414 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 shutdown_connections 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 1 -- 172.21.15.26:0/3732563020 wait complete. 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.952+0000 7fbb8d17db80 20 client.26414 trim_cache size 0 max 16384 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:16:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 msgr2=0x7fbb5c0a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 0x7fbb5c0a3b10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006010 tx=0x7fbb68009510 comp rx=0 tx=0).stop 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 msgr2=0x7fbb7406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00ef30 tx=0x7fbb6c00c450 comp rx=0 tx=0).stop 2023-10-06T21:16:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 shutdown_connections 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb7406b240 0x7fbb7406b6f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb74101100 0x7fbb74072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbb5c0a1650 0x7fbb5c0a3b10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 --2- 172.21.15.26:0/3781846871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb74106220 0x7fbb740729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 >> 172.21.15.26:0/3781846871 conn(0x7fbb740fc9a0 msgr2=0x7fbb740fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 shutdown_connections 2023-10-06T21:16:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:36.953+0000 7fbb797ff700 1 -- 172.21.15.26:0/3781846871 wait complete. 2023-10-06T21:16:37.132 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 355.0 s, 1755.3 tps, lat 56.881 ms stddev 65.971 2023-10-06T21:16:37.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:37 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:37.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:37 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:37.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:37 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:16:38.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:38 smithi078 ceph-mon[134896]: pgmap v193: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 944 op/s 2023-10-06T21:16:38.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:38 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:38.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:38 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:38.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:38 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:16:38.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:38 smithi167 ceph-mon[134494]: pgmap v193: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 944 op/s 2023-10-06T21:16:38.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:38 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:38.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:38 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:38.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:38 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:16:38.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:38 smithi026 ceph-mon[98490]: pgmap v193: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 944 op/s 2023-10-06T21:16:38.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:38 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:16:38.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:38 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:16:38.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:38 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:16:40.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:40 smithi167 ceph-mon[134494]: pgmap v194: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 734 op/s 2023-10-06T21:16:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:40 smithi026 ceph-mon[98490]: pgmap v194: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 734 op/s 2023-10-06T21:16:40.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:40 smithi078 ceph-mon[134896]: pgmap v194: 97 pgs: 97 active+clean; 7.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 734 op/s 2023-10-06T21:16:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 360.0 s, 511.4 tps, lat 54.036 ms stddev 55.102 2023-10-06T21:16:42.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:42 smithi167 ceph-mon[134494]: pgmap v195: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 19 MiB/s wr, 972 op/s 2023-10-06T21:16:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:42 smithi026 ceph-mon[98490]: pgmap v195: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 19 MiB/s wr, 972 op/s 2023-10-06T21:16:42.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:42 smithi078 ceph-mon[134896]: pgmap v195: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 19 MiB/s wr, 972 op/s 2023-10-06T21:16:44.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:44 smithi026 ceph-mon[98490]: pgmap v196: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-06T21:16:44.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:44 smithi078 ceph-mon[134896]: pgmap v196: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-06T21:16:44.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:44 smithi167 ceph-mon[134494]: pgmap v196: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-06T21:16:46.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:46 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:46.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:46 smithi026 ceph-mon[98490]: pgmap v197: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 13 MiB/s wr, 688 op/s 2023-10-06T21:16:46.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:46 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:46.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:46 smithi078 ceph-mon[134896]: pgmap v197: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 13 MiB/s wr, 688 op/s 2023-10-06T21:16:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:46 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:16:46.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:46 smithi167 ceph-mon[134494]: pgmap v197: 97 pgs: 97 active+clean; 7.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 13 MiB/s wr, 688 op/s 2023-10-06T21:16:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 365.0 s, 1599.4 tps, lat 107.077 ms stddev 489.809 2023-10-06T21:16:48.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:48 smithi026 ceph-mon[98490]: pgmap v198: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 579 op/s 2023-10-06T21:16:48.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:48 smithi078 ceph-mon[134896]: pgmap v198: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 579 op/s 2023-10-06T21:16:48.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:48 smithi167 ceph-mon[134494]: pgmap v198: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 579 op/s 2023-10-06T21:16:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.198+0000 7eff6a7ff700 1 -- 172.21.15.26:0/983718768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103cb0 msgr2=0x7eff64107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.198+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/983718768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103cb0 0x7eff64107cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7eff580098f0 tx=0x7eff58009c00 comp rx=0 tx=0).stop 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 -- 172.21.15.26:0/983718768 shutdown_connections 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/983718768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff6410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/983718768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64103300 0x7eff641036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/983718768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103cb0 0x7eff64107cf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 -- 172.21.15.26:0/983718768 >> 172.21.15.26:0/983718768 conn(0x7eff640feb40 msgr2=0x7eff64100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 -- 172.21.15.26:0/983718768 shutdown_connections 2023-10-06T21:16:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.199+0000 7eff6a7ff700 1 -- 172.21.15.26:0/983718768 wait complete. 2023-10-06T21:16:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.200+0000 7eff6a7ff700 1 Processor -- start 2023-10-06T21:16:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.200+0000 7eff6a7ff700 1 -- start start 2023-10-06T21:16:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103300 0x7eff640753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff64075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641a9ed0 con 0x7eff64108420 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641aa040 con 0x7eff64103300 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff6a7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641aa200 con 0x7eff64075e40 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.202+0000 7eff68d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff63fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103300 0x7eff640753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43708/0 (socket says 172.21.15.26:43708) 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 -- 172.21.15.26:0/1900727862 learned_addr learned my addr 172.21.15.26:0/1900727862 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff637fe700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff64075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 -- 172.21.15.26:0/1900727862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103300 msgr2=0x7eff640753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103300 0x7eff640753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 -- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 msgr2=0x7eff64075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff64075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff68d9c700 1 -- 172.21.15.26:0/1900727862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c009d00 con 0x7eff64075e40 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.203+0000 7eff637fe700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff64075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff68d9c700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e3c0 tx=0x7eff5c00e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff617fa700 1 -- 172.21.15.26:0/1900727862 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0092d0 con 0x7eff64075e40 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff617fa700 1 -- 172.21.15.26:0/1900727862 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5c00f040 con 0x7eff64075e40 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff617fa700 1 -- 172.21.15.26:0/1900727862 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007cc0 con 0x7eff64075e40 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff58009520 con 0x7eff64075e40 2023-10-06T21:16:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.204+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641aa7e0 con 0x7eff64075e40 2023-10-06T21:16:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.207+0000 7eff617fa700 1 -- 172.21.15.26:0/1900727862 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7eff5c003a00 con 0x7eff64075e40 2023-10-06T21:16:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.209+0000 7eff7e1f3b80 1 Processor -- start 2023-10-06T21:16:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 -- start start 2023-10-06T21:16:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce156ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce156f3d0 0x564ce15717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce14cd400 con 0x564ce156f3d0 2023-10-06T21:16:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce14bebc0 con 0x564ce1563c00 2023-10-06T21:16:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.211+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce14bc4e0 con 0x564ce1564520 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff63fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce156ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55306/0 (socket says 172.21.15.26:55306) 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 -- 172.21.15.26:0/1128420693 learned_addr learned my addr 172.21.15.26:0/1128420693 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce156f3d0 0x564ce15717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 -- 172.21.15.26:0/1128420693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 msgr2=0x564ce156ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce156ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 -- 172.21.15.26:0/1128420693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce156f3d0 msgr2=0x564ce15717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce156f3d0 0x564ce15717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.212+0000 7eff637fe700 1 -- 172.21.15.26:0/1128420693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ce12e2d80 con 0x564ce1563c00 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff637fe700 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff58005e90 tx=0x7eff58019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71276efde35fa5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff617fa700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 0x7eff4c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff4a7fc700 1 -- 172.21.15.26:0/1128420693 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800f460 con 0x564ce1563c00 2023-10-06T21:16:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff617fa700 1 -- 172.21.15.26:0/1900727862 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff5c0ccee0 con 0x7eff64075e40 2023-10-06T21:16:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff4a7fc700 1 -- 172.21.15.26:0/1128420693 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5801d030 con 0x564ce1563c00 2023-10-06T21:16:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff4a7fc700 1 -- 172.21.15.26:0/1128420693 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800b920 con 0x564ce1563c00 2023-10-06T21:16:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.213+0000 7eff63fff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 0x7eff4c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/1128420693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 msgr2=0x564ce1563fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff63fff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 0x7eff4c0a3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff54009c80 tx=0x7eff54009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff58005e90 tx=0x7eff58019040 comp rx=0 tx=0).stop 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/1128420693 shutdown_connections 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce156f3d0 0x564ce15717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1563fe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/1128420693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce156ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.214+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/1128420693 >> 172.21.15.26:0/1128420693 conn(0x564ce14d9200 msgr2=0x564ce1560080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.215+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/1128420693 shutdown_connections 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.215+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/1128420693 wait complete. 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.215+0000 7eff7e1f3b80 1 -- start start 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.215+0000 7eff7e1f3b80 1 Processor -- start 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce156c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce156c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce14b7dd0 con 0x564ce1564520 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce14546b0 con 0x564ce1563c00 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce1454350 con 0x564ce156f3d0 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff7e1f3b80 10 client.0 _ll_register_callbacks cb 0x564ce12b17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55310/0 (socket says 172.21.15.26:55310) 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 -- 172.21.15.26:0/551531693 learned_addr learned my addr 172.21.15.26:0/551531693 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff637fe700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce156c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 -- 172.21.15.26:0/551531693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 msgr2=0x564ce156c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce156c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 -- 172.21.15.26:0/551531693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 msgr2=0x564ce156c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce156c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.216+0000 7eff63fff700 1 -- 172.21.15.26:0/551531693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff64110810 con 0x564ce1563c00 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.217+0000 7eff63fff700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff54011c50 tx=0x7eff54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58575a7c3cbc1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.217+0000 7eff437fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.217+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5401e9f0 con 0x564ce1563c00 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.217+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff5401eb50 con 0x564ce1563c00 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.217+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ce1557d50 con 0x564ce1563c00 2023-10-06T21:16:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.218+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5401ed10 con 0x564ce1563c00 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.219+0000 7eff7e1f3b80 10 client.26426 fetch_fsmap learned FSMap version 41 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.219+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564ce12cbc00 con 0x564ce1563c00 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.219+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ce1557d50 con 0x564ce1563c00 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.219+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7eff54026410 con 0x564ce1563c00 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 10 client.26426 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 20 client.26426 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce157ef40 0x564ce1581360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 4 client.26426 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ce14b0db0 con 0x564ce157ef40 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.220+0000 7eff637fe700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce157ef40 0x564ce1581360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.221+0000 7eff637fe700 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce157ef40 0x564ce1581360 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695351a614df4044 server_cookie=ce294877b8077d00 in_seq=0 out_seq=0 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.221+0000 7eff437fe700 10 client.26426 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.221+0000 7eff437fe700 1 -- 172.21.15.26:0/551531693 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564ce14b0db0 con 0x564ce157ef40 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.221+0000 7eff437fe700 10 client.26426 handle_command_reply: tid=0 2023-10-06T21:16:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.221+0000 7eff7e1f3b80 1 client.26426 shutdown 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 msgr2=0x564ce156edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff54011c50 tx=0x7eff54004750 comp rx=0 tx=0).stop 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 shutdown_connections 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce156c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce157ef40 0x564ce1581360 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce156edd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/551531693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce156c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff437fe700 0 client.26426 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 >> 172.21.15.26:0/551531693 conn(0x564ce14d9200 msgr2=0x564ce155fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 shutdown_connections 2023-10-06T21:16:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/551531693 wait complete. 2023-10-06T21:16:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.222+0000 7eff7e1f3b80 20 client.26426 trim_cache size 0 max 16384 2023-10-06T21:16:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.243+0000 7eff7e1f3b80 1 Processor -- start 2023-10-06T21:16:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.244+0000 7eff7e1f3b80 1 -- start start 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 0x564ce156c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce156f3d0 0x564ce156b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce13388c0 con 0x564ce1563c00 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce155c860 con 0x564ce156f3d0 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce1576420 con 0x564ce1564520 2023-10-06T21:16:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff637fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff637fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43730/0 (socket says 172.21.15.26:43730) 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.245+0000 7eff637fe700 1 -- 172.21.15.26:0/169278646 learned_addr learned my addr 172.21.15.26:0/169278646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff63fff700 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 0x564ce156c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff637fe700 1 -- 172.21.15.26:0/169278646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce156f3d0 msgr2=0x564ce156b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff637fe700 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce156f3d0 0x564ce156b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff637fe700 1 -- 172.21.15.26:0/169278646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 msgr2=0x564ce156c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff637fe700 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 0x564ce156c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff637fe700 1 -- 172.21.15.26:0/169278646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ce13091f0 con 0x564ce1564520 2023-10-06T21:16:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.246+0000 7eff63fff700 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 0x564ce156c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:16:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.259+0000 7eff637fe700 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7eff580059d0 tx=0x7eff58009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba426681d1e15e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.266+0000 7eff4affd700 1 -- 172.21.15.26:0/169278646 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802b040 con 0x564ce1564520 2023-10-06T21:16:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.266+0000 7eff4affd700 1 -- 172.21.15.26:0/169278646 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5800f5a0 con 0x564ce1564520 2023-10-06T21:16:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.266+0000 7eff4affd700 1 -- 172.21.15.26:0/169278646 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58005530 con 0x564ce1564520 2023-10-06T21:16:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.267+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/169278646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 msgr2=0x564ce1575ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.267+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7eff580059d0 tx=0x7eff58009c00 comp rx=0 tx=0).stop 2023-10-06T21:16:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/169278646 shutdown_connections 2023-10-06T21:16:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce156f3d0 0x564ce156b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1563c00 0x564ce156c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/169278646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce1564520 0x564ce1575ee0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/169278646 >> 172.21.15.26:0/169278646 conn(0x564ce14d9200 msgr2=0x564ce1585d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/169278646 shutdown_connections 2023-10-06T21:16:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.270+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/169278646 wait complete. 2023-10-06T21:16:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.271+0000 7eff7e1f3b80 1 -- start start 2023-10-06T21:16:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.271+0000 7eff7e1f3b80 1 Processor -- start 2023-10-06T21:16:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1571ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce1572400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce1572940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce15d1ed0 con 0x564ce1564520 2023-10-06T21:16:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce15d2040 con 0x564ce1563c00 2023-10-06T21:16:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ce15e3210 con 0x564ce156f3d0 2023-10-06T21:16:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.272+0000 7eff7e1f3b80 10 client.0 _ll_register_callbacks cb 0x7eff54011810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:16:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.273+0000 7eff637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce1572400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.273+0000 7eff68d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce1572940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.273+0000 7eff637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce1572400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35116/0 (socket says 172.21.15.26:35116) 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.273+0000 7eff637fe700 1 -- 172.21.15.26:0/634615249 learned_addr learned my addr 172.21.15.26:0/634615249 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 -- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 msgr2=0x564ce1571ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1571ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 -- 172.21.15.26:0/634615249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 msgr2=0x564ce1572400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce1572400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 -- 172.21.15.26:0/634615249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ce12e2d80 con 0x564ce156f3d0 2023-10-06T21:16:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff68d9c700 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce1572940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0ca5e0 tx=0x7eff5c0962c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f29e90a54601aee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:16:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.274+0000 7eff437fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:16:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c099860 con 0x564ce156f3d0 2023-10-06T21:16:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff5c0112c0 con 0x564ce156f3d0 2023-10-06T21:16:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5801d030 con 0x564ce156f3d0 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c0b8170 con 0x564ce156f3d0 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff7e1f3b80 10 client.26380 fetch_fsmap learned FSMap version 41 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564ce12e1250 con 0x564ce156f3d0 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5c07c080 con 0x564ce156f3d0 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.275+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7eff5c09a160 con 0x564ce156f3d0 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 10 client.26380 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 20 client.26380 populate_metadata read hostname 'smithi026' 2023-10-06T21:16:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce15e49d0 0x564ce15e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:16:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 4 client.26380 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:16:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ce14b0ad0 con 0x564ce15e49d0 2023-10-06T21:16:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.276+0000 7eff63fff700 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce15e49d0 0x564ce15e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:16:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.277+0000 7eff63fff700 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce15e49d0 0x564ce15e6df0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704ce0f7cd59ab20 server_cookie=da78634ac19f8544 in_seq=0 out_seq=0 2023-10-06T21:16:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.277+0000 7eff437fe700 10 client.26380 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.305+0000 7eff437fe700 1 -- 172.21.15.26:0/634615249 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ce14b0ad0 con 0x564ce15e49d0 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.305+0000 7eff437fe700 10 client.26380 handle_command_reply: tid=0 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.305+0000 7eff7e1f3b80 1 client.26380 shutdown 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 msgr2=0x564ce1572940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce1572940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0ca5e0 tx=0x7eff5c0962c0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 shutdown_connections 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ce156f3d0 0x564ce1572940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564ce15e49d0 0x564ce15e6df0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ce1563c00 0x564ce1571ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 --2- 172.21.15.26:0/634615249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ce1564520 0x564ce1572400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 >> 172.21.15.26:0/634615249 conn(0x564ce14d9200 msgr2=0x564ce157a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff437fe700 0 client.26380 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 shutdown_connections 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 1 -- 172.21.15.26:0/634615249 wait complete. 2023-10-06T21:16:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.306+0000 7eff7e1f3b80 20 client.26380 trim_cache size 0 max 16384 2023-10-06T21:16:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:16:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:16:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:16:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 msgr2=0x7eff4c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 0x7eff4c0a3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff54009c80 tx=0x7eff54009260 comp rx=0 tx=0).stop 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 msgr2=0x7eff641a9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e3c0 tx=0x7eff5c00e6d0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 shutdown_connections 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff64075e40 0x7eff641a9820 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff64103300 0x7eff640753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7eff4c0a1670 0x7eff4c0a3b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 --2- 172.21.15.26:0/1900727862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff64108420 0x7eff64075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:16:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 >> 172.21.15.26:0/1900727862 conn(0x7eff640feb40 msgr2=0x7eff640ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:16:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 shutdown_connections 2023-10-06T21:16:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:16:50.307+0000 7eff6a7ff700 1 -- 172.21.15.26:0/1900727862 wait complete. 2023-10-06T21:16:50.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:50 smithi026 ceph-mon[98490]: pgmap v199: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.1 MiB/s wr, 437 op/s 2023-10-06T21:16:50.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:50 smithi078 ceph-mon[134896]: pgmap v199: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.1 MiB/s wr, 437 op/s 2023-10-06T21:16:50.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:50 smithi167 ceph-mon[134494]: pgmap v199: 97 pgs: 97 active+clean; 7.1 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.1 MiB/s wr, 437 op/s 2023-10-06T21:16:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 370.0 s, 1218.2 tps, lat 82.697 ms stddev 126.396 2023-10-06T21:16:52.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:52 smithi026 ceph-mon[98490]: pgmap v200: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 33 MiB/s wr, 1.89k op/s 2023-10-06T21:16:52.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:52 smithi078 ceph-mon[134896]: pgmap v200: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 33 MiB/s wr, 1.89k op/s 2023-10-06T21:16:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:52 smithi167 ceph-mon[134494]: pgmap v200: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 33 MiB/s wr, 1.89k op/s 2023-10-06T21:16:54.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:54 smithi026 ceph-mon[98490]: pgmap v201: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 1.65k op/s 2023-10-06T21:16:54.379 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:54 smithi078 ceph-mon[134896]: pgmap v201: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 1.65k op/s 2023-10-06T21:16:54.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:54 smithi167 ceph-mon[134494]: pgmap v201: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 1.65k op/s 2023-10-06T21:16:56.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:56 smithi167 ceph-mon[134494]: pgmap v202: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-10-06T21:16:56.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:56 smithi026 ceph-mon[98490]: pgmap v202: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-10-06T21:16:56.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:56 smithi078 ceph-mon[134896]: pgmap v202: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2023-10-06T21:16:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 375.0 s, 1896.5 tps, lat 52.921 ms stddev 62.341 2023-10-06T21:16:58.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:16:58 smithi167 ceph-mon[134494]: pgmap v203: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-10-06T21:16:58.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:16:58 smithi026 ceph-mon[98490]: pgmap v203: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-10-06T21:16:58.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:16:58 smithi078 ceph-mon[134896]: pgmap v203: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2023-10-06T21:17:00.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:00 smithi026 ceph-mon[98490]: pgmap v204: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 29 MiB/s wr, 1.73k op/s 2023-10-06T21:17:00.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:00 smithi078 ceph-mon[134896]: pgmap v204: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 29 MiB/s wr, 1.73k op/s 2023-10-06T21:17:00.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:00 smithi167 ceph-mon[134494]: pgmap v204: 97 pgs: 97 active+clean; 7.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 29 MiB/s wr, 1.73k op/s 2023-10-06T21:17:01.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:17:02.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 -- 172.21.15.26:0/1027977167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b0068fd0 msgr2=0x7f05b00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1027977167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b0068fd0 0x7f05b00693b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009920 tx=0x7f05a4009c30 comp rx=0 tx=0).stop 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 -- 172.21.15.26:0/1027977167 shutdown_connections 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1027977167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1027977167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b0068fd0 0x7f05b00693b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1027977167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b00698f0 0x7f05b010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.093+0000 7f05b61ff700 1 -- 172.21.15.26:0/1027977167 >> 172.21.15.26:0/1027977167 conn(0x7f05b0077ce0 msgr2=0x7f05b00780f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.094+0000 7f05b61ff700 1 -- 172.21.15.26:0/1027977167 shutdown_connections 2023-10-06T21:17:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.094+0000 7f05b61ff700 1 -- 172.21.15.26:0/1027977167 wait complete. 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 Processor -- start 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 -- start start 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b00698f0 0x7f05b010bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b010c930 con 0x7f05b00698f0 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a9ae0 con 0x7f05b010daf0 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.095+0000 7f05b61ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a9ca0 con 0x7f05b0068fd0 2023-10-06T21:17:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:46998/0 (socket says 172.21.15.26:46998) 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35274/0 (socket says 172.21.15.26:35274) 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 -- 172.21.15.26:0/1936381643 learned_addr learned my addr 172.21.15.26:0/1936381643 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05aeffd700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b00698f0 0x7f05b010bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 -- 172.21.15.26:0/1936381643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 msgr2=0x7f05b01a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 -- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b00698f0 msgr2=0x7f05b010bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b00698f0 0x7f05b010bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 -- 172.21.15.26:0/1936381643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009520 con 0x7f05b0068fd0 2023-10-06T21:17:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05affff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.096+0000 7f05af7fe700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009840 tx=0x7f05a400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.097+0000 7f05acff9700 1 -- 172.21.15.26:0/1936381643 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a401c070 con 0x7f05b0068fd0 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.097+0000 7f05acff9700 1 -- 172.21.15.26:0/1936381643 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a400bb20 con 0x7f05b0068fd0 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.097+0000 7f05acff9700 1 -- 172.21.15.26:0/1936381643 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40038e0 con 0x7f05b0068fd0 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.098+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b01a9f20 con 0x7f05b0068fd0 2023-10-06T21:17:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.098+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b01aa2f0 con 0x7f05b0068fd0 2023-10-06T21:17:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.105+0000 7f05acff9700 1 -- 172.21.15.26:0/1936381643 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f05a4020030 con 0x7f05b0068fd0 2023-10-06T21:17:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.106+0000 7f05c9b56b80 1 Processor -- start 2023-10-06T21:17:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.106+0000 7f05c9b56b80 1 -- start start 2023-10-06T21:17:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f827906460 con 0x55f8279a8890 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8278f7c20 con 0x55f82799d0c0 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8278f5540 con 0x55f82799d9e0 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35286/0 (socket says 172.21.15.26:35286) 2023-10-06T21:17:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05af7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.107+0000 7f05affff700 1 -- 172.21.15.26:0/2366218014 learned_addr learned my addr 172.21.15.26:0/2366218014 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.108+0000 7f05affff700 1 -- 172.21.15.26:0/2366218014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 msgr2=0x55f8279a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.108+0000 7f05affff700 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.108+0000 7f05affff700 1 -- 172.21.15.26:0/2366218014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 msgr2=0x55f8279aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.108+0000 7f05affff700 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.108+0000 7f05acff9700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 0x7f05980a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.109+0000 7f05affff700 1 -- 172.21.15.26:0/2366218014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f82771bde0 con 0x55f82799d0c0 2023-10-06T21:17:02.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.109+0000 7f05acff9700 1 -- 172.21.15.26:0/1936381643 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05a40cdef0 con 0x7f05b0068fd0 2023-10-06T21:17:02.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.110+0000 7f05affff700 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05a8003e10 tx=0x7f05a80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3e90a10419e7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.110+0000 7f0595ffb700 1 -- 172.21.15.26:0/2366218014 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80090d0 con 0x55f82799d0c0 2023-10-06T21:17:02.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.110+0000 7f0595ffb700 1 -- 172.21.15.26:0/2366218014 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a800f040 con 0x55f82799d0c0 2023-10-06T21:17:02.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.110+0000 7f0595ffb700 1 -- 172.21.15.26:0/2366218014 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800e7f0 con 0x55f82799d0c0 2023-10-06T21:17:02.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.110+0000 7f05affff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 0x7f05980a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05c9b56b80 1 -- 172.21.15.26:0/2366218014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 msgr2=0x55f82799d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05affff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 0x7f05980a3cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004010 tx=0x7f05a800e970 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f05a8003e10 tx=0x7f05a80099e0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05c9b56b80 1 -- 172.21.15.26:0/2366218014 shutdown_connections 2023-10-06T21:17:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.112+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f82799d4a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.113+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/2366218014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.113+0000 7f05c9b56b80 1 -- 172.21.15.26:0/2366218014 >> 172.21.15.26:0/2366218014 conn(0x55f827912260 msgr2=0x55f827999450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.113+0000 7f05c9b56b80 1 -- 172.21.15.26:0/2366218014 shutdown_connections 2023-10-06T21:17:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.113+0000 7f05c9b56b80 1 -- 172.21.15.26:0/2366218014 wait complete. 2023-10-06T21:17:02.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.113+0000 7f05c9b56b80 1 -- start start 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.114+0000 7f05c9b56b80 1 Processor -- start 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d0c0 0x55f8279a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f8279a8890 0x55f8279a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8278f0e30 con 0x55f82799d0c0 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82788d710 con 0x55f8279a8890 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82788d3b0 con 0x55f82799d9e0 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05c9b56b80 10 client.0 _ll_register_callbacks cb 0x55f8276ea800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.115+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:47010/0 (socket says 172.21.15.26:47010) 2023-10-06T21:17:02.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 -- 172.21.15.26:0/1257020271 learned_addr learned my addr 172.21.15.26:0/1257020271 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 -- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f8279a8890 msgr2=0x55f8279a5c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f8279a8890 0x55f8279a5c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 -- 172.21.15.26:0/1257020271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d0c0 msgr2=0x55f8279a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d0c0 0x55f8279a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 -- 172.21.15.26:0/1257020271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8003ac0 con 0x55f82799d9e0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f05aeffd700 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05a0011760 tx=0x7f05a0011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83574f6da51a2aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.116+0000 7f058effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004250 con 0x55f82799d9e0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f827990f10 con 0x55f82799d9e0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05a00043b0 con 0x55f82799d9e0 2023-10-06T21:17:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05a0004570 con 0x55f82799d9e0 2023-10-06T21:17:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f05c9b56b80 10 client.26392 fetch_fsmap learned FSMap version 41 2023-10-06T21:17:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f827704c60 con 0x55f82799d9e0 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.117+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a0049020 con 0x55f82799d9e0 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f05a000f9e0 con 0x55f82799d9e0 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 20 client.26392 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f8279b6d70 0x55f8279b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 4 client.26392 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8278e9e10 con 0x55f8279b6d70 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.118+0000 7f05af7fe700 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f8279b6d70 0x55f8279b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.120+0000 7f05af7fe700 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f8279b6d70 0x55f8279b9190 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3350cbc42da0ea97 server_cookie=7609b86c1830548a in_seq=0 out_seq=0 2023-10-06T21:17:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.120+0000 7f058effd700 10 client.26392 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.121+0000 7f058effd700 1 -- 172.21.15.26:0/1257020271 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8278e9e10 con 0x55f8279b6d70 2023-10-06T21:17:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.121+0000 7f058effd700 10 client.26392 handle_command_reply: tid=0 2023-10-06T21:17:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.121+0000 7f05c9b56b80 1 client.26392 shutdown 2023-10-06T21:17:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 380.0 s, 1748.4 tps, lat 57.196 ms stddev 86.634 2023-10-06T21:17:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 msgr2=0x55f8279a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05a0011760 tx=0x7f05a0011b20 comp rx=0 tx=0).stop 2023-10-06T21:17:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 shutdown_connections 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f8279b6d70 0x55f8279b9190 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f8279a8890 0x55f8279a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d0c0 0x55f8279a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/1257020271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d9e0 0x55f8279a5550 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 >> 172.21.15.26:0/1257020271 conn(0x55f827912260 msgr2=0x55f827998fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f058effd700 0 client.26392 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 shutdown_connections 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.122+0000 7f05c9b56b80 1 -- 172.21.15.26:0/1257020271 wait complete. 2023-10-06T21:17:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.123+0000 7f05c9b56b80 20 client.26392 trim_cache size 0 max 16384 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 Processor -- start 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 -- start start 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f8279a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 0x55f8279c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 0x55f8279c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f827771920 con 0x55f82799d9e0 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82799d720 con 0x55f82799d0c0 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.149+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f82799e2e0 con 0x55f8279a8890 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05af7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f8279a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 0x55f8279c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 0x55f8279c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 0x55f8279c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34196/0 (socket says 172.21.15.26:34196) 2023-10-06T21:17:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05af7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f8279a7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35314/0 (socket says 172.21.15.26:35314) 2023-10-06T21:17:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05aeffd700 1 -- 172.21.15.26:0/185181424 learned_addr learned my addr 172.21.15.26:0/185181424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 -- 172.21.15.26:0/185181424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 msgr2=0x55f8279a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f8279a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 -- 172.21.15.26:0/185181424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 msgr2=0x55f8279c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 0x55f8279c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.150+0000 7f05affff700 1 -- 172.21.15.26:0/185181424 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f827742250 con 0x55f8279a8890 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.151+0000 7f05affff700 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 0x55f8279c4ea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05a801c040 tx=0x7f05a8025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5ca89b87110147 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.151+0000 7f05967fc700 1 -- 172.21.15.26:0/185181424 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800e790 con 0x55f8279a8890 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.151+0000 7f05967fc700 1 -- 172.21.15.26:0/185181424 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a800eda0 con 0x55f8279a8890 2023-10-06T21:17:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.151+0000 7f05967fc700 1 -- 172.21.15.26:0/185181424 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8007820 con 0x55f8279a8890 2023-10-06T21:17:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.152+0000 7f05c9b56b80 1 -- 172.21.15.26:0/185181424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 msgr2=0x55f8279c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.152+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 0x55f8279c4ea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f05a801c040 tx=0x7f05a8025000 comp rx=0 tx=0).stop 2023-10-06T21:17:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 -- 172.21.15.26:0/185181424 shutdown_connections 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8279a8890 0x55f8279c4ea0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d0c0 0x55f8279a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/185181424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f82799d9e0 0x55f8279c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 -- 172.21.15.26:0/185181424 >> 172.21.15.26:0/185181424 conn(0x55f827912260 msgr2=0x55f8279a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 -- 172.21.15.26:0/185181424 shutdown_connections 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.153+0000 7f05c9b56b80 1 -- 172.21.15.26:0/185181424 wait complete. 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.154+0000 7f05c9b56b80 1 -- start start 2023-10-06T21:17:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.154+0000 7f05c9b56b80 1 Processor -- start 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 0x55f8279ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 0x55f8279ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f827a0be00 con 0x55f8279a8890 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f827a0bf70 con 0x55f82799d9e0 2023-10-06T21:17:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f827a0c130 con 0x55f82799d0c0 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05c9b56b80 10 client.0 _ll_register_callbacks cb 0x7f05a0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05affff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 0x55f8279ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05aeffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 0x55f8279ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35326/0 (socket says 172.21.15.26:35326) 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.155+0000 7f05aeffd700 1 -- 172.21.15.26:0/26678468 learned_addr learned my addr 172.21.15.26:0/26678468 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 0x55f8279ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 -- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 msgr2=0x55f8279ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 0x55f8279ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 -- 172.21.15.26:0/26678468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 msgr2=0x55f8279ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.156+0000 7f05af7fe700 1 -- 172.21.15.26:0/26678468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8028070 con 0x55f82799d0c0 2023-10-06T21:17:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.157+0000 7f05af7fe700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 0x55f8279ad3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05a40cb5f0 tx=0x7f05a4005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=406946cafda95eb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f058effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a409ace0 con 0x55f82799d0c0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05a409b610 con 0x55f82799d0c0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05a800f040 con 0x55f82799d0c0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05a409aa50 con 0x55f82799d0c0 2023-10-06T21:17:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f05c9b56b80 10 client.26398 fetch_fsmap learned FSMap version 41 2023-10-06T21:17:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f82771a2b0 con 0x55f82799d0c0 2023-10-06T21:17:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.158+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a800f040 con 0x55f82799d0c0 2023-10-06T21:17:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f05a40cb870 con 0x55f82799d0c0 2023-10-06T21:17:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 10 client.26398 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:17:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 10 client.26398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 20 client.26398 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f827a0d480 0x55f827a0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 4 client.26398 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.159+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8278e9b30 con 0x55f827a0d480 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f05aeffd700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f827a0d480 0x55f827a0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f05aeffd700 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f827a0d480 0x55f827a0f8a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca142846ba7b8e1 server_cookie=3336fcfaf15a4963 in_seq=0 out_seq=0 2023-10-06T21:17:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f058effd700 10 client.26398 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f058effd700 1 -- 172.21.15.26:0/26678468 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f8278e9b30 con 0x55f827a0d480 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f058effd700 10 client.26398 handle_command_reply: tid=0 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.160+0000 7f05c9b56b80 1 client.26398 shutdown 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 msgr2=0x55f8279ad3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 0x55f8279ad3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05a40cb5f0 tx=0x7f05a4005510 comp rx=0 tx=0).stop 2023-10-06T21:17:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 shutdown_connections 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8279a8890 0x55f8279ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f82799d0c0 0x55f8279ad3c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f827a0d480 0x55f827a0f8a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 --2- 172.21.15.26:0/26678468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82799d9e0 0x55f8279ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 >> 172.21.15.26:0/26678468 conn(0x55f827912260 msgr2=0x55f8279c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f058effd700 0 client.26398 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 shutdown_connections 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 1 -- 172.21.15.26:0/26678468 wait complete. 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.161+0000 7f05c9b56b80 20 client.26398 trim_cache size 0 max 16384 2023-10-06T21:17:02.165 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:17:02.166 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.163+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 msgr2=0x7f05980a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.163+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 0x7f05980a3cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004010 tx=0x7f05a800e970 comp rx=0 tx=0).stop 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.163+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 msgr2=0x7f05b010bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.163+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009840 tx=0x7f05a400fa00 comp rx=0 tx=0).stop 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 shutdown_connections 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05b010daf0 0x7f05b01a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05b0068fd0 0x7f05b010bab0 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009840 tx=0x7f05a400fa00 comp rx=0 tx=0).stop 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f05980a1800 0x7f05980a3cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 --2- 172.21.15.26:0/1936381643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05b00698f0 0x7f05b010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 >> 172.21.15.26:0/1936381643 conn(0x7f05b0077ce0 msgr2=0x7f05b010ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.164+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 shutdown_connections 2023-10-06T21:17:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:02.165+0000 7f05b61ff700 1 -- 172.21.15.26:0/1936381643 wait complete. 2023-10-06T21:17:02.192 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:17:02.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:02 smithi026 ceph-mon[98490]: pgmap v205: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 35 MiB/s wr, 2.14k op/s 2023-10-06T21:17:02.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:02 smithi078 ceph-mon[134896]: pgmap v205: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 35 MiB/s wr, 2.14k op/s 2023-10-06T21:17:02.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:02 smithi167 ceph-mon[134494]: pgmap v205: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 35 MiB/s wr, 2.14k op/s 2023-10-06T21:17:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.550+0000 7fe7549ff700 1 -- 172.21.15.26:0/1210978294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75010d6a0 msgr2=0x7fe75010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.550+0000 7fe7549ff700 1 --2- 172.21.15.26:0/1210978294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75010d6a0 0x7fe75010fb10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b0a0 tx=0x7fe74800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 -- 172.21.15.26:0/1210978294 shutdown_connections 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 --2- 172.21.15.26:0/1210978294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe75010d6a0 0x7fe75010fb10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 --2- 172.21.15.26:0/1210978294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 0x7fe7500736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 --2- 172.21.15.26:0/1210978294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe750073c00 0x7fe75010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 -- 172.21.15.26:0/1210978294 >> 172.21.15.26:0/1210978294 conn(0x7fe7500fc950 msgr2=0x7fe7500fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 -- 172.21.15.26:0/1210978294 shutdown_connections 2023-10-06T21:17:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.551+0000 7fe7549ff700 1 -- 172.21.15.26:0/1210978294 wait complete. 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 Processor -- start 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 -- start start 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 0x7fe7501065f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 0x7fe7501014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750101c70 con 0x7fe7500732e0 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750101de0 con 0x7fe750073c00 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.553+0000 7fe7549ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750101fa0 con 0x7fe75010d6a0 2023-10-06T21:17:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35342/0 (socket says 172.21.15.26:35342) 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe74ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 0x7fe7501014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 -- 172.21.15.26:0/924678894 learned_addr learned my addr 172.21.15.26:0/924678894 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe74e59c700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 0x7fe7501065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 -- 172.21.15.26:0/924678894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 msgr2=0x7fe7501014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 0x7fe7501014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 -- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 msgr2=0x7fe7501065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 0x7fe7501065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe747fff700 1 -- 172.21.15.26:0/924678894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748009d00 con 0x7fe750073c00 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.554+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 0x7fe7501014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe747fff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e9a0 tx=0x7fe73800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe7477fe700 1 -- 172.21.15.26:0/924678894 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738009790 con 0x7fe750073c00 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe7477fe700 1 -- 172.21.15.26:0/924678894 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe73800cbd0 con 0x7fe750073c00 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe7477fe700 1 -- 172.21.15.26:0/924678894 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738018680 con 0x7fe750073c00 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe750102280 con 0x7fe750073c00 2023-10-06T21:17:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.555+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75010c640 con 0x7fe750073c00 2023-10-06T21:17:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.562+0000 7fe768268b80 1 Processor -- start 2023-10-06T21:17:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.562+0000 7fe7477fe700 1 -- 172.21.15.26:0/924678894 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe73800cd40 con 0x7fe750073c00 2023-10-06T21:17:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.562+0000 7fe768268b80 1 -- start start 2023-10-06T21:17:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0621a400 con 0x560e062b1520 2023-10-06T21:17:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0620bbc0 con 0x560e062bc3d0 2023-10-06T21:17:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe768268b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e062094e0 con 0x560e062b0c00 2023-10-06T21:17:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.564+0000 7fe747fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.565+0000 7fe747fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35356/0 (socket says 172.21.15.26:35356) 2023-10-06T21:17:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.565+0000 7fe747fff700 1 -- 172.21.15.26:0/989040322 learned_addr learned my addr 172.21.15.26:0/989040322 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.565+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.565+0000 7fe74e59c700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 -- 172.21.15.26:0/989040322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 msgr2=0x560e062b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 -- 172.21.15.26:0/989040322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 msgr2=0x560e062bbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062bbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 -- 172.21.15.26:0/989040322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0602fd80 con 0x560e062bc3d0 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe7477fe700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 0x7fe73c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe747fff700 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ca940 tx=0x7fe7380d5440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb4efa9f8f29965 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.566+0000 7fe7477fe700 1 -- 172.21.15.26:0/924678894 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe738014070 con 0x7fe750073c00 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 0x7fe73c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe744ff9700 1 -- 172.21.15.26:0/989040322 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7380d0cf0 con 0x560e062bc3d0 2023-10-06T21:17:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe744ff9700 1 -- 172.21.15.26:0/989040322 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7380965a0 con 0x560e062bc3d0 2023-10-06T21:17:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe744ff9700 1 -- 172.21.15.26:0/989040322 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73809a960 con 0x560e062bc3d0 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe768268b80 1 -- 172.21.15.26:0/989040322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 msgr2=0x560e062be7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 0x7fe73c0a3b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe750107280 tx=0x7fe748006230 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.567+0000 7fe768268b80 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ca940 tx=0x7fe7380d5440 comp rx=0 tx=0).stop 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 -- 172.21.15.26:0/989040322 shutdown_connections 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062be7c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 --2- 172.21.15.26:0/989040322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 -- 172.21.15.26:0/989040322 >> 172.21.15.26:0/989040322 conn(0x560e06226200 msgr2=0x560e062ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 -- 172.21.15.26:0/989040322 shutdown_connections 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 -- 172.21.15.26:0/989040322 wait complete. 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.568+0000 7fe768268b80 1 -- start start 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 Processor -- start 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062bc3d0 0x560e062b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e06204dd0 con 0x560e062b1520 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e061a16b0 con 0x560e062b0c00 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e061a1350 con 0x560e062bc3d0 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe768268b80 10 client.0 _ll_register_callbacks cb 0x560e05ffe7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe74e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.569+0000 7fe74e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35358/0 (socket says 172.21.15.26:35358) 2023-10-06T21:17:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.570+0000 7fe74e59c700 1 -- 172.21.15.26:0/3463109845 learned_addr learned my addr 172.21.15.26:0/3463109845 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 -- 172.21.15.26:0/3463109845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062bc3d0 msgr2=0x560e062b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062bc3d0 0x560e062b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 -- 172.21.15.26:0/3463109845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 msgr2=0x560e062b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe747fff700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 -- 172.21.15.26:0/3463109845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7501107d0 con 0x560e062b0c00 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe747fff700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe74e59c700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe740010810 tx=0x7fe740010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ed3f40b1f4a11d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.571+0000 7fe735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000c9c0 con 0x560e062b0c00 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe740017030 con 0x560e062b0c00 2023-10-06T21:17:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e062a4d50 con 0x560e062b0c00 2023-10-06T21:17:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe74000cb20 con 0x560e062b0c00 2023-10-06T21:17:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe768268b80 10 client.26453 fetch_fsmap learned FSMap version 41 2023-10-06T21:17:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.572+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe75010c640 con 0x560e062b0c00 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e062a4d50 con 0x560e062b0c00 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe74001c430 con 0x560e062b0c00 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 10 client.26453 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 20 client.26453 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e062cbf90 0x560e062ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 4 client.26453 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.573+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e061fddb0 con 0x560e062cbf90 2023-10-06T21:17:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.574+0000 7fe747fff700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e062cbf90 0x560e062ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.574+0000 7fe747fff700 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e062cbf90 0x560e062ce3b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca8f0f714244a9ed server_cookie=da6c7d18a76364c4 in_seq=0 out_seq=0 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.574+0000 7fe735ffb700 10 client.26453 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.575+0000 7fe735ffb700 1 -- 172.21.15.26:0/3463109845 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e061fddb0 con 0x560e062cbf90 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.575+0000 7fe735ffb700 10 client.26453 handle_command_reply: tid=0 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.575+0000 7fe768268b80 1 client.26453 shutdown 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.576+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 msgr2=0x560e062bbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.576+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe740010810 tx=0x7fe740010bd0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 shutdown_connections 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062bc3d0 0x560e062b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e062cbf90 0x560e062ce3b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062b0c00 0x560e062bbe00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 --2- 172.21.15.26:0/3463109845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 >> 172.21.15.26:0/3463109845 conn(0x560e06226200 msgr2=0x560e062acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe735ffb700 0 client.26453 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 shutdown_connections 2023-10-06T21:17:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.577+0000 7fe768268b80 1 -- 172.21.15.26:0/3463109845 wait complete. 2023-10-06T21:17:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.578+0000 7fe768268b80 20 client.26453 trim_cache size 0 max 16384 2023-10-06T21:17:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.597+0000 7fe768268b80 1 Processor -- start 2023-10-06T21:17:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.597+0000 7fe768268b80 1 -- start start 2023-10-06T21:17:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.597+0000 7fe768268b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.597+0000 7fe768268b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe768268b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe768268b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e060858c0 con 0x560e062b1520 2023-10-06T21:17:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe768268b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e062b3fa0 con 0x560e062bc3d0 2023-10-06T21:17:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe768268b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e062c9550 con 0x560e062b0c00 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35362/0 (socket says 172.21.15.26:35362) 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 -- 172.21.15.26:0/2957587148 learned_addr learned my addr 172.21.15.26:0/2957587148 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe747fff700 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74e59c700 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 -- 172.21.15.26:0/2957587148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 msgr2=0x560e062bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062bb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 -- 172.21.15.26:0/2957587148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 msgr2=0x560e062b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.598+0000 7fe74ed9d700 1 -- 172.21.15.26:0/2957587148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e06018c00 con 0x560e062bc3d0 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.599+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe748015000 tx=0x7fe74801f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d920aaf87af2048a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.599+0000 7fe7457fa700 1 -- 172.21.15.26:0/2957587148 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800e040 con 0x560e062bc3d0 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.599+0000 7fe7457fa700 1 -- 172.21.15.26:0/2957587148 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe748021070 con 0x560e062bc3d0 2023-10-06T21:17:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.599+0000 7fe7457fa700 1 -- 172.21.15.26:0/2957587148 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748029420 con 0x560e062bc3d0 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 -- 172.21.15.26:0/2957587148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 msgr2=0x560e062b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe748015000 tx=0x7fe74801f070 comp rx=0 tx=0).stop 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 -- 172.21.15.26:0/2957587148 shutdown_connections 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e062b4970 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b0c00 0x560e062bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 --2- 172.21.15.26:0/2957587148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b1520 0x560e062b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 -- 172.21.15.26:0/2957587148 >> 172.21.15.26:0/2957587148 conn(0x560e06226200 msgr2=0x560e062c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 -- 172.21.15.26:0/2957587148 shutdown_connections 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.600+0000 7fe768268b80 1 -- 172.21.15.26:0/2957587148 wait complete. 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.601+0000 7fe768268b80 1 -- start start 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.601+0000 7fe768268b80 1 Processor -- start 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b0c00 0x560e062c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 0x560e062c1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e062c1bb0 con 0x560e062b0c00 2023-10-06T21:17:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e062c1d20 con 0x560e062bc3d0 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0631f320 con 0x560e062b1520 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe768268b80 10 client.0 _ll_register_callbacks cb 0x7fe74000ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe747fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 0x560e062c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe74ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe74ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35374/0 (socket says 172.21.15.26:35374) 2023-10-06T21:17:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.602+0000 7fe74ed9d700 1 -- 172.21.15.26:0/439090281 learned_addr learned my addr 172.21.15.26:0/439090281 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 -- 172.21.15.26:0/439090281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 msgr2=0x560e062c1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 0x560e062c1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 -- 172.21.15.26:0/439090281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b0c00 msgr2=0x560e062c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b0c00 0x560e062c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 -- 172.21.15.26:0/439090281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748003a00 con 0x560e062bc3d0 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe747fff700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 0x560e062c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe74ed9d700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe748009610 tx=0x7fe748007880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9d3b15ceb1c49a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.603+0000 7fe735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.604+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74800e070 con 0x560e062bc3d0 2023-10-06T21:17:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.604+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe7480052c0 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.605+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7380965a0 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.605+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe748036070 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.605+0000 7fe768268b80 10 client.26459 fetch_fsmap learned FSMap version 41 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.605+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560e060561f0 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7380965a0 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 41) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe748041020 con 0x560e062bc3d0 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 10 client.26459 fetch_fsmap finished waiting for FSMap version 41 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 20 client.26459 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e06320740 0x560e06322b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 4 client.26459 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:17:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e061fdad0 con 0x560e06320740 2023-10-06T21:17:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.606+0000 7fe74e59c700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e06320740 0x560e06322b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.607+0000 7fe74e59c700 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e06320740 0x560e06322b60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feed4ccd9ee9752d server_cookie=1f830253a6100a42 in_seq=0 out_seq=0 2023-10-06T21:17:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.607+0000 7fe735ffb700 10 client.26459 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.630+0000 7fe735ffb700 1 -- 172.21.15.26:0/439090281 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e061fdad0 con 0x560e06320740 2023-10-06T21:17:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.630+0000 7fe735ffb700 10 client.26459 handle_command_reply: tid=0 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 client.26459 shutdown 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 msgr2=0x560e0631ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe748009610 tx=0x7fe748007880 comp rx=0 tx=0).stop 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 shutdown_connections 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e06320740 0x560e06322b60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e062bc3d0 0x560e0631ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e062b0c00 0x560e062c0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 --2- 172.21.15.26:0/439090281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e062b1520 0x560e062c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe735ffb700 0 client.26459 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 >> 172.21.15.26:0/439090281 conn(0x560e06226200 msgr2=0x560e062b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 shutdown_connections 2023-10-06T21:17:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.632+0000 7fe768268b80 1 -- 172.21.15.26:0/439090281 wait complete. 2023-10-06T21:17:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.633+0000 7fe768268b80 20 client.26459 trim_cache size 0 max 16384 2023-10-06T21:17:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.634+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 msgr2=0x7fe73c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.634+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 0x7fe73c0a3b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe750107280 tx=0x7fe748006230 comp rx=0 tx=0).stop 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.634+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 msgr2=0x7fe750106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.634+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e9a0 tx=0x7fe73800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 shutdown_connections 2023-10-06T21:17:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75010d6a0 0x7fe7501014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe73c0a1670 0x7fe73c0a3b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7500732e0 0x7fe7501065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 --2- 172.21.15.26:0/924678894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe750073c00 0x7fe750106b50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.635+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 >> 172.21.15.26:0/924678894 conn(0x7fe7500fc950 msgr2=0x7fe750198650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.636+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 shutdown_connections 2023-10-06T21:17:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:03.636+0000 7fe7549ff700 1 -- 172.21.15.26:0/924678894 wait complete. 2023-10-06T21:17:04.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:04 smithi078 ceph-mon[134896]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-06T21:17:04.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:04 smithi167 ceph-mon[134494]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-06T21:17:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:04 smithi026 ceph-mon[98490]: pgmap v206: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-06T21:17:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:06 smithi167 ceph-mon[134494]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:17:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:06 smithi167 ceph-mon[134494]: pgmap v208: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-10-06T21:17:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:06 smithi026 ceph-mon[98490]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:17:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:06 smithi026 ceph-mon[98490]: pgmap v208: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-10-06T21:17:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:06 smithi078 ceph-mon[134896]: osdmap e90: 12 total, 12 up, 12 in 2023-10-06T21:17:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:06 smithi078 ceph-mon[134896]: pgmap v208: 97 pgs: 97 active+clean; 7.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 MiB/s wr, 1.29k op/s 2023-10-06T21:17:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 385.0 s, 1842.0 tps, lat 54.087 ms stddev 66.594 2023-10-06T21:17:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:08 smithi167 ceph-mon[134494]: pgmap v209: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:08.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:08 smithi026 ceph-mon[98490]: pgmap v209: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:08.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:08 smithi078 ceph-mon[134896]: pgmap v209: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:10.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:10 smithi167 ceph-mon[134494]: pgmap v210: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:10.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:10 smithi026 ceph-mon[98490]: pgmap v210: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:10.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:10 smithi078 ceph-mon[134896]: pgmap v210: 97 pgs: 8 active+clean+snaptrim, 74 active+clean, 15 active+clean+snaptrim_wait; 7.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2023-10-06T21:17:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 390.0 s, 2028.8 tps, lat 49.587 ms stddev 53.115 2023-10-06T21:17:12.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:12 smithi026 ceph-mon[98490]: pgmap v211: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-10-06T21:17:12.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:12 smithi078 ceph-mon[134896]: pgmap v211: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-10-06T21:17:12.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:12 smithi167 ceph-mon[134494]: pgmap v211: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 1.50k op/s 2023-10-06T21:17:14.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:14 smithi078 ceph-mon[134896]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:17:14.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:14 smithi078 ceph-mon[134896]: pgmap v213: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-10-06T21:17:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:14 smithi167 ceph-mon[134494]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:17:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:14 smithi167 ceph-mon[134494]: pgmap v213: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-10-06T21:17:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:14 smithi026 ceph-mon[98490]: osdmap e91: 12 total, 12 up, 12 in 2023-10-06T21:17:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:14 smithi026 ceph-mon[98490]: pgmap v213: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-10-06T21:17:16.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:16 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:16.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:16 smithi167 ceph-mon[134494]: pgmap v214: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-10-06T21:17:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:16 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:16.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:16 smithi026 ceph-mon[98490]: pgmap v214: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-10-06T21:17:16.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:16 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:16.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:16 smithi078 ceph-mon[134896]: pgmap v214: 97 pgs: 97 active+clean; 6.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-10-06T21:17:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 -- 172.21.15.26:0/2139047872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8474103cb0 msgr2=0x7f8474107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 --2- 172.21.15.26:0/2139047872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8474103cb0 0x7f8474107cf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8468009990 tx=0x7f8468009ca0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 -- 172.21.15.26:0/2139047872 shutdown_connections 2023-10-06T21:17:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 --2- 172.21.15.26:0/2139047872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8474108420 0x7f847410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 --2- 172.21.15.26:0/2139047872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474103300 0x7f84741036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 --2- 172.21.15.26:0/2139047872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8474103cb0 0x7f8474107cf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 -- 172.21.15.26:0/2139047872 >> 172.21.15.26:0/2139047872 conn(0x7f84740feb20 msgr2=0x7f8474100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 -- 172.21.15.26:0/2139047872 shutdown_connections 2023-10-06T21:17:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.877+0000 7f84791ff700 1 -- 172.21.15.26:0/2139047872 wait complete. 2023-10-06T21:17:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 Processor -- start 2023-10-06T21:17:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 -- start start 2023-10-06T21:17:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8474103300 0x7f84741991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847419dd70 0x7f847419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.879+0000 7f84791ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419e7f0 con 0x7f8474108420 2023-10-06T21:17:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.880+0000 7f84791ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419e930 con 0x7f847419dd70 2023-10-06T21:17:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.880+0000 7f84791ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419eaa0 con 0x7f8474103300 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.881+0000 7f847259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.881+0000 7f847259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37270/0 (socket says 172.21.15.26:37270) 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.881+0000 7f847259c700 1 -- 172.21.15.26:0/2007222225 learned_addr learned my addr 172.21.15.26:0/2007222225 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.882+0000 7f847259c700 1 -- 172.21.15.26:0/2007222225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8474103300 msgr2=0x7f84741991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.882+0000 7f847259c700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8474103300 0x7f84741991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.882+0000 7f847259c700 1 -- 172.21.15.26:0/2007222225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847419dd70 msgr2=0x7f847419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.882+0000 7f847259c700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847419dd70 0x7f847419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.882+0000 7f847259c700 1 -- 172.21.15.26:0/2007222225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680095f0 con 0x7f8474108420 2023-10-06T21:17:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.883+0000 7f847259c700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8468000c00 tx=0x7f846800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.889+0000 7f845bfff700 1 -- 172.21.15.26:0/2007222225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846801d070 con 0x7f8474108420 2023-10-06T21:17:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.889+0000 7f845bfff700 1 -- 172.21.15.26:0/2007222225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8468005510 con 0x7f8474108420 2023-10-06T21:17:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.889+0000 7f845bfff700 1 -- 172.21.15.26:0/2007222225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468004bd0 con 0x7f8474108420 2023-10-06T21:17:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.889+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847406a8d0 con 0x7f8474108420 2023-10-06T21:17:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.889+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847406ad30 con 0x7f8474108420 2023-10-06T21:17:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.901+0000 7f845bfff700 1 -- 172.21.15.26:0/2007222225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8468004d30 con 0x7f8474108420 2023-10-06T21:17:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.907+0000 7f848cbf5b80 1 Processor -- start 2023-10-06T21:17:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.908+0000 7f845bfff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 0x7f845c0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.908+0000 7f845bfff700 1 -- 172.21.15.26:0/2007222225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f84680ce010 con 0x7f8474108420 2023-10-06T21:17:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 -- start start 2023-10-06T21:17:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f847359e700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 0x7f845c0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bbefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bcc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753b28400 con 0x558753bbf520 2023-10-06T21:17:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753b19bc0 con 0x558753bbec00 2023-10-06T21:17:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.910+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753b174e0 con 0x558753bca3d0 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.911+0000 7f847259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.911+0000 7f847259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37282/0 (socket says 172.21.15.26:37282) 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.911+0000 7f847259c700 1 -- 172.21.15.26:0/2726080153 learned_addr learned my addr 172.21.15.26:0/2726080153 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847359e700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 0x7f845c0a3c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f846c009710 tx=0x7f846c009ef0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847259c700 1 -- 172.21.15.26:0/2726080153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 msgr2=0x558753bcc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847259c700 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bcc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847259c700 1 -- 172.21.15.26:0/2726080153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 msgr2=0x558753bbefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847259c700 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bbefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.912+0000 7f847259c700 1 -- 172.21.15.26:0/2726080153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55875393dd80 con 0x558753bbf520 2023-10-06T21:17:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.920+0000 7f847259c700 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8468097510 tx=0x7f846809a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f735b72df32a40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.921+0000 7f84597fa700 1 -- 172.21.15.26:0/2726080153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846809fea0 con 0x558753bbf520 2023-10-06T21:17:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.921+0000 7f84597fa700 1 -- 172.21.15.26:0/2726080153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846809b010 con 0x558753bbf520 2023-10-06T21:17:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.921+0000 7f84597fa700 1 -- 172.21.15.26:0/2726080153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846809b270 con 0x558753bbf520 2023-10-06T21:17:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.925+0000 7f848cbf5b80 1 -- 172.21.15.26:0/2726080153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 msgr2=0x558753bc9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.925+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8468097510 tx=0x7f846809a970 comp rx=0 tx=0).stop 2023-10-06T21:17:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 -- 172.21.15.26:0/2726080153 shutdown_connections 2023-10-06T21:17:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bcc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bbefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/2726080153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bc9e90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 -- 172.21.15.26:0/2726080153 >> 172.21.15.26:0/2726080153 conn(0x558753b34200 msgr2=0x558753bbb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.926+0000 7f848cbf5b80 1 -- 172.21.15.26:0/2726080153 shutdown_connections 2023-10-06T21:17:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.927+0000 7f848cbf5b80 1 -- 172.21.15.26:0/2726080153 wait complete. 2023-10-06T21:17:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.928+0000 7f848cbf5b80 1 -- start start 2023-10-06T21:17:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.928+0000 7f848cbf5b80 1 Processor -- start 2023-10-06T21:17:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbf520 0x558753bd3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bde040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753b12dd0 con 0x558753bca3d0 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753aaf6b0 con 0x558753bbec00 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753aaf350 con 0x558753bbf520 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.929+0000 7f848cbf5b80 10 client.0 _ll_register_callbacks cb 0x55875390c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.930+0000 7f8472d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.930+0000 7f8472d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:50992/0 (socket says 172.21.15.26:50992) 2023-10-06T21:17:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.930+0000 7f8472d9d700 1 -- 172.21.15.26:0/1677178838 learned_addr learned my addr 172.21.15.26:0/1677178838 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.931+0000 7f8472d9d700 1 -- 172.21.15.26:0/1677178838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbf520 msgr2=0x558753bd3500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:17:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.931+0000 7f8472d9d700 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbf520 0x558753bd3500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.931+0000 7f8472d9d700 1 -- 172.21.15.26:0/1677178838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 msgr2=0x558753bde040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.931+0000 7f8472d9d700 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bde040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.931+0000 7f8472d9d700 1 -- 172.21.15.26:0/1677178838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847406ad30 con 0x558753bbec00 2023-10-06T21:17:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.932+0000 7f8472d9d700 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8464007ca0 tx=0x7f846400d5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235dd76dbe5df5ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.934+0000 7f8456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.937+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464007ea0 con 0x558753bbec00 2023-10-06T21:17:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.937+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f846400dde0 con 0x558753bbec00 2023-10-06T21:17:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.937+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558753bb2d50 con 0x558753bbec00 2023-10-06T21:17:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.938+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8464010460 con 0x558753bbec00 2023-10-06T21:17:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.940+0000 7f848cbf5b80 10 client.26465 fetch_fsmap learned FSMap version 42 2023-10-06T21:17:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.940+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558753926c00 con 0x558753bbec00 2023-10-06T21:17:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558753bb2d50 con 0x558753bbec00 2023-10-06T21:17:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f84640235d0 con 0x558753bbec00 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 20 client.26465 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753bce740 0x558753bd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 4 client.26465 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558753b0bdb0 con 0x558753bce740 2023-10-06T21:17:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.941+0000 7f847259c700 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753bce740 0x558753bd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.946+0000 7f847259c700 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753bce740 0x558753bd0b60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d1b6aaa71b6c4a server_cookie=6dd4b9379a46a972 in_seq=0 out_seq=0 2023-10-06T21:17:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.946+0000 7f8456ff5700 10 client.26465 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.947+0000 7f8456ff5700 1 -- 172.21.15.26:0/1677178838 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558753b0bdb0 con 0x558753bce740 2023-10-06T21:17:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.947+0000 7f8456ff5700 10 client.26465 handle_command_reply: tid=0 2023-10-06T21:17:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.947+0000 7f848cbf5b80 1 client.26465 shutdown 2023-10-06T21:17:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 msgr2=0x558753bd2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8464007ca0 tx=0x7f846400d5c0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 shutdown_connections 2023-10-06T21:17:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bde040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753bce740 0x558753bd0b60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd2fc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/1677178838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbf520 0x558753bd3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 >> 172.21.15.26:0/1677178838 conn(0x558753b34200 msgr2=0x558753bbac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.948+0000 7f8456ff5700 0 client.26465 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.949+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 shutdown_connections 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.949+0000 7f848cbf5b80 1 -- 172.21.15.26:0/1677178838 wait complete. 2023-10-06T21:17:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.949+0000 7f848cbf5b80 20 client.26465 trim_cache size 0 max 16384 2023-10-06T21:17:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 Processor -- start 2023-10-06T21:17:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 -- start start 2023-10-06T21:17:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbec00 0x558753bd8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.975+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587539938c0 con 0x558753bca3d0 2023-10-06T21:17:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753bc9ad0 con 0x558753bbf520 2023-10-06T21:17:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753bc9ce0 con 0x558753bbec00 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51000/0 (socket says 172.21.15.26:51000) 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 -- 172.21.15.26:0/3088170488 learned_addr learned my addr 172.21.15.26:0/3088170488 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847359e700 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f8472d9d700 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbec00 0x558753bd8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 -- 172.21.15.26:0/3088170488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbec00 msgr2=0x558753bd8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbec00 0x558753bd8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 -- 172.21.15.26:0/3088170488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 msgr2=0x558753bca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.976+0000 7f847259c700 1 -- 172.21.15.26:0/3088170488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587539641f0 con 0x558753bbf520 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.977+0000 7f847259c700 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8468005f50 tx=0x7f846800ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7b5dad4898fbebd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.977+0000 7f8459ffb700 1 -- 172.21.15.26:0/3088170488 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84680b7070 con 0x558753bbf520 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.977+0000 7f8459ffb700 1 -- 172.21.15.26:0/3088170488 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846800fbf0 con 0x558753bbf520 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.977+0000 7f8459ffb700 1 -- 172.21.15.26:0/3088170488 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846809a500 con 0x558753bbf520 2023-10-06T21:17:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 -- 172.21.15.26:0/3088170488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 msgr2=0x558753bba250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8468005f50 tx=0x7f846800ff10 comp rx=0 tx=0).stop 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 -- 172.21.15.26:0/3088170488 shutdown_connections 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bca3d0 0x558753bca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bbec00 0x558753bd8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/3088170488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbf520 0x558753bba250 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 -- 172.21.15.26:0/3088170488 >> 172.21.15.26:0/3088170488 conn(0x558753b34200 msgr2=0x558753bb97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 -- 172.21.15.26:0/3088170488 shutdown_connections 2023-10-06T21:17:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.978+0000 7f848cbf5b80 1 -- 172.21.15.26:0/3088170488 wait complete. 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.979+0000 7f848cbf5b80 1 -- start start 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.979+0000 7f848cbf5b80 1 Processor -- start 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bd64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bd69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753c3d650 con 0x558753bbf520 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753c3d790 con 0x558753bbec00 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558753c3d8d0 con 0x558753bca3d0 2023-10-06T21:17:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f848cbf5b80 10 client.0 _ll_register_callbacks cb 0x7f84640076f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f8472d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f847359e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bd69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f8472d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51004/0 (socket says 172.21.15.26:51004) 2023-10-06T21:17:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.980+0000 7f8472d9d700 1 -- 172.21.15.26:0/524744420 learned_addr learned my addr 172.21.15.26:0/524744420 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847259c700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bd64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 -- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 msgr2=0x558753bd5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 -- 172.21.15.26:0/524744420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 msgr2=0x558753bd64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bd64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 -- 172.21.15.26:0/524744420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55875393dd80 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847259c700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bd64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f847359e700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bd69f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x558753b17520 tx=0x7f846c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d69d51e48bdbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.981+0000 7f8456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c01d070 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f846c017a60 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f846809b010 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f846c017c20 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f848cbf5b80 10 client.26419 fetch_fsmap learned FSMap version 42 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.982+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55875393c250 con 0x558753bca3d0 2023-10-06T21:17:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846809b010 con 0x558753bca3d0 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f846c029070 con 0x558753bca3d0 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 10 client.26419 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 20 client.26419 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753c3f010 0x558753c41430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 4 client.26419 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558753b0bad0 con 0x558753c3f010 2023-10-06T21:17:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.983+0000 7f8472d9d700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753c3f010 0x558753c41430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.984+0000 7f8472d9d700 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753c3f010 0x558753c41430 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08977811fe6f4af server_cookie=81371f828f11334f in_seq=0 out_seq=0 2023-10-06T21:17:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:16.984+0000 7f8456ff5700 10 client.26419 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.001+0000 7f8456ff5700 1 -- 172.21.15.26:0/524744420 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558753b0bad0 con 0x558753c3f010 2023-10-06T21:17:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.001+0000 7f8456ff5700 10 client.26419 handle_command_reply: tid=0 2023-10-06T21:17:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.001+0000 7f848cbf5b80 1 client.26419 shutdown 2023-10-06T21:17:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.002+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 msgr2=0x558753bd69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.002+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bd69f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x558753b17520 tx=0x7f846c004ad0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 shutdown_connections 2023-10-06T21:17:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558753bca3d0 0x558753bd69f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558753c3f010 0x558753c41430 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558753bbec00 0x558753bd5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 --2- 172.21.15.26:0/524744420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558753bbf520 0x558753bd64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 >> 172.21.15.26:0/524744420 conn(0x558753b34200 msgr2=0x558753bc4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f8456ff5700 0 client.26419 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.003+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 shutdown_connections 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.004+0000 7f848cbf5b80 1 -- 172.21.15.26:0/524744420 wait complete. 2023-10-06T21:17:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.004+0000 7f848cbf5b80 20 client.26419 trim_cache size 0 max 16384 2023-10-06T21:17:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:17:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:17:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:17:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 msgr2=0x7f845c0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 0x7f845c0a3c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f846c009710 tx=0x7f846c009ef0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 msgr2=0x7f8474199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8468000c00 tx=0x7f846800f740 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 shutdown_connections 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f847419dd70 0x7f847419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8474103300 0x7f84741991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f845c0a17c0 0x7f845c0a3c80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 --2- 172.21.15.26:0/2007222225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8474108420 0x7f8474199700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 >> 172.21.15.26:0/2007222225 conn(0x7f84740feb20 msgr2=0x7f84740ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 shutdown_connections 2023-10-06T21:17:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:17.005+0000 7f84791ff700 1 -- 172.21.15.26:0/2007222225 wait complete. 2023-10-06T21:17:17.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 395.0 s, 1801.8 tps, lat 55.113 ms stddev 58.977 2023-10-06T21:17:18.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:18 smithi026 ceph-mon[98490]: pgmap v215: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:18 smithi078 ceph-mon[134896]: pgmap v215: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:18.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:18 smithi167 ceph-mon[134494]: pgmap v215: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:20.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:20 smithi026 ceph-mon[98490]: pgmap v216: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:20 smithi078 ceph-mon[134896]: pgmap v216: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:20 smithi167 ceph-mon[134494]: pgmap v216: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 734 op/s 2023-10-06T21:17:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 400.0 s, 2234.0 tps, lat 45.261 ms stddev 49.913 2023-10-06T21:17:22.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:22 smithi026 ceph-mon[98490]: pgmap v217: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 736 op/s 2023-10-06T21:17:22.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:22 smithi078 ceph-mon[134896]: pgmap v217: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 736 op/s 2023-10-06T21:17:22.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:22 smithi167 ceph-mon[134494]: pgmap v217: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 736 op/s 2023-10-06T21:17:24.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:24 smithi078 ceph-mon[134896]: pgmap v218: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-10-06T21:17:24.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:24 smithi167 ceph-mon[134494]: pgmap v218: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-10-06T21:17:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:24 smithi026 ceph-mon[98490]: pgmap v218: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 724 op/s 2023-10-06T21:17:26.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:26 smithi026 ceph-mon[98490]: pgmap v219: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 752 op/s 2023-10-06T21:17:26.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:26 smithi078 ceph-mon[134896]: pgmap v219: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 752 op/s 2023-10-06T21:17:26.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:26 smithi167 ceph-mon[134494]: pgmap v219: 97 pgs: 97 active+clean; 6.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 752 op/s 2023-10-06T21:17:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 405.0 s, 1729.8 tps, lat 57.040 ms stddev 104.937 2023-10-06T21:17:28.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:28 smithi026 ceph-mon[98490]: pgmap v220: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.79k op/s 2023-10-06T21:17:28.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:28 smithi078 ceph-mon[134896]: pgmap v220: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.79k op/s 2023-10-06T21:17:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:28 smithi167 ceph-mon[134494]: pgmap v220: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.79k op/s 2023-10-06T21:17:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 -- 172.21.15.26:0/2992557214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 msgr2=0x7feaf410fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 --2- 172.21.15.26:0/2992557214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf410fb20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feaec00b0a0 tx=0x7feaec00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 -- 172.21.15.26:0/2992557214 shutdown_connections 2023-10-06T21:17:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 --2- 172.21.15.26:0/2992557214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf410fb20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 --2- 172.21.15.26:0/2992557214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073280 0x7feaf4073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 --2- 172.21.15.26:0/2992557214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073ba0 0x7feaf410d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.243+0000 7feafa1ff700 1 -- 172.21.15.26:0/2992557214 >> 172.21.15.26:0/2992557214 conn(0x7feaf40fc940 msgr2=0x7feaf40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.244+0000 7feafa1ff700 1 -- 172.21.15.26:0/2992557214 shutdown_connections 2023-10-06T21:17:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.244+0000 7feafa1ff700 1 -- 172.21.15.26:0/2992557214 wait complete. 2023-10-06T21:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.246+0000 7feafa1ff700 1 Processor -- start 2023-10-06T21:17:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.246+0000 7feafa1ff700 1 -- start start 2023-10-06T21:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 0x7feaf41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 0x7feaf4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e4e0 con 0x7feaf4073280 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e650 con 0x7feaf410d6f0 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feafa1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e810 con 0x7feaf4073ba0 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56898/0 (socket says 172.21.15.26:56898) 2023-10-06T21:17:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf3fff700 1 -- 172.21.15.26:0/3922805000 learned_addr learned my addr 172.21.15.26:0/3922805000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf37fe700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 0x7feaf41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 0x7feaf4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.247+0000 7feaf3fff700 1 -- 172.21.15.26:0/3922805000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 msgr2=0x7feaf4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 0x7feaf4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf3fff700 1 -- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 msgr2=0x7feaf41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 0x7feaf41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf3fff700 1 -- 172.21.15.26:0/3922805000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009d00 con 0x7feaf410d6f0 2023-10-06T21:17:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf37fe700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 0x7feaf41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:17:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.248+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 0x7feaf4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:17:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.249+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feaec0062a0 tx=0x7feaec004b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.249+0000 7feaf0ff9700 1 -- 172.21.15.26:0/3922805000 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec021070 con 0x7feaf410d6f0 2023-10-06T21:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.249+0000 7feaf0ff9700 1 -- 172.21.15.26:0/3922805000 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaec004020 con 0x7feaf410d6f0 2023-10-06T21:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.249+0000 7feaf0ff9700 1 -- 172.21.15.26:0/3922805000 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00e040 con 0x7feaf410d6f0 2023-10-06T21:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.250+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf419ea90 con 0x7feaf410d6f0 2023-10-06T21:17:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.250+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf419ed10 con 0x7feaf410d6f0 2023-10-06T21:17:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.262+0000 7feb0dabeb80 1 Processor -- start 2023-10-06T21:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.266+0000 7feb0dabeb80 1 -- start start 2023-10-06T21:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0d30 0x55c9e84ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e8401400 con 0x55c9e84a0410 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e83f2bc0 con 0x55c9e84a0d30 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e83f04e0 con 0x55c9e84abbe0 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.267+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.268+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49902/0 (socket says 172.21.15.26:49902) 2023-10-06T21:17:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.268+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1325182125 learned_addr learned my addr 172.21.15.26:0/1325182125 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.269+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.270+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1325182125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0d30 msgr2=0x55c9e84ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.270+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0d30 0x55c9e84ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.270+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1325182125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 msgr2=0x55c9e84a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.270+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.270+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1325182125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e8216d80 con 0x55c9e84abbe0 2023-10-06T21:17:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.273+0000 7feaf0ff9700 1 -- 172.21.15.26:0/3922805000 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7feaec003a00 con 0x7feaf410d6f0 2023-10-06T21:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.275+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feae800cfd0 tx=0x7feae800c250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5fbf64089dbfe99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.275+0000 7feaf0ff9700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 0x7feadc0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.275+0000 7feaf0ff9700 1 -- 172.21.15.26:0/3922805000 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feaec0ce120 con 0x7feaf410d6f0 2023-10-06T21:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.282+0000 7feaf37fe700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 0x7feadc0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.283+0000 7feaf37fe700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 0x7feadc0a3bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feae4005ea0 tx=0x7feae400a2b0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.289+0000 7fead9ffb700 1 -- 172.21.15.26:0/1325182125 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae800c070 con 0x55c9e84abbe0 2023-10-06T21:17:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.289+0000 7fead9ffb700 1 -- 172.21.15.26:0/1325182125 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae8013ce0 con 0x55c9e84abbe0 2023-10-06T21:17:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.289+0000 7fead9ffb700 1 -- 172.21.15.26:0/1325182125 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80155f0 con 0x55c9e84abbe0 2023-10-06T21:17:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.290+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1325182125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 msgr2=0x55c9e84adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.290+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feae800cfd0 tx=0x7feae800c250 comp rx=0 tx=0).stop 2023-10-06T21:17:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.292+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1325182125 shutdown_connections 2023-10-06T21:17:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.292+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84adfd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.292+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.292+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1325182125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0d30 0x55c9e84ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.292+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1325182125 >> 172.21.15.26:0/1325182125 conn(0x55c9e840d200 msgr2=0x55c9e8493ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.293+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1325182125 shutdown_connections 2023-10-06T21:17:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.293+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1325182125 wait complete. 2023-10-06T21:17:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.294+0000 7feb0dabeb80 1 -- start start 2023-10-06T21:17:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.294+0000 7feb0dabeb80 1 Processor -- start 2023-10-06T21:17:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84abbe0 0x55c9e84a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e83ebdd0 con 0x55c9e84a0410 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e83886b0 con 0x55c9e84abbe0 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e8388350 con 0x55c9e84a0d30 2023-10-06T21:17:30.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.295+0000 7feb0dabeb80 10 client.0 _ll_register_callbacks cb 0x55c9e81e57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.296+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.297+0000 7feaf37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.297+0000 7feaf37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57674/0 (socket says 172.21.15.26:57674) 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.297+0000 7feaf37fe700 1 -- 172.21.15.26:0/1609962453 learned_addr learned my addr 172.21.15.26:0/1609962453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.297+0000 7feaf3fff700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84abbe0 0x55c9e84a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.298+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84abbe0 msgr2=0x55c9e84a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.298+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84abbe0 0x55c9e84a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.298+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1609962453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 msgr2=0x55c9e84a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.298+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.298+0000 7feaf2ffd700 1 -- 172.21.15.26:0/1609962453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae800ccb0 con 0x55c9e84a0d30 2023-10-06T21:17:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.299+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84a8eb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feae80048a0 tx=0x7feae800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20104ccf9c0506ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.299+0000 7fead2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:17:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.299+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8026070 con 0x55c9e84a0d30 2023-10-06T21:17:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.299+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feae8013e50 con 0x55c9e84a0d30 2023-10-06T21:17:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.300+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9e848bd50 con 0x55c9e84a0d30 2023-10-06T21:17:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.300+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7feae800c040 con 0x55c9e84a0d30 2023-10-06T21:17:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.300+0000 7feb0dabeb80 10 client.26425 fetch_fsmap learned FSMap version 42 2023-10-06T21:17:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.300+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c9e81ffc00 con 0x55c9e84a0d30 2023-10-06T21:17:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9e848bd50 con 0x55c9e84a0d30 2023-10-06T21:17:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7feae8029020 con 0x55c9e84a0d30 2023-10-06T21:17:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 10 client.26425 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T21:17:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 10 client.26425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 20 client.26425 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e84cc6d0 0x55c9e84ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 4 client.26425 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9e83e4db0 con 0x55c9e84cc6d0 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.302+0000 7feaf37fe700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e84cc6d0 0x55c9e84ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.303+0000 7feaf37fe700 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e84cc6d0 0x55c9e84ceaf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2522b77782bfc6 server_cookie=fb54837fe5bc56f0 in_seq=0 out_seq=0 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.303+0000 7fead2ffd700 10 client.26425 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.304+0000 7fead2ffd700 1 -- 172.21.15.26:0/1609962453 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9e83e4db0 con 0x55c9e84cc6d0 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.304+0000 7fead2ffd700 10 client.26425 handle_command_reply: tid=0 2023-10-06T21:17:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.304+0000 7feb0dabeb80 1 client.26425 shutdown 2023-10-06T21:17:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 msgr2=0x55c9e84a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84a8eb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feae80048a0 tx=0x7feae800c8f0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 shutdown_connections 2023-10-06T21:17:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e84cc6d0 0x55c9e84ceaf0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84abbe0 0x55c9e84a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0410 0x55c9e84a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/1609962453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84a8eb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.305+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 >> 172.21.15.26:0/1609962453 conn(0x55c9e840d200 msgr2=0x55c9e8493be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.306+0000 7fead2ffd700 0 client.26425 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.306+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 shutdown_connections 2023-10-06T21:17:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.306+0000 7feb0dabeb80 1 -- 172.21.15.26:0/1609962453 wait complete. 2023-10-06T21:17:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.306+0000 7feb0dabeb80 20 client.26425 trim_cache size 0 max 16384 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 Processor -- start 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 -- start start 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0410 0x55c9e84c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0d30 0x55c9e84a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e826c8c0 con 0x55c9e84a0d30 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e84908d0 con 0x55c9e84a0410 2023-10-06T21:17:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e84abaa0 con 0x55c9e84abbe0 2023-10-06T21:17:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.330+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49932/0 (socket says 172.21.15.26:49932) 2023-10-06T21:17:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 -- 172.21.15.26:0/3718869988 learned_addr learned my addr 172.21.15.26:0/3718869988 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf2ffd700 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0d30 0x55c9e84a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 -- 172.21.15.26:0/3718869988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0410 msgr2=0x55c9e84c3760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0410 0x55c9e84c3760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 -- 172.21.15.26:0/3718869988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0d30 msgr2=0x55c9e84a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0d30 0x55c9e84a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 -- 172.21.15.26:0/3718869988 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e84a8780 con 0x55c9e84abbe0 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.331+0000 7feaf3fff700 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaec0cb860 tx=0x7feaec09ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1739966f2ea0e435 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.332+0000 7feada7fc700 1 -- 172.21.15.26:0/3718869988 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec09b820 con 0x55c9e84abbe0 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.332+0000 7feada7fc700 1 -- 172.21.15.26:0/3718869988 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaec09b120 con 0x55c9e84abbe0 2023-10-06T21:17:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.332+0000 7feada7fc700 1 -- 172.21.15.26:0/3718869988 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec099550 con 0x55c9e84abbe0 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 -- 172.21.15.26:0/3718869988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 msgr2=0x55c9e84a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaec0cb860 tx=0x7feaec09ac60 comp rx=0 tx=0).stop 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 -- 172.21.15.26:0/3718869988 shutdown_connections 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84abbe0 0x55c9e84a5c70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84a0410 0x55c9e84c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/3718869988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84a0d30 0x55c9e84a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.333+0000 7feb0dabeb80 1 -- 172.21.15.26:0/3718869988 >> 172.21.15.26:0/3718869988 conn(0x55c9e840d200 msgr2=0x55c9e84a3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.334+0000 7feb0dabeb80 1 -- 172.21.15.26:0/3718869988 shutdown_connections 2023-10-06T21:17:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.334+0000 7feb0dabeb80 1 -- 172.21.15.26:0/3718869988 wait complete. 2023-10-06T21:17:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.335+0000 7feb0dabeb80 1 -- start start 2023-10-06T21:17:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.335+0000 7feb0dabeb80 1 Processor -- start 2023-10-06T21:17:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84abbe0 0x55c9e84c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e84c3cd0 con 0x55c9e84abbe0 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e851fa00 con 0x55c9e84c21d0 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e851fb40 con 0x55c9e84a0d30 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feb0dabeb80 10 client.0 _ll_register_callbacks cb 0x7feae800ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:56944/0 (socket says 172.21.15.26:56944) 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 -- 172.21.15.26:0/418634965 learned_addr learned my addr 172.21.15.26:0/418634965 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf37fe700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 -- 172.21.15.26:0/418634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 msgr2=0x55c9e84c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 -- 172.21.15.26:0/418634965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84abbe0 msgr2=0x55c9e84c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84abbe0 0x55c9e84c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.336+0000 7feaf3fff700 1 -- 172.21.15.26:0/418634965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf419ea90 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7feaf3fff700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feaec009fd0 tx=0x7feaec003fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7acebeca4b668c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7fead2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0b7070 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feaec0a3420 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feae8013ce0 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaec09b820 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.337+0000 7feb0dabeb80 10 client.26483 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c9e823d1f0 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feae8013ce0 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feaec084360 con 0x55c9e84c21d0 2023-10-06T21:17:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 10 client.26483 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 20 client.26483 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e8520d80 0x55c9e8523170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 4 client.26483 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9e83e4ad0 con 0x55c9e8520d80 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.338+0000 7feaf37fe700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e8520d80 0x55c9e8523170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.339+0000 7feaf37fe700 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e8520d80 0x55c9e8523170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14859ed84ee90ac9 server_cookie=e557b6ffae0361bd in_seq=0 out_seq=0 2023-10-06T21:17:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.339+0000 7fead2ffd700 10 client.26483 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.353+0000 7fead2ffd700 1 -- 172.21.15.26:0/418634965 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9e83e4ad0 con 0x55c9e8520d80 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.353+0000 7fead2ffd700 10 client.26483 handle_command_reply: tid=0 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.353+0000 7feb0dabeb80 1 client.26483 shutdown 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 msgr2=0x55c9e851f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feaec009fd0 tx=0x7feaec003fb0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 shutdown_connections 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c9e8520d80 0x55c9e8523170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9e84c21d0 0x55c9e851f4c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9e84a0d30 0x55c9e84c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 --2- 172.21.15.26:0/418634965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9e84abbe0 0x55c9e84c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 >> 172.21.15.26:0/418634965 conn(0x55c9e840d200 msgr2=0x55c9e84a8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7fead2ffd700 0 client.26483 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 shutdown_connections 2023-10-06T21:17:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.354+0000 7feb0dabeb80 1 -- 172.21.15.26:0/418634965 wait complete. 2023-10-06T21:17:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.355+0000 7feb0dabeb80 20 client.26483 trim_cache size 0 max 16384 2023-10-06T21:17:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:17:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:17:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:17:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 msgr2=0x7feadc0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 0x7feadc0a3bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feae4005ea0 tx=0x7feae400a2b0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 msgr2=0x7feaf419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feaec0062a0 tx=0x7feaec004b00 comp rx=0 tx=0).stop 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 shutdown_connections 2023-10-06T21:17:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaf410d6f0 0x7feaf419dea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7feadc0a1730 0x7feadc0a3bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feaf4073280 0x7feaf41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 --2- 172.21.15.26:0/3922805000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feaf4073ba0 0x7feaf4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 >> 172.21.15.26:0/3922805000 conn(0x7feaf40fc940 msgr2=0x7feaf4105150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 shutdown_connections 2023-10-06T21:17:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:30.356+0000 7feafa1ff700 1 -- 172.21.15.26:0/3922805000 wait complete. 2023-10-06T21:17:30.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:30 smithi026 ceph-mon[98490]: pgmap v221: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-10-06T21:17:30.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:30 smithi078 ceph-mon[134896]: pgmap v221: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-10-06T21:17:30.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:30 smithi167 ceph-mon[134494]: pgmap v221: 97 pgs: 97 active+clean; 6.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-10-06T21:17:31.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:31 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:31.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:31 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:31 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:17:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 410.0 s, 1213.6 tps, lat 81.923 ms stddev 98.356 2023-10-06T21:17:32.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 -- 172.21.15.26:0/2816561221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c100710 msgr2=0x7f6b3c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 --2- 172.21.15.26:0/2816561221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c100710 0x7f6b3c104bd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009990 tx=0x7f6b2c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 -- 172.21.15.26:0/2816561221 shutdown_connections 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 --2- 172.21.15.26:0/2816561221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 --2- 172.21.15.26:0/2816561221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c0ffdf0 0x7f6b3c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 --2- 172.21.15.26:0/2816561221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c100710 0x7f6b3c104bd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 -- 172.21.15.26:0/2816561221 >> 172.21.15.26:0/2816561221 conn(0x7f6b3c0fb820 msgr2=0x7f6b3c0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.541+0000 7f6b415ff700 1 -- 172.21.15.26:0/2816561221 shutdown_connections 2023-10-06T21:17:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.542+0000 7f6b415ff700 1 -- 172.21.15.26:0/2816561221 wait complete. 2023-10-06T21:17:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.543+0000 7f6b415ff700 1 Processor -- start 2023-10-06T21:17:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.543+0000 7f6b415ff700 1 -- start start 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c0ffdf0 0x7f6b3c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a2640 con 0x7f6b3c100710 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a27b0 con 0x7f6b3c1051f0 2023-10-06T21:17:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.544+0000 7f6b415ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c1a2970 con 0x7f6b3c0ffdf0 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48506/0 (socket says 172.21.15.26:48506) 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3a7fc700 1 -- 172.21.15.26:0/1259507603 learned_addr learned my addr 172.21.15.26:0/1259507603 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 -- 172.21.15.26:0/1259507603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c0ffdf0 msgr2=0x7f6b3c19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c0ffdf0 0x7f6b3c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 -- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 msgr2=0x7f6b3c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 -- 172.21.15.26:0/1259507603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0095f0 con 0x7f6b3c1051f0 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.545+0000 7f6b3b7fe700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1a2000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400efc0 tx=0x7f6b3400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.546+0000 7f6b23fff700 1 -- 172.21.15.26:0/1259507603 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34018070 con 0x7f6b3c1051f0 2023-10-06T21:17:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.546+0000 7f6b23fff700 1 -- 172.21.15.26:0/1259507603 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b3400f040 con 0x7f6b3c1051f0 2023-10-06T21:17:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.546+0000 7f6b23fff700 1 -- 172.21.15.26:0/1259507603 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400cdf0 con 0x7f6b3c1051f0 2023-10-06T21:17:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.546+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c1a2bf0 con 0x7f6b3c1051f0 2023-10-06T21:17:32.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.546+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c1aa490 con 0x7f6b3c1051f0 2023-10-06T21:17:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.550+0000 7f6b23fff700 1 -- 172.21.15.26:0/1259507603 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6b34003a00 con 0x7f6b3c1051f0 2023-10-06T21:17:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.551+0000 7f6b54ee5b80 1 Processor -- start 2023-10-06T21:17:32.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.551+0000 7f6b54ee5b80 1 -- start start 2023-10-06T21:17:32.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a2d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a38890 0x55b6f2a3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2996460 con 0x55b6f2a38890 2023-10-06T21:17:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2987c20 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2985540 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43592/0 (socket says 172.21.15.26:43592) 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b3affd700 1 -- 172.21.15.26:0/1886324460 learned_addr learned my addr 172.21.15.26:0/1886324460 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a2d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b23fff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 0x7f6b240a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a38890 0x55b6f2a3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.552+0000 7f6b23fff700 1 -- 172.21.15.26:0/1259507603 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b340ccb50 con 0x7f6b3c1051f0 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 -- 172.21.15.26:0/1886324460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d0c0 msgr2=0x55b6f2a2d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a2d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 -- 172.21.15.26:0/1886324460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a38890 msgr2=0x55b6f2a3ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a38890 0x55b6f2a3ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 -- 172.21.15.26:0/1886324460 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f27abde0 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3b7fe700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 0x7f6b240a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.553+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098a0 tx=0x7f6b30009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa8c6e02c8e72901 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.554+0000 7f6b217fa700 1 -- 172.21.15.26:0/1886324460 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3000b870 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.554+0000 7f6b217fa700 1 -- 172.21.15.26:0/1886324460 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b3000be90 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.554+0000 7f6b217fa700 1 -- 172.21.15.26:0/1886324460 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30010380 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1886324460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 msgr2=0x55b6f2a38350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6b300098a0 tx=0x7f6b30009bb0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1886324460 shutdown_connections 2023-10-06T21:17:32.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a38890 0x55b6f2a3ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a2d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1886324460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a38350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.555+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1886324460 >> 172.21.15.26:0/1886324460 conn(0x55b6f29a2260 msgr2=0x55b6f2a29450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.556+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1886324460 shutdown_connections 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.556+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1886324460 wait complete. 2023-10-06T21:17:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.557+0000 7f6b54ee5b80 1 -- start start 2023-10-06T21:17:32.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.557+0000 7f6b54ee5b80 1 Processor -- start 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.558+0000 7f6b3b7fe700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 0x7f6b240a3b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b340ca290 tx=0x7f6b34004cc0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a35af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a36030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2980e30 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f291d710 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f291d3b0 con 0x55b6f2a38890 2023-10-06T21:17:32.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.559+0000 7f6b54ee5b80 10 client.0 _ll_register_callbacks cb 0x55b6f277a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:32.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a35af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a35af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37852/0 (socket says 172.21.15.26:37852) 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3affd700 1 -- 172.21.15.26:0/4205762074 learned_addr learned my addr 172.21.15.26:0/4205762074 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a36030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 -- 172.21.15.26:0/4205762074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 msgr2=0x55b6f2a36030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a36030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 -- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 msgr2=0x55b6f2a35af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a35af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.560+0000 7f6b3a7fc700 1 -- 172.21.15.26:0/4205762074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b300094d0 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a355b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c19ec70 tx=0x7f6b2c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db9a4aef6f456b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:17:32.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00fd80 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6f2a20f10 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b2c02a3f0 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b2c02a5b0 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b54ee5b80 10 client.16245 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.561+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b6f2794c60 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b2c037020 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b2c0293e0 con 0x55b6f2a2d0c0 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 20 client.16245 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a483d0 0x55b6f2a4a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 4 client.16245 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:32.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.562+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6f2979e10 con 0x55b6f2a483d0 2023-10-06T21:17:32.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.563+0000 7f6b3affd700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a483d0 0x55b6f2a4a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.564+0000 7f6b3affd700 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a483d0 0x55b6f2a4a7f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e4a5a23cdddaf0 server_cookie=c084311fcb5ca791 in_seq=0 out_seq=0 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.564+0000 7f6b1eff5700 10 client.16245 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.564+0000 7f6b1eff5700 1 -- 172.21.15.26:0/4205762074 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6f2979e10 con 0x55b6f2a483d0 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.564+0000 7f6b1eff5700 10 client.16245 handle_command_reply: tid=0 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.564+0000 7f6b54ee5b80 1 client.16245 shutdown 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 msgr2=0x55b6f2a355b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a355b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c19ec70 tx=0x7f6b2c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 shutdown_connections 2023-10-06T21:17:32.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a36030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d0c0 0x55b6f2a355b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a483d0 0x55b6f2a4a7f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/4205762074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a35af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b1eff5700 0 client.16245 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 >> 172.21.15.26:0/4205762074 conn(0x55b6f29a2260 msgr2=0x55b6f2a28fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 shutdown_connections 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/4205762074 wait complete. 2023-10-06T21:17:32.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.565+0000 7f6b54ee5b80 20 client.16245 trim_cache size 0 max 16384 2023-10-06T21:17:32.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:32 smithi026 ceph-mon[98490]: pgmap v222: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2023-10-06T21:17:32.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.591+0000 7f6b54ee5b80 1 Processor -- start 2023-10-06T21:17:32.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.591+0000 7f6b54ee5b80 1 -- start start 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a29670 0x55b6f2a32dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2801920 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2a28240 con 0x55b6f2a38890 2023-10-06T21:17:32.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2a3f910 con 0x55b6f2a29670 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37860/0 (socket says 172.21.15.26:37860) 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 -- 172.21.15.26:0/2052089421 learned_addr learned my addr 172.21.15.26:0/2052089421 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a29670 0x55b6f2a32dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 -- 172.21.15.26:0/2052089421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a29670 msgr2=0x55b6f2a32dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a29670 0x55b6f2a32dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 -- 172.21.15.26:0/2052089421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 msgr2=0x55b6f2a3f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3affd700 1 -- 172.21.15.26:0/2052089421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f27d2250 con 0x55b6f2a38890 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.592+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.593+0000 7f6b3affd700 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b30004910 tx=0x7f6b30009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e859e14da849895f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.593+0000 7f6b21ffb700 1 -- 172.21.15.26:0/2052089421 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30020040 con 0x55b6f2a38890 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.593+0000 7f6b21ffb700 1 -- 172.21.15.26:0/2052089421 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b300245b0 con 0x55b6f2a38890 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.593+0000 7f6b21ffb700 1 -- 172.21.15.26:0/2052089421 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30024730 con 0x55b6f2a38890 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/2052089421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 msgr2=0x55b6f2a32880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b30004910 tx=0x7f6b30009460 comp rx=0 tx=0).stop 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/2052089421 shutdown_connections 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a29670 0x55b6f2a32dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/2052089421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a38890 0x55b6f2a32880 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/2052089421 >> 172.21.15.26:0/2052089421 conn(0x55b6f29a2260 msgr2=0x55b6f2a308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/2052089421 shutdown_connections 2023-10-06T21:17:32.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/2052089421 wait complete. 2023-10-06T21:17:32.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.594+0000 7f6b54ee5b80 1 -- start start 2023-10-06T21:17:32.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 Processor -- start 2023-10-06T21:17:32.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a29670 0x55b6f2a3da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a9c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2a3e690 con 0x55b6f2a29670 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2a3e800 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f2a9c710 con 0x55b6f2a38890 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.595+0000 7f6b54ee5b80 10 client.0 _ll_register_callbacks cb 0x7f6b2c00f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37866/0 (socket says 172.21.15.26:37866) 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 -- 172.21.15.26:0/1848786333 learned_addr learned my addr 172.21.15.26:0/1848786333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a29670 0x55b6f2a3da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a9c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 -- 172.21.15.26:0/1848786333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 msgr2=0x55b6f2a9c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a9c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 -- 172.21.15.26:0/1848786333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a29670 msgr2=0x55b6f2a3da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a29670 0x55b6f2a3da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 -- 172.21.15.26:0/1848786333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6f27abde0 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a9c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b3affd700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b30009270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eead1781913a4e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.596+0000 7f6b1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.597+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30035070 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.597+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b30011cd0 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.597+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6f2979b30 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.598+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b30020050 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.598+0000 7f6b54ee5b80 10 client.26501 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:32.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.598+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b6f27aa2b0 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6f2979b30 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b30038030 con 0x55b6f2a2d9e0 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 10 client.26501 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 20 client.26501 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a9dab0 0x55b6f2a9fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 4 client.26501 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.599+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b6f27ccf70 con 0x55b6f2a9dab0 2023-10-06T21:17:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.600+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a9dab0 0x55b6f2a9fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.601+0000 7f6b3a7fc700 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a9dab0 0x55b6f2a9fea0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307a6da3a97f10c9 server_cookie=a3a113be39aca3f4 in_seq=0 out_seq=0 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.601+0000 7f6b1eff5700 10 client.26501 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.602+0000 7f6b1eff5700 1 -- 172.21.15.26:0/1848786333 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b6f27ccf70 con 0x55b6f2a9dab0 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.602+0000 7f6b1eff5700 10 client.26501 handle_command_reply: tid=0 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.602+0000 7f6b54ee5b80 1 client.26501 shutdown 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.602+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 msgr2=0x55b6f2a3df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.602+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b30009270 comp rx=0 tx=0).stop 2023-10-06T21:17:32.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 shutdown_connections 2023-10-06T21:17:32.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6f2a38890 0x55b6f2a9c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6f2a29670 0x55b6f2a3da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6f2a9dab0 0x55b6f2a9fea0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 --2- 172.21.15.26:0/1848786333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6f2a2d9e0 0x55b6f2a3df40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 >> 172.21.15.26:0/1848786333 conn(0x55b6f29a2260 msgr2=0x55b6f2a28660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.603+0000 7f6b1eff5700 0 client.26501 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.604+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 shutdown_connections 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.604+0000 7f6b54ee5b80 1 -- 172.21.15.26:0/1848786333 wait complete. 2023-10-06T21:17:32.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.604+0000 7f6b54ee5b80 20 client.26501 trim_cache size 0 max 16384 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:17:32.608 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:17:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 msgr2=0x7f6b240a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 0x7f6b240a3b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b340ca290 tx=0x7f6b34004cc0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 msgr2=0x7f6b3c1a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1a2000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400efc0 tx=0x7f6b3400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 shutdown_connections 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c1051f0 0x7f6b3c1a2000 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c0ffdf0 0x7f6b3c19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b240a1670 0x7f6b240a3b30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 --2- 172.21.15.26:0/1259507603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b3c100710 0x7f6b3c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 >> 172.21.15.26:0/1259507603 conn(0x7f6b3c0fb820 msgr2=0x7f6b3c102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 shutdown_connections 2023-10-06T21:17:32.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:17:32.605+0000 7f6b415ff700 1 -- 172.21.15.26:0/1259507603 wait complete. 2023-10-06T21:17:32.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:32 smithi078 ceph-mon[134896]: pgmap v222: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2023-10-06T21:17:32.633 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:17:32.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:32 smithi167 ceph-mon[134494]: pgmap v222: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.07k op/s 2023-10-06T21:17:34.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:34 smithi026 ceph-mon[98490]: pgmap v223: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-10-06T21:17:34.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:34 smithi078 ceph-mon[134896]: pgmap v223: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-10-06T21:17:34.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:34 smithi167 ceph-mon[134494]: pgmap v223: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 1.80k op/s 2023-10-06T21:17:36.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:36 smithi026 ceph-mon[98490]: pgmap v224: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-06T21:17:36.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:36 smithi078 ceph-mon[134896]: pgmap v224: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-06T21:17:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:36 smithi167 ceph-mon[134494]: pgmap v224: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 1.83k op/s 2023-10-06T21:17:37.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 415.0 s, 1399.5 tps, lat 71.978 ms stddev 84.775 2023-10-06T21:17:38.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:38 smithi026 ceph-mon[98490]: pgmap v225: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.00k op/s 2023-10-06T21:17:38.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:38 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:38.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:38 smithi078 ceph-mon[134896]: pgmap v225: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.00k op/s 2023-10-06T21:17:38.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:38 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:38.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:38 smithi167 ceph-mon[134494]: pgmap v225: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 2.00k op/s 2023-10-06T21:17:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:38 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:17:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:40 smithi026 ceph-mon[98490]: pgmap v226: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 802 op/s 2023-10-06T21:17:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:40 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:40 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:40.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:40 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:17:40.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:40 smithi078 ceph-mon[134896]: pgmap v226: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 802 op/s 2023-10-06T21:17:40.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:40 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:40.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:40 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:40.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:40 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:17:40.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:40 smithi167 ceph-mon[134494]: pgmap v226: 97 pgs: 97 active+clean; 6.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 802 op/s 2023-10-06T21:17:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:40 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:17:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:40 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:17:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:40 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:17:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 420.0 s, 1575.3 tps, lat 63.255 ms stddev 71.535 2023-10-06T21:17:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:42 smithi078 ceph-mon[134896]: pgmap v227: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2023-10-06T21:17:42.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:42 smithi167 ceph-mon[134494]: pgmap v227: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2023-10-06T21:17:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:42 smithi026 ceph-mon[98490]: pgmap v227: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2023-10-06T21:17:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 -- 172.21.15.26:0/3056293236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4107840 msgr2=0x7f27e4107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 --2- 172.21.15.26:0/3056293236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4107840 0x7f27e4107c20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0099c0 tx=0x7f27cc009cd0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 -- 172.21.15.26:0/3056293236 shutdown_connections 2023-10-06T21:17:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 --2- 172.21.15.26:0/3056293236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e410c880 0x7f27e4114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 --2- 172.21.15.26:0/3056293236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e4107840 0x7f27e4107c20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 --2- 172.21.15.26:0/3056293236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e41081f0 0x7f27e410c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.583+0000 7f27e93ff700 1 -- 172.21.15.26:0/3056293236 >> 172.21.15.26:0/3056293236 conn(0x7f27e4076d50 msgr2=0x7f27e4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.584+0000 7f27e93ff700 1 -- 172.21.15.26:0/3056293236 shutdown_connections 2023-10-06T21:17:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.584+0000 7f27e93ff700 1 -- 172.21.15.26:0/3056293236 wait complete. 2023-10-06T21:17:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.585+0000 7f27e93ff700 1 Processor -- start 2023-10-06T21:17:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 -- start start 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 0x7f27e407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 0x7f27e407bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41b2660 con 0x7f27e4107840 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41b27d0 con 0x7f27e410c880 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e93ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41b2990 con 0x7f27e407baf0 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27da7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 0x7f27e407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43964/0 (socket says 172.21.15.26:43964) 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.586+0000 7f27e2ffd700 1 -- 172.21.15.26:0/93339873 learned_addr learned my addr 172.21.15.26:0/93339873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e37fe700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 0x7f27e407bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 -- 172.21.15.26:0/93339873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 msgr2=0x7f27e407bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 0x7f27e407bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 -- 172.21.15.26:0/93339873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 msgr2=0x7f27e407b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 0x7f27e407b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 -- 172.21.15.26:0/93339873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0095f0 con 0x7f27e4107840 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e37fe700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 0x7f27e407bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27da7fc700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 0x7f27e407b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:17:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.587+0000 7f27e2ffd700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f27cc003760 tx=0x7f27cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.591+0000 7f27e0ff9700 1 -- 172.21.15.26:0/93339873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc01d070 con 0x7f27e4107840 2023-10-06T21:17:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.591+0000 7f27e0ff9700 1 -- 172.21.15.26:0/93339873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27cc005da0 con 0x7f27e4107840 2023-10-06T21:17:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.591+0000 7f27e0ff9700 1 -- 172.21.15.26:0/93339873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc004a40 con 0x7f27e4107840 2023-10-06T21:17:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.591+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41b2c10 con 0x7f27e4107840 2023-10-06T21:17:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.591+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41b3040 con 0x7f27e4107840 2023-10-06T21:17:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.598+0000 7f27e0ff9700 1 -- 172.21.15.26:0/93339873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f27cc005250 con 0x7f27e4107840 2023-10-06T21:17:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.599+0000 7f27fcc9ab80 1 Processor -- start 2023-10-06T21:17:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.602+0000 7f27fcc9ab80 1 -- start start 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18450400 con 0x557d184e7520 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18441bc0 con 0x557d184e6c00 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1843f4e0 con 0x557d184f23d0 2023-10-06T21:17:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40742/0 (socket says 172.21.15.26:40742) 2023-10-06T21:17:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27e37fe700 1 -- 172.21.15.26:0/4271459236 learned_addr learned my addr 172.21.15.26:0/4271459236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.603+0000 7f27da7fc700 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 -- 172.21.15.26:0/4271459236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 msgr2=0x557d184f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 -- 172.21.15.26:0/4271459236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 msgr2=0x557d184f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 -- 172.21.15.26:0/4271459236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d18265d80 con 0x557d184e6c00 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e0ff9700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 0x7f27d00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e37fe700 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009d30 tx=0x7f27dc0106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36d4881e9b492e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e0ff9700 1 -- 172.21.15.26:0/93339873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f27cc0cdd30 con 0x7f27e4107840 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27e2ffd700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 0x7f27d00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27d97fa700 1 -- 172.21.15.26:0/4271459236 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc010c10 con 0x557d184e6c00 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27d97fa700 1 -- 172.21.15.26:0/4271459236 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc015070 con 0x557d184e6c00 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.604+0000 7f27d97fa700 1 -- 172.21.15.26:0/4271459236 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011680 con 0x557d184e6c00 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.605+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/4271459236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 msgr2=0x557d184e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.605+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009d30 tx=0x7f27dc0106b0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/4271459236 shutdown_connections 2023-10-06T21:17:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184e6fe0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/4271459236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/4271459236 >> 172.21.15.26:0/4271459236 conn(0x557d1845c200 msgr2=0x557d184e3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27e2ffd700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 0x7f27d00a3c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09aa90 tx=0x7f27cc009310 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/4271459236 shutdown_connections 2023-10-06T21:17:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.606+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/4271459236 wait complete. 2023-10-06T21:17:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.607+0000 7f27fcc9ab80 1 -- start start 2023-10-06T21:17:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.607+0000 7f27fcc9ab80 1 Processor -- start 2023-10-06T21:17:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184ef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1843add0 con 0x557d184e7520 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d183d76b0 con 0x557d184e6c00 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d183d7350 con 0x557d184f23d0 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27fcc9ab80 10 client.0 _ll_register_callbacks cb 0x557d182347a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40756/0 (socket says 172.21.15.26:40756) 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27e37fe700 1 -- 172.21.15.26:0/2514800673 learned_addr learned my addr 172.21.15.26:0/2514800673 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.608+0000 7f27e37fe700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 -- 172.21.15.26:0/2514800673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 msgr2=0x557d184ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 -- 172.21.15.26:0/2514800673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 msgr2=0x557d184ef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184ef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 -- 172.21.15.26:0/2514800673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc009960 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27e37fe700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27dc010ed0 tx=0x7f27dc01cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4156e30cd94e50bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc01c890 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f27dc017070 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.609+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d184dad50 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc01c9f0 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 10 client.26513 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557d1824ec00 con 0x557d184e6c00 2023-10-06T21:17:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d184dad50 con 0x557d184e6c00 2023-10-06T21:17:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27dc004a20 con 0x557d184e6c00 2023-10-06T21:17:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 10 client.26513 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 10 client.26513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 20 client.26513 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18501dc0 0x557d185041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 4 client.26513 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.610+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d18433db0 con 0x557d18501dc0 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.611+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18501dc0 0x557d185041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.611+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18501dc0 0x557d185041e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be8c22caca5864a server_cookie=821b384189aba26 in_seq=0 out_seq=0 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.611+0000 7f27ca7fc700 10 client.26513 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.612+0000 7f27ca7fc700 1 -- 172.21.15.26:0/2514800673 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d18433db0 con 0x557d18501dc0 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.612+0000 7f27ca7fc700 10 client.26513 handle_command_reply: tid=0 2023-10-06T21:17:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.612+0000 7f27fcc9ab80 1 client.26513 shutdown 2023-10-06T21:17:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.614+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 msgr2=0x557d184f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.614+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27dc010ed0 tx=0x7f27dc01cde0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 shutdown_connections 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18501dc0 0x557d185041e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f1df0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2514800673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184ef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 >> 172.21.15.26:0/2514800673 conn(0x557d1845c200 msgr2=0x557d184e2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27ca7fc700 0 client.26513 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 shutdown_connections 2023-10-06T21:17:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.615+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2514800673 wait complete. 2023-10-06T21:17:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.616+0000 7f27fcc9ab80 20 client.26513 trim_cache size 0 max 16384 2023-10-06T21:17:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.636+0000 7f27fcc9ab80 1 Processor -- start 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.636+0000 7f27fcc9ab80 1 -- start start 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d182bb8c0 con 0x557d184e7520 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d184df860 con 0x557d184e6c00 2023-10-06T21:17:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d184e3dc0 con 0x557d184f23d0 2023-10-06T21:17:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40770/0 (socket says 172.21.15.26:40770) 2023-10-06T21:17:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.637+0000 7f27e37fe700 1 -- 172.21.15.26:0/2673229453 learned_addr learned my addr 172.21.15.26:0/2673229453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.638+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 -- 172.21.15.26:0/2673229453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 msgr2=0x557d184ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 -- 172.21.15.26:0/2673229453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 msgr2=0x557d184ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 -- 172.21.15.26:0/2673229453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d1828c1f0 con 0x557d184e7520 2023-10-06T21:17:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.639+0000 7f27da7fc700 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x557d184f0620 tx=0x7f27d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905fdb9e4919f6f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.640+0000 7f27d9ffb700 1 -- 172.21.15.26:0/2673229453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d401e550 con 0x557d184e7520 2023-10-06T21:17:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.640+0000 7f27d9ffb700 1 -- 172.21.15.26:0/2673229453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d400f040 con 0x557d184e7520 2023-10-06T21:17:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.640+0000 7f27d9ffb700 1 -- 172.21.15.26:0/2673229453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400e750 con 0x557d184e7520 2023-10-06T21:17:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2673229453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 msgr2=0x557d184f1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x557d184f0620 tx=0x7f27d400b040 comp rx=0 tx=0).stop 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2673229453 shutdown_connections 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/2673229453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f1990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2673229453 >> 172.21.15.26:0/2673229453 conn(0x557d1845c200 msgr2=0x557d184fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.641+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2673229453 shutdown_connections 2023-10-06T21:17:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.642+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/2673229453 wait complete. 2023-10-06T21:17:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.642+0000 7f27fcc9ab80 1 -- start start 2023-10-06T21:17:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.643+0000 7f27fcc9ab80 1 Processor -- start 2023-10-06T21:17:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d184f7a80 con 0x557d184e7520 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d184f7bf0 con 0x557d184e6c00 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d184f7db0 con 0x557d184f23d0 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27fcc9ab80 10 client.0 _ll_register_callbacks cb 0x7f27dc010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27e37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40778/0 (socket says 172.21.15.26:40778) 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27e37fe700 1 -- 172.21.15.26:0/1768794001 learned_addr learned my addr 172.21.15.26:0/1768794001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27da7fc700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.644+0000 7f27e37fe700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 -- 172.21.15.26:0/1768794001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 msgr2=0x557d184f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 -- 172.21.15.26:0/1768794001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 msgr2=0x557d184f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 -- 172.21.15.26:0/1768794001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d40091e0 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27e37fe700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009c30 tx=0x7f27dc01cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38dade33eef994a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc01cb60 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f27dc017040 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27dc015070 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc0117f0 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.645+0000 7f27fcc9ab80 10 client.26519 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557d18264250 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc015070 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27dc005460 con 0x557d184e6c00 2023-10-06T21:17:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 10 client.26519 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 10 client.26519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 20 client.26519 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18556b80 0x557d18558f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 4 client.26519 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d18433ad0 con 0x557d18556b80 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.646+0000 7f27da7fc700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18556b80 0x557d18558f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.647+0000 7f27ca7fc700 10 client.26519 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.647+0000 7f27da7fc700 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18556b80 0x557d18558f70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c15b0cc77e6c46 server_cookie=930aa318f28b5299 in_seq=0 out_seq=0 2023-10-06T21:17:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.665+0000 7f27ca7fc700 1 -- 172.21.15.26:0/1768794001 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d18433ad0 con 0x557d18556b80 2023-10-06T21:17:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.665+0000 7f27ca7fc700 10 client.26519 handle_command_reply: tid=0 2023-10-06T21:17:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.665+0000 7f27fcc9ab80 1 client.26519 shutdown 2023-10-06T21:17:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.666+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 msgr2=0x557d184f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.666+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009c30 tx=0x7f27dc01cd40 comp rx=0 tx=0).stop 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 shutdown_connections 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d184f23d0 0x557d184f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557d18556b80 0x557d18558f70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d184e6c00 0x557d184f8cd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 --2- 172.21.15.26:0/1768794001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d184e7520 0x557d184f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27ca7fc700 0 client.26519 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 >> 172.21.15.26:0/1768794001 conn(0x557d1845c200 msgr2=0x557d184ea3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 shutdown_connections 2023-10-06T21:17:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 1 -- 172.21.15.26:0/1768794001 wait complete. 2023-10-06T21:17:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.667+0000 7f27fcc9ab80 20 client.26519 trim_cache size 0 max 16384 2023-10-06T21:17:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:17:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:17:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:17:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.668+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 msgr2=0x7f27d00a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.668+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 0x7f27d00a3c00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27cc09aa90 tx=0x7f27cc009310 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 msgr2=0x7f27e40817e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f27cc003760 tx=0x7f27cc00f690 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 shutdown_connections 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e407baf0 0x7f27e407bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27d00a1740 0x7f27d00a3c00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e4107840 0x7f27e40817e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 --2- 172.21.15.26:0/93339873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27e410c880 0x7f27e407b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.669+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 >> 172.21.15.26:0/93339873 conn(0x7f27e4076d50 msgr2=0x7f27e4077170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.670+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 shutdown_connections 2023-10-06T21:17:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:43.670+0000 7f27e93ff700 1 -- 172.21.15.26:0/93339873 wait complete. 2023-10-06T21:17:44.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:44 smithi078 ceph-mon[134896]: pgmap v228: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 617 op/s 2023-10-06T21:17:44.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:44 smithi167 ceph-mon[134494]: pgmap v228: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 617 op/s 2023-10-06T21:17:44.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:44 smithi026 ceph-mon[98490]: pgmap v228: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 617 op/s 2023-10-06T21:17:46.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:46 smithi026 ceph-mon[98490]: pgmap v229: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 653 op/s 2023-10-06T21:17:46.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:46 smithi078 ceph-mon[134896]: pgmap v229: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 653 op/s 2023-10-06T21:17:46.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:46 smithi167 ceph-mon[134494]: pgmap v229: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 653 op/s 2023-10-06T21:17:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 425.0 s, 1538.8 tps, lat 65.408 ms stddev 71.016 2023-10-06T21:17:48.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:48 smithi026 ceph-mon[98490]: pgmap v230: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 938 op/s 2023-10-06T21:17:48.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:48 smithi078 ceph-mon[134896]: pgmap v230: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 938 op/s 2023-10-06T21:17:48.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:48 smithi167 ceph-mon[134494]: pgmap v230: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 938 op/s 2023-10-06T21:17:50.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:50 smithi026 ceph-mon[98490]: pgmap v231: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 627 op/s 2023-10-06T21:17:50.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:50 smithi078 ceph-mon[134896]: pgmap v231: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 627 op/s 2023-10-06T21:17:50.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:50 smithi167 ceph-mon[134494]: pgmap v231: 97 pgs: 97 active+clean; 7.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 627 op/s 2023-10-06T21:17:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 430.0 s, 1432.8 tps, lat 70.269 ms stddev 97.088 2023-10-06T21:17:52.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:52 smithi026 ceph-mon[98490]: pgmap v232: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-06T21:17:52.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:52 smithi078 ceph-mon[134896]: pgmap v232: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-06T21:17:52.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:52 smithi167 ceph-mon[134494]: pgmap v232: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2023-10-06T21:17:54.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:54 smithi078 ceph-mon[134896]: pgmap v233: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-10-06T21:17:54.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:54 smithi167 ceph-mon[134494]: pgmap v233: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-10-06T21:17:54.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:54 smithi026 ceph-mon[98490]: pgmap v233: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.50k op/s 2023-10-06T21:17:56.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:56 smithi026 ceph-mon[98490]: pgmap v234: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:17:56.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:56 smithi078 ceph-mon[134896]: pgmap v234: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:17:56.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:56 smithi167 ceph-mon[134494]: pgmap v234: 97 pgs: 97 active+clean; 7.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.908+0000 7fdf54dff700 1 -- 172.21.15.26:0/323447113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5011a6a0 msgr2=0x7fdf5011ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.908+0000 7fdf54dff700 1 --2- 172.21.15.26:0/323447113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5011a6a0 0x7fdf5011ca90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009f90 tx=0x7fdf4800b330 comp rx=0 tx=0).stop 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 -- 172.21.15.26:0/323447113 shutdown_connections 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 --2- 172.21.15.26:0/323447113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5011a6a0 0x7fdf5011ca90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 --2- 172.21.15.26:0/323447113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf5010b5d0 0x7fdf50107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 --2- 172.21.15.26:0/323447113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf50108480 0x7fdf50108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 -- 172.21.15.26:0/323447113 >> 172.21.15.26:0/323447113 conn(0x7fdf5006d3e0 msgr2=0x7fdf5006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 -- 172.21.15.26:0/323447113 shutdown_connections 2023-10-06T21:17:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.909+0000 7fdf54dff700 1 -- 172.21.15.26:0/323447113 wait complete. 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.910+0000 7fdf54dff700 1 Processor -- start 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 -- start start 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf50108480 0x7fdf501a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf5011a6a0 0x7fdf501aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501ab170 con 0x7fdf5010b5d0 2023-10-06T21:17:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501ab2e0 con 0x7fdf5011a6a0 2023-10-06T21:17:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.911+0000 7fdf54dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501ab4a0 con 0x7fdf50108480 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51468/0 (socket says 172.21.15.26:51468) 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/1593821714 learned_addr learned my addr 172.21.15.26:0/1593821714 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/1593821714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf50108480 msgr2=0x7fdf501a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf50108480 0x7fdf501a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/1593821714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf5011a6a0 msgr2=0x7fdf501aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf5011a6a0 0x7fdf501aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.912+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/1593821714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400095f0 con 0x7fdf5010b5d0 2023-10-06T21:17:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b500 tx=0x7fdf4400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf3f7fe700 1 -- 172.21.15.26:0/1593821714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44011830 con 0x7fdf5010b5d0 2023-10-06T21:17:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48009c40 con 0x7fdf5010b5d0 2023-10-06T21:17:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf3f7fe700 1 -- 172.21.15.26:0/1593821714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf44011e50 con 0x7fdf5010b5d0 2023-10-06T21:17:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf3f7fe700 1 -- 172.21.15.26:0/1593821714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400f2e0 con 0x7fdf5010b5d0 2023-10-06T21:17:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.913+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501b2de0 con 0x7fdf5010b5d0 2023-10-06T21:17:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.921+0000 7fdf686eeb80 1 Processor -- start 2023-10-06T21:17:56.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.921+0000 7fdf686eeb80 1 -- start start 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bed4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4bee27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4be3e400 con 0x557a4bed4c00 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4be2fbc0 con 0x557a4bed5520 2023-10-06T21:17:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4be2d4e0 con 0x557a4bee03d0 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf4e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf4e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55546/0 (socket says 172.21.15.26:55546) 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf4e59c700 1 -- 172.21.15.26:0/4111075445 learned_addr learned my addr 172.21.15.26:0/4111075445 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.922+0000 7fdf3f7fe700 1 -- 172.21.15.26:0/1593821714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdf44010580 con 0x7fdf5010b5d0 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.923+0000 7fdf4e59c700 1 -- 172.21.15.26:0/4111075445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 msgr2=0x557a4bee27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.923+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4bee27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.923+0000 7fdf4e59c700 1 -- 172.21.15.26:0/4111075445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 msgr2=0x557a4bed4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.923+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bed4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.923+0000 7fdf4e59c700 1 -- 172.21.15.26:0/4111075445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a4bc53d80 con 0x557a4bed5520 2023-10-06T21:17:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.924+0000 7fdf3f7fe700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 0x7fdf380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.924+0000 7fdf3f7fe700 1 -- 172.21.15.26:0/1593821714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdf440ccb20 con 0x7fdf5010b5d0 2023-10-06T21:17:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.925+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009f30 tx=0x7fdf4000e630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d970ca2995a7ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.925+0000 7fdf4ed9d700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 0x7fdf380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.926+0000 7fdf4ed9d700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 0x7fdf380a3ab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009790 tx=0x7fdf480096e0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.928+0000 7fdf3cff9700 1 -- 172.21.15.26:0/4111075445 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009a80 con 0x557a4bed5520 2023-10-06T21:17:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.928+0000 7fdf3cff9700 1 -- 172.21.15.26:0/4111075445 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf400108f0 con 0x557a4bed5520 2023-10-06T21:17:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.928+0000 7fdf3cff9700 1 -- 172.21.15.26:0/4111075445 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400188b0 con 0x557a4bed5520 2023-10-06T21:17:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 -- 172.21.15.26:0/4111075445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 msgr2=0x557a4bedfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009f30 tx=0x7fdf4000e630 comp rx=0 tx=0).stop 2023-10-06T21:17:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 -- 172.21.15.26:0/4111075445 shutdown_connections 2023-10-06T21:17:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4bee27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bed4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/4111075445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedfe90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 -- 172.21.15.26:0/4111075445 >> 172.21.15.26:0/4111075445 conn(0x557a4be4a200 msgr2=0x557a4bed1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 -- 172.21.15.26:0/4111075445 shutdown_connections 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.929+0000 7fdf686eeb80 1 -- 172.21.15.26:0/4111075445 wait complete. 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.930+0000 7fdf686eeb80 1 -- start start 2023-10-06T21:17:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.930+0000 7fdf686eeb80 1 Processor -- start 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bedd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4beddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4be28dd0 con 0x557a4bed4c00 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bdc56b0 con 0x557a4bed5520 2023-10-06T21:17:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bdc5350 con 0x557a4bee03d0 2023-10-06T21:17:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.931+0000 7fdf686eeb80 10 client.0 _ll_register_callbacks cb 0x557a4bc227a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bedd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55550/0 (socket says 172.21.15.26:55550) 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2501589355 learned_addr learned my addr 172.21.15.26:0/2501589355 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4ed9d700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4beddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2501589355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 msgr2=0x557a4beddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4beddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2501589355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 msgr2=0x557a4bedd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bedd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.932+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2501589355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf40009be0 con 0x557a4bed5520 2023-10-06T21:17:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.933+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400bcc0 tx=0x7fdf440997e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e45f48eec5f3539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.933+0000 7fdf35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4409a000 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdf440174d0 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a4bec8d50 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4409a2a0 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf686eeb80 10 client.26525 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557a4bc3cc00 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.936+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a4bec8d50 con 0x557a4bed5520 2023-10-06T21:17:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf44099090 con 0x557a4bed5520 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 10 client.26525 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 20 client.26525 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4beeffe0 0x557a4bef2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 4 client.26525 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a4be21db0 con 0x557a4beeffe0 2023-10-06T21:17:56.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.937+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4beeffe0 0x557a4bef2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.938+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4beeffe0 0x557a4bef2400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fdb0d070cfe6610 server_cookie=215e3434d9f3e28 in_seq=0 out_seq=0 2023-10-06T21:17:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.938+0000 7fdf35ffb700 10 client.26525 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.938+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2501589355 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a4be21db0 con 0x557a4beeffe0 2023-10-06T21:17:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.938+0000 7fdf35ffb700 10 client.26525 handle_command_reply: tid=0 2023-10-06T21:17:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.938+0000 7fdf686eeb80 1 client.26525 shutdown 2023-10-06T21:17:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 msgr2=0x557a4bedd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400bcc0 tx=0x7fdf440997e0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 shutdown_connections 2023-10-06T21:17:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bee03d0 0x557a4beddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bed4c00 0x557a4bedd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4beeffe0 0x557a4bef2400 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2501589355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bedd770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 >> 172.21.15.26:0/2501589355 conn(0x557a4be4a200 msgr2=0x557a4bed0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf35ffb700 0 client.26525 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 shutdown_connections 2023-10-06T21:17:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2501589355 wait complete. 2023-10-06T21:17:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.939+0000 7fdf686eeb80 20 client.26525 trim_cache size 0 max 16384 2023-10-06T21:17:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.959+0000 7fdf686eeb80 1 Processor -- start 2023-10-06T21:17:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.959+0000 7fdf686eeb80 1 -- start start 2023-10-06T21:17:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.959+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bed7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.959+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bed8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bca98c0 con 0x557a4bee03d0 2023-10-06T21:17:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bed90a0 con 0x557a4bed5520 2023-10-06T21:17:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bef2700 con 0x557a4bed4c00 2023-10-06T21:17:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55564/0 (socket says 172.21.15.26:55564) 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/3209962364 learned_addr learned my addr 172.21.15.26:0/3209962364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bed7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4ed9d700 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bed8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/3209962364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 msgr2=0x557a4bed7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bed7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/3209962364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 msgr2=0x557a4bed8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bed8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/3209962364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a4bc7a1f0 con 0x557a4bed5520 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.960+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b4d0 tx=0x7fdf440ccd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9d0730aa16c27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.961+0000 7fdf3d7fa700 1 -- 172.21.15.26:0/3209962364 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400d040 con 0x557a4bed5520 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.961+0000 7fdf3d7fa700 1 -- 172.21.15.26:0/3209962364 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf440a1af0 con 0x557a4bed5520 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.961+0000 7fdf3d7fa700 1 -- 172.21.15.26:0/3209962364 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4409a040 con 0x557a4bed5520 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.961+0000 7fdf686eeb80 1 -- 172.21.15.26:0/3209962364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 msgr2=0x557a4beec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.961+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b4d0 tx=0x7fdf440ccd60 comp rx=0 tx=0).stop 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.962+0000 7fdf686eeb80 1 -- 172.21.15.26:0/3209962364 shutdown_connections 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.962+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bed8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.962+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bed7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.962+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/3209962364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4beec190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.962+0000 7fdf686eeb80 1 -- 172.21.15.26:0/3209962364 >> 172.21.15.26:0/3209962364 conn(0x557a4be4a200 msgr2=0x557a4bef1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.964+0000 7fdf686eeb80 1 -- 172.21.15.26:0/3209962364 shutdown_connections 2023-10-06T21:17:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.964+0000 7fdf686eeb80 1 -- 172.21.15.26:0/3209962364 wait complete. 2023-10-06T21:17:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.964+0000 7fdf686eeb80 1 -- start start 2023-10-06T21:17:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.965+0000 7fdf686eeb80 1 Processor -- start 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.965+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bee4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bee59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bf42e90 con 0x557a4bee03d0 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bf43000 con 0x557a4bed5520 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4bf431c0 con 0x557a4bed4c00 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf686eeb80 10 client.0 _ll_register_callbacks cb 0x7fdf44099970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf4dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55578/0 (socket says 172.21.15.26:55578) 2023-10-06T21:17:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2565094859 learned_addr learned my addr 172.21.15.26:0/2565094859 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:17:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.966+0000 7fdf4ed9d700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bee59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.967+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2565094859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 msgr2=0x557a4bee4f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.967+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bee4f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.967+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2565094859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 msgr2=0x557a4bee59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.967+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bee59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.967+0000 7fdf4dd9b700 1 -- 172.21.15.26:0/2565094859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a4bc3cc00 con 0x557a4bed5520 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf4dd9b700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf44099b30 tx=0x7fdf44010980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75145020bce1cce1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400d040 con 0x557a4bed5520 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdf440174d0 con 0x557a4bed5520 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf400108f0 con 0x557a4bed5520 2023-10-06T21:17:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf440b2270 con 0x557a4bed5520 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf686eeb80 10 client.26531 fetch_fsmap learned FSMap version 43 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557a4bc52250 con 0x557a4bed5520 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.968+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf400108f0 con 0x557a4bed5520 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf44065170 con 0x557a4bed5520 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 10 client.26531 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:17:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 20 client.26531 populate_metadata read hostname 'smithi026' 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4bf44400 0x557a4bf467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 4 client.26531 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a4be21ad0 con 0x557a4bf44400 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.969+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4bf44400 0x557a4bf467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.970+0000 7fdf4e59c700 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4bf44400 0x557a4bf467f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee77777090aadec3 server_cookie=8012e6c1b18e0759 in_seq=0 out_seq=0 2023-10-06T21:17:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.970+0000 7fdf35ffb700 10 client.26531 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf35ffb700 1 -- 172.21.15.26:0/2565094859 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a4be21ad0 con 0x557a4bf44400 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf35ffb700 10 client.26531 handle_command_reply: tid=0 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 client.26531 shutdown 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 msgr2=0x557a4bee5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf44099b30 tx=0x7fdf44010980 comp rx=0 tx=0).stop 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 shutdown_connections 2023-10-06T21:17:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a4bee03d0 0x557a4bee59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a4bed4c00 0x557a4bee4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a4bf44400 0x557a4bf467f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 --2- 172.21.15.26:0/2565094859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a4bed5520 0x557a4bee5470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 >> 172.21.15.26:0/2565094859 conn(0x557a4be4a200 msgr2=0x557a4beec190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf35ffb700 0 client.26531 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 shutdown_connections 2023-10-06T21:17:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.994+0000 7fdf686eeb80 1 -- 172.21.15.26:0/2565094859 wait complete. 2023-10-06T21:17:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.995+0000 7fdf686eeb80 20 client.26531 trim_cache size 0 max 16384 2023-10-06T21:17:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:17:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:17:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:17:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.995+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 msgr2=0x7fdf380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.995+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 0x7fdf380a3ab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009790 tx=0x7fdf480096e0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.995+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 msgr2=0x7fdf501a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.995+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b500 tx=0x7fdf4400b8c0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 shutdown_connections 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf5011a6a0 0x7fdf501aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf50108480 0x7fdf501a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdf380a15f0 0x7fdf380a3ab0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 --2- 172.21.15.26:0/1593821714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5010b5d0 0x7fdf501a64c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:17:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 >> 172.21.15.26:0/1593821714 conn(0x7fdf5006d3e0 msgr2=0x7fdf5010fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:17:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 shutdown_connections 2023-10-06T21:17:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:17:56.996+0000 7fdf54dff700 1 -- 172.21.15.26:0/1593821714 wait complete. 2023-10-06T21:17:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 435.0 s, 1289.8 tps, lat 76.788 ms stddev 133.550 2023-10-06T21:17:58.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:17:58 smithi078 ceph-mon[134896]: pgmap v235: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.75k op/s 2023-10-06T21:17:58.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:17:58 smithi167 ceph-mon[134494]: pgmap v235: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.75k op/s 2023-10-06T21:17:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:17:58 smithi026 ceph-mon[98490]: pgmap v235: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 1.75k op/s 2023-10-06T21:18:00.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:00 smithi078 ceph-mon[134896]: pgmap v236: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2023-10-06T21:18:00.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:00 smithi167 ceph-mon[134494]: pgmap v236: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2023-10-06T21:18:00.857 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:00 smithi026 ceph-mon[98490]: pgmap v236: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2023-10-06T21:18:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 440.0 s, 1512.2 tps, lat 66.368 ms stddev 92.821 2023-10-06T21:18:02.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:02 smithi078 ceph-mon[134896]: pgmap v237: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-10-06T21:18:02.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:18:02.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:02 smithi026 ceph-mon[98490]: pgmap v237: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-10-06T21:18:02.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:02 smithi167 ceph-mon[134494]: pgmap v237: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-10-06T21:18:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 -- 172.21.15.26:0/1054255345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 msgr2=0x7f40f8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 --2- 172.21.15.26:0/1054255345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f8101490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009990 tx=0x7f40ec009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 -- 172.21.15.26:0/1054255345 shutdown_connections 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 --2- 172.21.15.26:0/1054255345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 0x7f40f810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 --2- 172.21.15.26:0/1054255345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f8101490 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 --2- 172.21.15.26:0/1054255345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f8101a60 0x7f40f8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.930+0000 7f40fefff700 1 -- 172.21.15.26:0/1054255345 >> 172.21.15.26:0/1054255345 conn(0x7f40f80fc930 msgr2=0x7f40f80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.931+0000 7f40fefff700 1 -- 172.21.15.26:0/1054255345 shutdown_connections 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.931+0000 7f40fefff700 1 -- 172.21.15.26:0/1054255345 wait complete. 2023-10-06T21:18:02.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.932+0000 7f40fefff700 1 Processor -- start 2023-10-06T21:18:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.932+0000 7f40fefff700 1 -- start start 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.932+0000 7f40fefff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.932+0000 7f40fefff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 0x7f40f819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fefff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f81a2000 0x7f40f81a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fefff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f81a2a80 con 0x7f40f81061d0 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fefff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f81a2bf0 con 0x7f40f81010b0 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fefff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f81a2db0 con 0x7f40f81a2000 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41870/0 (socket says 172.21.15.26:41870) 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 -- 172.21.15.26:0/2840911216 learned_addr learned my addr 172.21.15.26:0/2840911216 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40f7fff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 0x7f40f819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fd59c700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f81a2000 0x7f40f81a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 -- 172.21.15.26:0/2840911216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f81a2000 msgr2=0x7f40f81a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f81a2000 0x7f40f81a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 -- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 msgr2=0x7f40f819dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 0x7f40f819dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.933+0000 7f40fcd9b700 1 -- 172.21.15.26:0/2840911216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0095f0 con 0x7f40f81010b0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005b40 tx=0x7f40ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40f5ffb700 1 -- 172.21.15.26:0/2840911216 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01d070 con 0x7f40f81010b0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f81aa2b0 con 0x7f40f81010b0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40f5ffb700 1 -- 172.21.15.26:0/2840911216 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec005da0 con 0x7f40f81010b0 2023-10-06T21:18:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40f5ffb700 1 -- 172.21.15.26:0/2840911216 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0173f0 con 0x7f40f81010b0 2023-10-06T21:18:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.934+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f81aa780 con 0x7f40f81010b0 2023-10-06T21:18:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.940+0000 7f40f5ffb700 1 -- 172.21.15.26:0/2840911216 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f40ec022600 con 0x7f40f81010b0 2023-10-06T21:18:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.941+0000 7f411284bb80 1 Processor -- start 2023-10-06T21:18:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.944+0000 7f40f5ffb700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 0x7f40e80a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.944+0000 7f40f5ffb700 1 -- 172.21.15.26:0/2840911216 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f40ec0ce3b0 con 0x7f40f81010b0 2023-10-06T21:18:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f40fd59c700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 0x7f40e80a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 -- start start 2023-10-06T21:18:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc40c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb9c460 con 0x561a5cc3e890 2023-10-06T21:18:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb8dc20 con 0x561a5cc330c0 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f411284bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb8b540 con 0x561a5cc339e0 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.945+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41876/0 (socket says 172.21.15.26:41876) 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1824394657 learned_addr learned my addr 172.21.15.26:0/1824394657 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fd59c700 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc40c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1824394657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 msgr2=0x561a5cc3e350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3e350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1824394657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 msgr2=0x561a5cc40c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc40c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1824394657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5c9b1de0 con 0x561a5cc330c0 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fd59c700 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc40c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fd59c700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 0x7f40e80a3b90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009390 tx=0x7f40f0009650 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0978b0 tx=0x7f40ec005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746f535e9fd9347e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40e2ffd700 1 -- 172.21.15.26:0/1824394657 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec09b3b0 con 0x561a5cc330c0 2023-10-06T21:18:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.946+0000 7f40e2ffd700 1 -- 172.21.15.26:0/1824394657 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec09f500 con 0x561a5cc330c0 2023-10-06T21:18:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.947+0000 7f40e2ffd700 1 -- 172.21.15.26:0/1824394657 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec09b850 con 0x561a5cc330c0 2023-10-06T21:18:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.947+0000 7f411284bb80 1 -- 172.21.15.26:0/1824394657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 msgr2=0x561a5cc334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.947+0000 7f411284bb80 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0978b0 tx=0x7f40ec005f10 comp rx=0 tx=0).stop 2023-10-06T21:18:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 -- 172.21.15.26:0/1824394657 shutdown_connections 2023-10-06T21:18:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc40c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc334a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 --2- 172.21.15.26:0/1824394657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 -- 172.21.15.26:0/1824394657 >> 172.21.15.26:0/1824394657 conn(0x561a5cba8260 msgr2=0x561a5cc2f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 -- 172.21.15.26:0/1824394657 shutdown_connections 2023-10-06T21:18:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.948+0000 7f411284bb80 1 -- 172.21.15.26:0/1824394657 wait complete. 2023-10-06T21:18:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.949+0000 7f411284bb80 1 -- start start 2023-10-06T21:18:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.949+0000 7f411284bb80 1 Processor -- start 2023-10-06T21:18:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc3ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc3f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb86e30 con 0x561a5cc3e890 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb23710 con 0x561a5cc330c0 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cb233b0 con 0x561a5cc339e0 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f411284bb80 10 client.0 _ll_register_callbacks cb 0x561a5c980800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f40f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f40f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44476/0 (socket says 172.21.15.26:44476) 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f40f7fff700 1 -- 172.21.15.26:0/2222447203 learned_addr learned my addr 172.21.15.26:0/2222447203 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.950+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc3ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 -- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 msgr2=0x561a5cc3ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc3ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 -- 172.21.15.26:0/2222447203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 msgr2=0x561a5cc3f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc3f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 -- 172.21.15.26:0/2222447203 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f81aa780 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc3ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40f7fff700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bde0 tx=0x7f40e40106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36b157768f3c465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e401c680 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a5cc26f10 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.951+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f40e4017070 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e4010e50 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 10 client.26479 fetch_fsmap learned FSMap version 43 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561a5c99ac60 con 0x561a5cc339e0 2023-10-06T21:18:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a5cc26f10 con 0x561a5cc339e0 2023-10-06T21:18:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40e40118a0 con 0x561a5cc339e0 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 10 client.26479 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 10 client.26479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 20 client.26479 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5cc4df70 0x561a5cc50390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 4 client.26479 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a5cb7fe10 con 0x561a5cc4df70 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.952+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5cc4df70 0x561a5cc50390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.953+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5cc4df70 0x561a5cc50390 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbe54399e218b1f server_cookie=baaaee046d37e777 in_seq=0 out_seq=0 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.953+0000 7f40d7fff700 10 client.26479 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.954+0000 7f40d7fff700 1 -- 172.21.15.26:0/2222447203 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a5cb7fe10 con 0x561a5cc4df70 2023-10-06T21:18:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.954+0000 7f40d7fff700 10 client.26479 handle_command_reply: tid=0 2023-10-06T21:18:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 client.26479 shutdown 2023-10-06T21:18:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 msgr2=0x561a5cc3f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bde0 tx=0x7f40e40106b0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 shutdown_connections 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc3f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5cc4df70 0x561a5cc50390 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc3ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 --2- 172.21.15.26:0/2222447203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc339e0 0x561a5cc3f1b0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bde0 tx=0x7f40e40106b0 comp rx=0 tx=0).stop 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.955+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 >> 172.21.15.26:0/2222447203 conn(0x561a5cba8260 msgr2=0x561a5cc2efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.956+0000 7f40d7fff700 0 client.26479 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.956+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 shutdown_connections 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.957+0000 7f411284bb80 1 -- 172.21.15.26:0/2222447203 wait complete. 2023-10-06T21:18:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.957+0000 7f411284bb80 20 client.26479 trim_cache size 0 max 16384 2023-10-06T21:18:03.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:02.998+0000 7f411284bb80 1 Processor -- start 2023-10-06T21:18:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 -- start start 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc3d080 0x561a5cc2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5ca07920 con 0x561a5cc3e890 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cc302a0 con 0x561a5cc330c0 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.006+0000 7f411284bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5cc34310 con 0x561a5cc3d080 2023-10-06T21:18:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.013+0000 7f40f7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41898/0 (socket says 172.21.15.26:41898) 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1229663907 learned_addr learned my addr 172.21.15.26:0/1229663907 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1229663907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc3d080 msgr2=0x561a5cc2ed70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc3d080 0x561a5cc2ed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1229663907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 msgr2=0x561a5cc42ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.016+0000 7f40fcd9b700 1 -- 172.21.15.26:0/1229663907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5cc54fc0 con 0x561a5cc330c0 2023-10-06T21:18:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.021+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b5c0 tx=0x7f40ec0ce2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f7d4e8a3993809 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:03.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.027+0000 7f40e37fe700 1 -- 172.21.15.26:0/1229663907 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec099750 con 0x561a5cc330c0 2023-10-06T21:18:03.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.027+0000 7f40e37fe700 1 -- 172.21.15.26:0/1229663907 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec09a8a0 con 0x561a5cc330c0 2023-10-06T21:18:03.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.027+0000 7f40e37fe700 1 -- 172.21.15.26:0/1229663907 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0a3220 con 0x561a5cc330c0 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 -- 172.21.15.26:0/1229663907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 msgr2=0x561a5cc2de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b5c0 tx=0x7f40ec0ce2e0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 -- 172.21.15.26:0/1229663907 shutdown_connections 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc3d080 0x561a5cc2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc330c0 0x561a5cc2de20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 --2- 172.21.15.26:0/1229663907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 -- 172.21.15.26:0/1229663907 >> 172.21.15.26:0/1229663907 conn(0x561a5cba8260 msgr2=0x561a5cc2e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:03.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 -- 172.21.15.26:0/1229663907 shutdown_connections 2023-10-06T21:18:03.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.028+0000 7f411284bb80 1 -- 172.21.15.26:0/1229663907 wait complete. 2023-10-06T21:18:03.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.029+0000 7f411284bb80 1 -- start start 2023-10-06T21:18:03.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.029+0000 7f411284bb80 1 Processor -- start 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc330c0 0x561a5cc42370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5ccb1da0 con 0x561a5cc3e890 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5ccb1f10 con 0x561a5cc3d080 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5ccb20d0 con 0x561a5cc330c0 2023-10-06T21:18:03.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.030+0000 7f411284bb80 10 client.0 _ll_register_callbacks cb 0x7f40e400b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:03.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.031+0000 7f40f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:03.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.031+0000 7f40f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41904/0 (socket says 172.21.15.26:41904) 2023-10-06T21:18:03.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.031+0000 7f40f7fff700 1 -- 172.21.15.26:0/3902985369 learned_addr learned my addr 172.21.15.26:0/3902985369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.031+0000 7f40fd59c700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 -- 172.21.15.26:0/3902985369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc330c0 msgr2=0x561a5cc42370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc330c0 0x561a5cc42370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 -- 172.21.15.26:0/3902985369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 msgr2=0x561a5cc42df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 -- 172.21.15.26:0/3902985369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a5c9b1de0 con 0x561a5cc3d080 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40fd59c700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40f7fff700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4010b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292a8d582ca190f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:03.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40054c0 con 0x561a5cc3d080 2023-10-06T21:18:03.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f40e4017070 con 0x561a5cc3d080 2023-10-06T21:18:03.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.032+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40ec09f500 con 0x561a5cc3d080 2023-10-06T21:18:03.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.033+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e4010e50 con 0x561a5cc3d080 2023-10-06T21:18:03.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.033+0000 7f411284bb80 10 client.26549 fetch_fsmap learned FSMap version 43 2023-10-06T21:18:03.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.033+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561a5c9d8250 con 0x561a5cc3d080 2023-10-06T21:18:03.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.033+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec09f500 con 0x561a5cc3d080 2023-10-06T21:18:03.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40e402a030 con 0x561a5cc3d080 2023-10-06T21:18:03.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 10 client.26549 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 20 client.26549 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5ccb3820 0x561a5ccb5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 4 client.26549 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a5cb7fb30 con 0x561a5ccb3820 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.034+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5ccb3820 0x561a5ccb5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f40fcd9b700 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5ccb3820 0x561a5ccb5c40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929e41c6dd6a3a76 server_cookie=77f10a2b03e2b4b4 in_seq=0 out_seq=0 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f40d7fff700 10 client.26549 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f40d7fff700 1 -- 172.21.15.26:0/3902985369 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561a5cb7fb30 con 0x561a5ccb3820 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f40d7fff700 10 client.26549 handle_command_reply: tid=0 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f411284bb80 1 client.26549 shutdown 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 msgr2=0x561a5cc428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4010b60 comp rx=0 tx=0).stop 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.035+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 shutdown_connections 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a5cc3e890 0x561a5cc42df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5cc330c0 0x561a5cc42370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561a5ccb3820 0x561a5ccb5c40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 --2- 172.21.15.26:0/3902985369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a5cc3d080 0x561a5cc428b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f40d7fff700 0 client.26549 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 >> 172.21.15.26:0/3902985369 conn(0x561a5cba8260 msgr2=0x561a5cc3c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 shutdown_connections 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 1 -- 172.21.15.26:0/3902985369 wait complete. 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.036+0000 7f411284bb80 20 client.26549 trim_cache size 0 max 16384 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:18:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:18:03.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 msgr2=0x7f40e80a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 0x7f40e80a3b90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009390 tx=0x7f40f0009650 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 msgr2=0x7f40f819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005b40 tx=0x7f40ec00f690 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 shutdown_connections 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40f81a2000 0x7f40f81a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81010b0 0x7f40f819d700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f40e80a16d0 0x7f40e80a3b90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 --2- 172.21.15.26:0/2840911216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f81061d0 0x7f40f819dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 >> 172.21.15.26:0/2840911216 conn(0x7f40f80fc930 msgr2=0x7f40f8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 shutdown_connections 2023-10-06T21:18:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:03.037+0000 7f40fefff700 1 -- 172.21.15.26:0/2840911216 wait complete. 2023-10-06T21:18:03.063 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:18:04.089 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:04 smithi078 ceph-mon[134896]: pgmap v238: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 652 op/s 2023-10-06T21:18:04.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:04 smithi167 ceph-mon[134494]: pgmap v238: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 652 op/s 2023-10-06T21:18:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:04 smithi026 ceph-mon[98490]: pgmap v238: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 652 op/s 2023-10-06T21:18:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:06 smithi026 ceph-mon[98490]: pgmap v239: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-10-06T21:18:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:06 smithi078 ceph-mon[134896]: pgmap v239: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-10-06T21:18:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:06 smithi167 ceph-mon[134494]: pgmap v239: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-10-06T21:18:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 445.0 s, 1791.5 tps, lat 55.567 ms stddev 70.414 2023-10-06T21:18:08.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:08 smithi078 ceph-mon[134896]: pgmap v240: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-10-06T21:18:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:08 smithi167 ceph-mon[134494]: pgmap v240: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-10-06T21:18:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:08 smithi026 ceph-mon[98490]: pgmap v240: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 -- 172.21.15.26:0/2485529222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108910 msgr2=0x7fdbb810c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/2485529222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108910 0x7fdbb810c150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdba800b120 tx=0x7fdba800b430 comp rx=0 tx=0).stop 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 -- 172.21.15.26:0/2485529222 shutdown_connections 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/2485529222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb810c880 0x7fdbb8114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/2485529222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb8107f60 0x7fdbb8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/2485529222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8108910 0x7fdbb810c150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 -- 172.21.15.26:0/2485529222 >> 172.21.15.26:0/2485529222 conn(0x7fdbb8076c10 msgr2=0x7fdbb8079030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 -- 172.21.15.26:0/2485529222 shutdown_connections 2023-10-06T21:18:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.219+0000 7fdbbcdff700 1 -- 172.21.15.26:0/2485529222 wait complete. 2023-10-06T21:18:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.220+0000 7fdbbcdff700 1 Processor -- start 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.220+0000 7fdbbcdff700 1 -- start start 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb810c880 0x7fdbb81a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb819d3f0 0x7fdbb819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb819de70 con 0x7fdbb819d3f0 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb819dfb0 con 0x7fdbb8107f60 2023-10-06T21:18:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbbcdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb819e170 con 0x7fdbb810c880 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41912/0 (socket says 172.21.15.26:41912) 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.221+0000 7fdbb659c700 1 -- 172.21.15.26:0/221769844 learned_addr learned my addr 172.21.15.26:0/221769844 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb819d3f0 0x7fdbb819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb659c700 1 -- 172.21.15.26:0/221769844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb810c880 msgr2=0x7fdbb81a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb659c700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb810c880 0x7fdbb81a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb659c700 1 -- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb819d3f0 msgr2=0x7fdbb819d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb659c700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb819d3f0 0x7fdbb819d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.222+0000 7fdbb659c700 1 -- 172.21.15.26:0/221769844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009d00 con 0x7fdbb8107f60 2023-10-06T21:18:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.223+0000 7fdbb659c700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9a0 tx=0x7fdba000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.223+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/221769844 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0009790 con 0x7fdbb8107f60 2023-10-06T21:18:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.223+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/221769844 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba000cbd0 con 0x7fdbb8107f60 2023-10-06T21:18:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.223+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/221769844 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0018680 con 0x7fdbb8107f60 2023-10-06T21:18:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.224+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb81aa400 con 0x7fdbb8107f60 2023-10-06T21:18:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.224+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb81aa8c0 con 0x7fdbb8107f60 2023-10-06T21:18:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.230+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/221769844 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdba000cd40 con 0x7fdbb8107f60 2023-10-06T21:18:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.231+0000 7fdbd06bcb80 1 Processor -- start 2023-10-06T21:18:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 -- start start 2023-10-06T21:18:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbaf7fe700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 0x7fdba40a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 0x55d80a7c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbaf7fe700 1 -- 172.21.15.26:0/221769844 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdba0014070 con 0x7fdbb8107f60 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a725400 con 0x55d80a7bbc00 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a716bc0 con 0x55d80a7c73d0 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a7144e0 con 0x55d80a7bc520 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.232+0000 7fdbb6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41924/0 (socket says 172.21.15.26:41924) 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/4040746686 learned_addr learned my addr 172.21.15.26:0/4040746686 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 0x7fdba40a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 0x55d80a7c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb659c700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/4040746686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 msgr2=0x55d80a7c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 0x55d80a7c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/4040746686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 msgr2=0x55d80a7bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7bbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/4040746686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d80a53ad80 con 0x55d80a7c73d0 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 0x55d80a7c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb659c700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0014ba0 tx=0x7fdbb0014eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b091cbf7a59b1d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.233+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 0x7fdba40a3c00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00116c0 tx=0x7fdbb0009000 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.235+0000 7fdbacff9700 1 -- 172.21.15.26:0/4040746686 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0014590 con 0x55d80a7c73d0 2023-10-06T21:18:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.235+0000 7fdbacff9700 1 -- 172.21.15.26:0/4040746686 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb001a040 con 0x55d80a7c73d0 2023-10-06T21:18:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.235+0000 7fdbacff9700 1 -- 172.21.15.26:0/4040746686 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0004cd0 con 0x55d80a7c73d0 2023-10-06T21:18:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.236+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/4040746686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 msgr2=0x55d80a7c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.236+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0014ba0 tx=0x7fdbb0014eb0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/4040746686 shutdown_connections 2023-10-06T21:18:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7c73d0 0x55d80a7c97c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/4040746686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7bc520 0x55d80a7c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/4040746686 >> 172.21.15.26:0/4040746686 conn(0x55d80a731200 msgr2=0x55d80a7b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/4040746686 shutdown_connections 2023-10-06T21:18:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.237+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/4040746686 wait complete. 2023-10-06T21:18:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.238+0000 7fdbd06bcb80 1 -- start start 2023-10-06T21:18:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.239+0000 7fdbd06bcb80 1 Processor -- start 2023-10-06T21:18:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.239+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.239+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.239+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.239+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a70fdd0 con 0x55d80a7bc520 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a6ac6b0 con 0x55d80a7bbc00 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a6ac350 con 0x55d80a7c73d0 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbd06bcb80 10 client.0 _ll_register_callbacks cb 0x55d80a5097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41928/0 (socket says 172.21.15.26:41928) 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 -- 172.21.15.26:0/1547176286 learned_addr learned my addr 172.21.15.26:0/1547176286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb6d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:44548/0 (socket says 172.21.15.26:44548) 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 -- 172.21.15.26:0/1547176286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 msgr2=0x55d80a7c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 -- 172.21.15.26:0/1547176286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 msgr2=0x55d80a7c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb659c700 1 -- 172.21.15.26:0/1547176286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb8114c10 con 0x55d80a7bbc00 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.240+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdbb659c700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdba009a200 tx=0x7fdba009aab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd48222340552a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdb9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00cc040 con 0x55d80a7bbc00 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdba00a2600 con 0x55d80a7bbc00 2023-10-06T21:18:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d80a7afd50 con 0x55d80a7bbc00 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba009e170 con 0x55d80a7bbc00 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdbd06bcb80 10 client.26561 fetch_fsmap learned FSMap version 43 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.241+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fdbb0014750 con 0x55d80a7bbc00 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdba00a6080 con 0x55d80a7bbc00 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdba00991d0 con 0x55d80a7bbc00 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 10 client.26561 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:18:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 20 client.26561 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a7d7010 0x55d80a7d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 4 client.26561 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d80a708db0 con 0x55d80a7d7010 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.242+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a7d7010 0x55d80a7d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.243+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a7d7010 0x55d80a7d9430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34fd08fc4cbe8824 server_cookie=cf4f96c1a729fd92 in_seq=0 out_seq=0 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.243+0000 7fdb9dffb700 10 client.26561 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdb9dffb700 1 -- 172.21.15.26:0/1547176286 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d80a708db0 con 0x55d80a7d7010 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdb9dffb700 10 client.26561 handle_command_reply: tid=0 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 client.26561 shutdown 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 msgr2=0x55d80a7c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdba009a200 tx=0x7fdba009aab0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 shutdown_connections 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a7d7010 0x55d80a7d9430 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6e80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/1547176286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdb9dffb700 0 client.26561 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 >> 172.21.15.26:0/1547176286 conn(0x55d80a731200 msgr2=0x55d80a7b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 shutdown_connections 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.244+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/1547176286 wait complete. 2023-10-06T21:18:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.245+0000 7fdbd06bcb80 20 client.26561 trim_cache size 0 max 16384 2023-10-06T21:18:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.268+0000 7fdbd06bcb80 1 Processor -- start 2023-10-06T21:18:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 -- start start 2023-10-06T21:18:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a5908c0 con 0x55d80a7bc520 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a7b8b70 con 0x55d80a7bbc00 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a7af920 con 0x55d80a7c73d0 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbb5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbb5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41878/0 (socket says 172.21.15.26:41878) 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbb5d9b700 1 -- 172.21.15.26:0/2626584898 learned_addr learned my addr 172.21.15.26:0/2626584898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbb659c700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.269+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/2626584898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 msgr2=0x55d80a7c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/2626584898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 msgr2=0x55d80a7c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 -- 172.21.15.26:0/2626584898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d80a523c00 con 0x55d80a7c73d0 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c3e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f930 tx=0x7fdbb000ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2da15c4ac511df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.270+0000 7fdbad7fa700 1 -- 172.21.15.26:0/2626584898 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0014590 con 0x55d80a7c73d0 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbad7fa700 1 -- 172.21.15.26:0/2626584898 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb0005ce0 con 0x55d80a7c73d0 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbad7fa700 1 -- 172.21.15.26:0/2626584898 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0004950 con 0x55d80a7c73d0 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/2626584898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 msgr2=0x55d80a7c3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c3e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f930 tx=0x7fdbb000ff60 comp rx=0 tx=0).stop 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/2626584898 shutdown_connections 2023-10-06T21:18:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a7c3e50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bbc00 0x55d80a7c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/2626584898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bc520 0x55d80a7c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.271+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/2626584898 >> 172.21.15.26:0/2626584898 conn(0x55d80a731200 msgr2=0x55d80a7bf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.272+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/2626584898 shutdown_connections 2023-10-06T21:18:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.275+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/2626584898 wait complete. 2023-10-06T21:18:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.276+0000 7fdbd06bcb80 1 -- start start 2023-10-06T21:18:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.276+0000 7fdbd06bcb80 1 Processor -- start 2023-10-06T21:18:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a829e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a7cc8f0 con 0x55d80a7bbc00 2023-10-06T21:18:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a82a370 con 0x55d80a7bc520 2023-10-06T21:18:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80a82a530 con 0x55d80a7c73d0 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbd06bcb80 10 client.0 _ll_register_callbacks cb 0x7fdba0099bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbb5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbb5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41958/0 (socket says 172.21.15.26:41958) 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbb5d9b700 1 -- 172.21.15.26:0/3944199971 learned_addr learned my addr 172.21.15.26:0/3944199971 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.277+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a829e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 -- 172.21.15.26:0/3944199971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 msgr2=0x55d80a829e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a829e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 -- 172.21.15.26:0/3944199971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 msgr2=0x55d80a7cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 -- 172.21.15.26:0/3944199971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba800bc50 con 0x55d80a7bc520 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb6d9d700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a829e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbb5d9b700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdba800afe0 tx=0x7fdba8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32f1297a2224f67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdb9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800e040 con 0x55d80a7bc520 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbb001a040 con 0x55d80a7bc520 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.278+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdba80074e0 con 0x55d80a7bc520 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba80076a0 con 0x55d80a7bc520 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 10 client.26567 fetch_fsmap learned FSMap version 43 2023-10-06T21:18:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d80a53ad80 con 0x55d80a7bc520 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb001a040 con 0x55d80a7bc520 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdba802a060 con 0x55d80a7bc520 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 10 client.26567 fetch_fsmap finished waiting for FSMap version 43 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 20 client.26567 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a82be20 0x55d80a82e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 4 client.26567 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:18:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.279+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d80a708ad0 con 0x55d80a82be20 2023-10-06T21:18:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.280+0000 7fdbb659c700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a82be20 0x55d80a82e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.280+0000 7fdb9dffb700 10 client.26567 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.280+0000 7fdbb659c700 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a82be20 0x55d80a82e240 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75384f51b3086a5 server_cookie=efa3c40e12ad9dfe in_seq=0 out_seq=0 2023-10-06T21:18:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.301+0000 7fdb9dffb700 1 -- 172.21.15.26:0/3944199971 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d80a708ad0 con 0x55d80a82be20 2023-10-06T21:18:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.301+0000 7fdb9dffb700 10 client.26567 handle_command_reply: tid=0 2023-10-06T21:18:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.301+0000 7fdbd06bcb80 1 client.26567 shutdown 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 msgr2=0x55d80a7cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdba800afe0 tx=0x7fdba8004c80 comp rx=0 tx=0).stop 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 shutdown_connections 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d80a7c73d0 0x55d80a829e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d80a7bbc00 0x55d80a7cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d80a82be20 0x55d80a82e240 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 --2- 172.21.15.26:0/3944199971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d80a7bc520 0x55d80a7cc160 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.302+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 >> 172.21.15.26:0/3944199971 conn(0x55d80a731200 msgr2=0x55d80a7b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.303+0000 7fdb9dffb700 0 client.26567 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.303+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 shutdown_connections 2023-10-06T21:18:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.303+0000 7fdbd06bcb80 1 -- 172.21.15.26:0/3944199971 wait complete. 2023-10-06T21:18:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.304+0000 7fdbd06bcb80 20 client.26567 trim_cache size 0 max 16384 2023-10-06T21:18:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:18:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:18:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:18:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.304+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 msgr2=0x7fdba40a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.304+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 0x7fdba40a3c00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00116c0 tx=0x7fdbb0009000 comp rx=0 tx=0).stop 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.304+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 msgr2=0x7fdbb81a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.304+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e9a0 tx=0x7fdba000ed60 comp rx=0 tx=0).stop 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 shutdown_connections 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdba40a1740 0x7fdba40a3c00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb819d3f0 0x7fdbb819d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbb8107f60 0x7fdbb81a3590 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 --2- 172.21.15.26:0/221769844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb810c880 0x7fdbb81a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 >> 172.21.15.26:0/221769844 conn(0x7fdbb8076c10 msgr2=0x7fdbb810bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 shutdown_connections 2023-10-06T21:18:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:10.305+0000 7fdbbcdff700 1 -- 172.21.15.26:0/221769844 wait complete. 2023-10-06T21:18:10.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:10 smithi167 ceph-mon[134494]: pgmap v241: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-10-06T21:18:10.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:10 smithi026 ceph-mon[98490]: pgmap v241: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-10-06T21:18:10.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:10 smithi078 ceph-mon[134896]: pgmap v241: 97 pgs: 97 active+clean; 7.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-10-06T21:18:11.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:11 smithi167 ceph-mon[134494]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:18:11.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:11 smithi026 ceph-mon[98490]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:18:11.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:11 smithi078 ceph-mon[134896]: osdmap e92: 12 total, 12 up, 12 in 2023-10-06T21:18:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 450.0 s, 1445.2 tps, lat 69.271 ms stddev 91.837 2023-10-06T21:18:12.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:12 smithi167 ceph-mon[134494]: pgmap v243: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-06T21:18:12.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:12 smithi026 ceph-mon[98490]: pgmap v243: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-06T21:18:12.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:12 smithi078 ceph-mon[134896]: pgmap v243: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-06T21:18:14.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:14 smithi078 ceph-mon[134896]: pgmap v244: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:18:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:14 smithi167 ceph-mon[134494]: pgmap v244: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:18:14.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:14 smithi026 ceph-mon[98490]: pgmap v244: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:18:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:16 smithi026 ceph-mon[98490]: pgmap v245: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-10-06T21:18:16.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:16 smithi078 ceph-mon[134896]: pgmap v245: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-10-06T21:18:16.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:16 smithi167 ceph-mon[134494]: pgmap v245: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 7.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-10-06T21:18:17.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 455.0 s, 1547.6 tps, lat 64.114 ms stddev 72.356 2023-10-06T21:18:18.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:18 smithi078 ceph-mon[134896]: pgmap v246: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 713 op/s 2023-10-06T21:18:18.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:18 smithi078 ceph-mon[134896]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:18:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:18 smithi167 ceph-mon[134494]: pgmap v246: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 713 op/s 2023-10-06T21:18:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:18 smithi167 ceph-mon[134494]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:18:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:18 smithi026 ceph-mon[98490]: pgmap v246: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 713 op/s 2023-10-06T21:18:18.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:18 smithi026 ceph-mon[98490]: osdmap e93: 12 total, 12 up, 12 in 2023-10-06T21:18:20.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:20 smithi078 ceph-mon[134896]: pgmap v248: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 590 op/s 2023-10-06T21:18:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:20 smithi167 ceph-mon[134494]: pgmap v248: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 590 op/s 2023-10-06T21:18:20.839 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:20 smithi026 ceph-mon[98490]: pgmap v248: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 590 op/s 2023-10-06T21:18:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 460.0 s, 1903.0 tps, lat 52.794 ms stddev 63.387 2023-10-06T21:18:22.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:22 smithi078 ceph-mon[134896]: pgmap v249: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 717 op/s 2023-10-06T21:18:22.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:22 smithi167 ceph-mon[134494]: pgmap v249: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 717 op/s 2023-10-06T21:18:22.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:22 smithi026 ceph-mon[98490]: pgmap v249: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 717 op/s 2023-10-06T21:18:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 -- 172.21.15.26:0/1739071021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01010c0 msgr2=0x7fe0e01014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/1739071021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01010c0 0x7fe0e01014a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009990 tx=0x7fe0dc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 -- 172.21.15.26:0/1739071021 shutdown_connections 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/1739071021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/1739071021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01010c0 0x7fe0e01014a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/1739071021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e0101a70 0x7fe0e0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.544+0000 7fe0e77ff700 1 -- 172.21.15.26:0/1739071021 >> 172.21.15.26:0/1739071021 conn(0x7fe0e00fc940 msgr2=0x7fe0e00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.545+0000 7fe0e77ff700 1 -- 172.21.15.26:0/1739071021 shutdown_connections 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.545+0000 7fe0e77ff700 1 -- 172.21.15.26:0/1739071021 wait complete. 2023-10-06T21:18:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.546+0000 7fe0e77ff700 1 Processor -- start 2023-10-06T21:18:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.546+0000 7fe0e77ff700 1 -- start start 2023-10-06T21:18:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 0x7fe0e010d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 0x7fe0e010b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e010bb50 con 0x7fe0e00729d0 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e010bcc0 con 0x7fe0e01010c0 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e77ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e010be80 con 0x7fe0e01061e0 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 0x7fe0e010d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.550 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e5d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 0x7fe0e010b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 0x7fe0e010d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33538/0 (socket says 172.21.15.26:33538) 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.547+0000 7fe0e559b700 1 -- 172.21.15.26:0/3359256893 learned_addr learned my addr 172.21.15.26:0/3359256893 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 -- 172.21.15.26:0/3359256893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 msgr2=0x7fe0e010dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 -- 172.21.15.26:0/3359256893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 msgr2=0x7fe0e010d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 0x7fe0e010d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 -- 172.21.15.26:0/3359256893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0095f0 con 0x7fe0e00729d0 2023-10-06T21:18:23.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.548+0000 7fe0e5d9c700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 0x7fe0e010b5e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8007f10 tx=0x7fe0d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.549+0000 7fe0d67fc700 1 -- 172.21.15.26:0/3359256893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800ddd0 con 0x7fe0e00729d0 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.549+0000 7fe0d67fc700 1 -- 172.21.15.26:0/3359256893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d800f040 con 0x7fe0e00729d0 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.549+0000 7fe0d67fc700 1 -- 172.21.15.26:0/3359256893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8005a90 con 0x7fe0e00729d0 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.549+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e010c160 con 0x7fe0e00729d0 2023-10-06T21:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.549+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e0112e10 con 0x7fe0e00729d0 2023-10-06T21:18:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.555+0000 7fe0d67fc700 1 -- 172.21.15.26:0/3359256893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe0d8004ad0 con 0x7fe0e00729d0 2023-10-06T21:18:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.557+0000 7fe0fb131b80 1 Processor -- start 2023-10-06T21:18:23.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 -- start start 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3115520 0x55cdf311fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf31203d0 0x55cdf31227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf307e400 con 0x55cdf3115520 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf306fbc0 con 0x55cdf31203d0 2023-10-06T21:18:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.558+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf306d4e0 con 0x55cdf3114c00 2023-10-06T21:18:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34352/0 (socket says 172.21.15.26:34352) 2023-10-06T21:18:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/1174537138 learned_addr learned my addr 172.21.15.26:0/1174537138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:23.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf31203d0 0x55cdf31227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0d67fc700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 0x7fe0cc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e559b700 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3115520 0x55cdf311fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/1174537138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf31203d0 msgr2=0x55cdf31227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf31203d0 0x55cdf31227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/1174537138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3115520 msgr2=0x55cdf311fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.559+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3115520 0x55cdf311fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/1174537138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf2e93d80 con 0x55cdf3114c00 2023-10-06T21:18:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0d67fc700 1 -- 172.21.15.26:0/3359256893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe0d80cc960 con 0x7fe0e00729d0 2023-10-06T21:18:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0015620 tx=0x7fe0d000c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31620a59f1a3c1f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0cb7fe700 1 -- 172.21.15.26:0/1174537138 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0015e40 con 0x55cdf3114c00 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0cb7fe700 1 -- 172.21.15.26:0/1174537138 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d0018930 con 0x55cdf3114c00 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.560+0000 7fe0cb7fe700 1 -- 172.21.15.26:0/1174537138 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0016770 con 0x55cdf3114c00 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0e5d9c700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 0x7fe0cc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 -- 172.21.15.26:0/1174537138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 msgr2=0x55cdf3114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0015620 tx=0x7fe0d000c210 comp rx=0 tx=0).stop 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 -- 172.21.15.26:0/1174537138 shutdown_connections 2023-10-06T21:18:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf31203d0 0x55cdf31227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf3114fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/1174537138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3115520 0x55cdf311fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.561+0000 7fe0fb131b80 1 -- 172.21.15.26:0/1174537138 >> 172.21.15.26:0/1174537138 conn(0x55cdf308a200 msgr2=0x55cdf3111080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.562+0000 7fe0fb131b80 1 -- 172.21.15.26:0/1174537138 shutdown_connections 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.562+0000 7fe0e5d9c700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 0x7fe0cc0a3b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80ca100 tx=0x7fe0d8004d80 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.562+0000 7fe0fb131b80 1 -- 172.21.15.26:0/1174537138 wait complete. 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.562+0000 7fe0fb131b80 1 -- start start 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.562+0000 7fe0fb131b80 1 Processor -- start 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf311d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3068dd0 con 0x55cdf31203d0 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf30056b0 con 0x55cdf3115520 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3005350 con 0x55cdf3114c00 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0fb131b80 10 client.0 _ll_register_callbacks cb 0x55cdf2e627a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.563+0000 7fe0e4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34354/0 (socket says 172.21.15.26:34354) 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/2265359622 learned_addr learned my addr 172.21.15.26:0/2265359622 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf311d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e559b700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 msgr2=0x55cdf311d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/2265359622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 msgr2=0x55cdf311d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf311d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.564+0000 7fe0e4d9a700 1 -- 172.21.15.26:0/2265359622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d0015300 con 0x55cdf3114c00 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.565+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0004790 tx=0x7fe0d0016370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47c1c1a2e457c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.565+0000 7fe0c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:18:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.567+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0018670 con 0x55cdf3114c00 2023-10-06T21:18:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.567+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe0d00166d0 con 0x55cdf3114c00 2023-10-06T21:18:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.568+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdf3108d50 con 0x55cdf3114c00 2023-10-06T21:18:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.569+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d00187d0 con 0x55cdf3114c00 2023-10-06T21:18:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.569+0000 7fe0fb131b80 10 client.26503 fetch_fsmap learned FSMap version 44 2023-10-06T21:18:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.569+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cdf2e7cc00 con 0x55cdf3114c00 2023-10-06T21:18:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.570+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdf3108d50 con 0x55cdf3114c00 2023-10-06T21:18:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe0d001f030 con 0x55cdf3114c00 2023-10-06T21:18:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:18:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 20 client.26503 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf312fc40 0x55cdf3132030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 4 client.26503 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:23.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdf3061db0 con 0x55cdf312fc40 2023-10-06T21:18:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.571+0000 7fe0e559b700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf312fc40 0x55cdf3132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.572+0000 7fe0c8ff9700 10 client.26503 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.572+0000 7fe0e559b700 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf312fc40 0x55cdf3132030 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3b0461904ddb67 server_cookie=9d97d673c0a3c6ba in_seq=0 out_seq=0 2023-10-06T21:18:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.573+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/2265359622 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cdf3061db0 con 0x55cdf312fc40 2023-10-06T21:18:23.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.573+0000 7fe0c8ff9700 10 client.26503 handle_command_reply: tid=0 2023-10-06T21:18:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.573+0000 7fe0fb131b80 1 client.26503 shutdown 2023-10-06T21:18:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.573+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 msgr2=0x55cdf311fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.573+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0004790 tx=0x7fe0d0016370 comp rx=0 tx=0).stop 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 shutdown_connections 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf311d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3114c00 0x55cdf311fdf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf312fc40 0x55cdf3132030 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/2265359622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 >> 172.21.15.26:0/2265359622 conn(0x55cdf308a200 msgr2=0x55cdf3110c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0c8ff9700 0 client.26503 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 shutdown_connections 2023-10-06T21:18:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 1 -- 172.21.15.26:0/2265359622 wait complete. 2023-10-06T21:18:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.574+0000 7fe0fb131b80 20 client.26503 trim_cache size 0 max 16384 2023-10-06T21:18:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.595+0000 7fe0fb131b80 1 Processor -- start 2023-10-06T21:18:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.595+0000 7fe0fb131b80 1 -- start start 2023-10-06T21:18:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3114c00 0x55cdf3117aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf31203d0 0x55cdf311cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf2ee98c0 con 0x55cdf3114c00 2023-10-06T21:18:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf310d860 con 0x55cdf3115520 2023-10-06T21:18:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3122ba0 con 0x55cdf31203d0 2023-10-06T21:18:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33554/0 (socket says 172.21.15.26:33554) 2023-10-06T21:18:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0e559b700 1 -- 172.21.15.26:0/3725559536 learned_addr learned my addr 172.21.15.26:0/3725559536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.596+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3114c00 0x55cdf3117aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf31203d0 0x55cdf311cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 -- 172.21.15.26:0/3725559536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf31203d0 msgr2=0x55cdf311cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf31203d0 0x55cdf311cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 -- 172.21.15.26:0/3725559536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3114c00 msgr2=0x55cdf3117aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3114c00 0x55cdf3117aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 -- 172.21.15.26:0/3725559536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf2eba1f0 con 0x55cdf3115520 2023-10-06T21:18:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.597+0000 7fe0e559b700 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc000c00 tx=0x7fe0dc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9434b05a4d7bad3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.598+0000 7fe0cbfff700 1 -- 172.21.15.26:0/3725559536 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc01d070 con 0x55cdf3115520 2023-10-06T21:18:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.598+0000 7fe0cbfff700 1 -- 172.21.15.26:0/3725559536 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0dc01e040 con 0x55cdf3115520 2023-10-06T21:18:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.598+0000 7fe0cbfff700 1 -- 172.21.15.26:0/3725559536 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00ba60 con 0x55cdf3115520 2023-10-06T21:18:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.599+0000 7fe0fb131b80 1 -- 172.21.15.26:0/3725559536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 msgr2=0x55cdf311c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.599+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc000c00 tx=0x7fe0dc009310 comp rx=0 tx=0).stop 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 -- 172.21.15.26:0/3725559536 shutdown_connections 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf31203d0 0x55cdf311cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf3114c00 0x55cdf3117aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/3725559536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3115520 0x55cdf311c850 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 -- 172.21.15.26:0/3725559536 >> 172.21.15.26:0/3725559536 conn(0x55cdf308a200 msgr2=0x55cdf3110250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.600+0000 7fe0fb131b80 1 -- 172.21.15.26:0/3725559536 shutdown_connections 2023-10-06T21:18:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.602+0000 7fe0fb131b80 1 -- 172.21.15.26:0/3725559536 wait complete. 2023-10-06T21:18:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.602+0000 7fe0fb131b80 1 -- start start 2023-10-06T21:18:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.602+0000 7fe0fb131b80 1 Processor -- start 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3114c00 0x55cdf31270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf3119ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf311a3f0 con 0x55cdf31203d0 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf311a560 con 0x55cdf3114c00 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3183a80 con 0x55cdf3115520 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0fb131b80 10 client.0 _ll_register_callbacks cb 0x7fe0d001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0e4d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3114c00 0x55cdf31270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0e559b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34382/0 (socket says 172.21.15.26:34382) 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.603+0000 7fe0e559b700 1 -- 172.21.15.26:0/453078884 learned_addr learned my addr 172.21.15.26:0/453078884 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf3119ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 -- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3114c00 msgr2=0x55cdf31270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3114c00 0x55cdf31270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 -- 172.21.15.26:0/453078884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 msgr2=0x55cdf3119ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf3119ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 -- 172.21.15.26:0/453078884 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e010e5d0 con 0x55cdf3115520 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0e559b700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005ad0 tx=0x7fe0dc0281f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e32ce3922371395d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.604+0000 7fe0c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc01d040 con 0x55cdf3115520 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0d0018930 con 0x55cdf3115520 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe0dc028390 con 0x55cdf3115520 2023-10-06T21:18:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0dc005320 con 0x55cdf3115520 2023-10-06T21:18:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0fb131b80 10 client.26509 fetch_fsmap learned FSMap version 44 2023-10-06T21:18:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55cdf2e92250 con 0x55cdf3115520 2023-10-06T21:18:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.605+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d0018930 con 0x55cdf3115520 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.606+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 44) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fe0dc003920 con 0x55cdf3115520 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 10 client.26509 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 20 client.26509 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf3184c90 0x55cdf3187080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 4 client.26509 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:18:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.608+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cdf3061ad0 con 0x55cdf3184c90 2023-10-06T21:18:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.609+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf3184c90 0x55cdf3187080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.610+0000 7fe0e4d9a700 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf3184c90 0x55cdf3187080 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e65e8f38fac6f6 server_cookie=704d9a06ac4194d1 in_seq=0 out_seq=0 2023-10-06T21:18:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.610+0000 7fe0c8ff9700 10 client.26509 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0c8ff9700 1 -- 172.21.15.26:0/453078884 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cdf3061ad0 con 0x55cdf3184c90 2023-10-06T21:18:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0c8ff9700 10 client.26509 handle_command_reply: tid=0 2023-10-06T21:18:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 client.26509 shutdown 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 msgr2=0x55cdf3119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005ad0 tx=0x7fe0dc0281f0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 shutdown_connections 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cdf31203d0 0x55cdf3119ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cdf3184c90 0x55cdf3187080 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdf3114c00 0x55cdf31270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 --2- 172.21.15.26:0/453078884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdf3115520 0x55cdf3119760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0c8ff9700 0 client.26509 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.627+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 >> 172.21.15.26:0/453078884 conn(0x55cdf308a200 msgr2=0x55cdf3110ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.628+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 shutdown_connections 2023-10-06T21:18:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.628+0000 7fe0fb131b80 1 -- 172.21.15.26:0/453078884 wait complete. 2023-10-06T21:18:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.628+0000 7fe0fb131b80 20 client.26509 trim_cache size 0 max 16384 2023-10-06T21:18:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:18:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:18:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:18:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:18:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 msgr2=0x7fe0cc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 0x7fe0cc0a3b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80ca100 tx=0x7fe0d8004d80 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 msgr2=0x7fe0e010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 0x7fe0e010b5e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8007f10 tx=0x7fe0d800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 shutdown_connections 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0cc0a16b0 0x7fe0cc0a3b70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0e00729d0 0x7fe0e010b5e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0e01010c0 0x7fe0e010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 --2- 172.21.15.26:0/3359256893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0e01061e0 0x7fe0e010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 >> 172.21.15.26:0/3359256893 conn(0x7fe0e00fc940 msgr2=0x7fe0e0104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 shutdown_connections 2023-10-06T21:18:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:23.629+0000 7fe0e77ff700 1 -- 172.21.15.26:0/3359256893 wait complete. 2023-10-06T21:18:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:24 smithi078 ceph-mon[134896]: pgmap v250: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-10-06T21:18:24.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:24 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:24.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:24 smithi167 ceph-mon[134494]: pgmap v250: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-10-06T21:18:24.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:24 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:24.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:24 smithi026 ceph-mon[98490]: pgmap v250: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 703 op/s 2023-10-06T21:18:24.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:24 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:26.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:26 smithi078 ceph-mon[134896]: pgmap v251: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 849 op/s 2023-10-06T21:18:26.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:26 smithi167 ceph-mon[134494]: pgmap v251: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 849 op/s 2023-10-06T21:18:26.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:26 smithi026 ceph-mon[98490]: pgmap v251: 97 pgs: 97 active+clean; 6.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 849 op/s 2023-10-06T21:18:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 465.0 s, 1451.8 tps, lat 69.349 ms stddev 121.493 2023-10-06T21:18:28.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:28 smithi167 ceph-mon[134494]: pgmap v252: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-10-06T21:18:28.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:28 smithi026 ceph-mon[98490]: pgmap v252: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-10-06T21:18:28.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:28 smithi078 ceph-mon[134896]: pgmap v252: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.88k op/s 2023-10-06T21:18:30.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:30 smithi167 ceph-mon[134494]: pgmap v253: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-10-06T21:18:30.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:30 smithi026 ceph-mon[98490]: pgmap v253: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-10-06T21:18:30.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:30 smithi078 ceph-mon[134896]: pgmap v253: 97 pgs: 97 active+clean; 6.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-10-06T21:18:31.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:31 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:31.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:31 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:31.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:31 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 470.0 s, 1680.9 tps, lat 59.569 ms stddev 65.914 2023-10-06T21:18:32.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:32 smithi167 ceph-mon[134494]: pgmap v254: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:18:32.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:32 smithi026 ceph-mon[98490]: pgmap v254: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:18:32.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:32 smithi078 ceph-mon[134896]: pgmap v254: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:18:33.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:18:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 -- 172.21.15.26:0/2687494963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481010c0 msgr2=0x7f58481014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 --2- 172.21.15.26:0/2687494963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481010c0 0x7f58481014a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f583800b0a0 tx=0x7f583800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f5845d9b700 1 --2- 172.21.15.26:0/2687494963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5848101a70 0x7f5848105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 -- 172.21.15.26:0/2687494963 shutdown_connections 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 --2- 172.21.15.26:0/2687494963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58481061e0 0x7f584810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 --2- 172.21.15.26:0/2687494963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481010c0 0x7f58481014a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 --2- 172.21.15.26:0/2687494963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5848101a70 0x7f5848105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.392+0000 7f584cdff700 1 -- 172.21.15.26:0/2687494963 >> 172.21.15.26:0/2687494963 conn(0x7f58480fc960 msgr2=0x7f58480fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.393+0000 7f584cdff700 1 -- 172.21.15.26:0/2687494963 shutdown_connections 2023-10-06T21:18:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.393+0000 7f584cdff700 1 -- 172.21.15.26:0/2687494963 wait complete. 2023-10-06T21:18:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.394+0000 7f584cdff700 1 Processor -- start 2023-10-06T21:18:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.394+0000 7f584cdff700 1 -- start start 2023-10-06T21:18:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58481010c0 0x7f5848196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 0x7f584819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819c380 con 0x7f584819b900 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819c4f0 con 0x7f58481061e0 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584cdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819c6b0 con 0x7f58481010c0 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5845d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5846d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 0x7f584819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f584659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58481010c0 0x7f5848196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5845d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33346/0 (socket says 172.21.15.26:33346) 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5845d9b700 1 -- 172.21.15.26:0/318531817 learned_addr learned my addr 172.21.15.26:0/318531817 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5846d9d700 1 -- 172.21.15.26:0/318531817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58481010c0 msgr2=0x7f5848196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5846d9d700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58481010c0 0x7f5848196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5846d9d700 1 -- 172.21.15.26:0/318531817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 msgr2=0x7f58481974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.395+0000 7f5846d9d700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.396+0000 7f5846d9d700 1 -- 172.21.15.26:0/318531817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009d00 con 0x7f584819b900 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.396+0000 7f5845d9b700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.396+0000 7f5846d9d700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 0x7f584819bd80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f584000b3f0 tx=0x7f584000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.396+0000 7f58377fe700 1 -- 172.21.15.26:0/318531817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840011630 con 0x7f584819b900 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.396+0000 7f58377fe700 1 -- 172.21.15.26:0/318531817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5840011c50 con 0x7f584819b900 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.397+0000 7f58377fe700 1 -- 172.21.15.26:0/318531817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840010cc0 con 0x7f584819b900 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.397+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848073140 con 0x7f584819b900 2023-10-06T21:18:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.397+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848073600 con 0x7f584819b900 2023-10-06T21:18:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.403+0000 7f58377fe700 1 -- 172.21.15.26:0/318531817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5840010380 con 0x7f584819b900 2023-10-06T21:18:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.406+0000 7f586070bb80 1 Processor -- start 2023-10-06T21:18:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.406+0000 7f58377fe700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 0x7f5830005ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.407+0000 7f58377fe700 1 -- 172.21.15.26:0/318531817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f58400cc840 con 0x7f584819b900 2023-10-06T21:18:33.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.407+0000 7f5845d9b700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 0x7f5830005ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.408+0000 7f5845d9b700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 0x7f5830005ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5848198570 tx=0x7f583c007480 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 -- start start 2023-10-06T21:18:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc542cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc4a3460 con 0x5597bc5428d0 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc494c20 con 0x5597bc54e0a0 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.409+0000 7f586070bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc492540 con 0x5597bc5431f0 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f584659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc542cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33348/0 (socket says 172.21.15.26:33348) 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 -- 172.21.15.26:0/2194456483 learned_addr learned my addr 172.21.15.26:0/2194456483 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:33.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5845d9b700 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 -- 172.21.15.26:0/2194456483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 msgr2=0x5597bc54db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 -- 172.21.15.26:0/2194456483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 msgr2=0x5597bc542cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc542cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 -- 172.21.15.26:0/2194456483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597bc2b8de0 con 0x5597bc54e0a0 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.410+0000 7f5846d9d700 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58400998b0 tx=0x7f5840099570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d00aec59ba32ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.411+0000 7f5834ff9700 1 -- 172.21.15.26:0/2194456483 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584009da70 con 0x5597bc54e0a0 2023-10-06T21:18:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.411+0000 7f5834ff9700 1 -- 172.21.15.26:0/2194456483 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58400a1cb0 con 0x5597bc54e0a0 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.411+0000 7f5834ff9700 1 -- 172.21.15.26:0/2194456483 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400b90b0 con 0x5597bc54e0a0 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 -- 172.21.15.26:0/2194456483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 msgr2=0x5597bc550490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58400998b0 tx=0x7f5840099570 comp rx=0 tx=0).stop 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 -- 172.21.15.26:0/2194456483 shutdown_connections 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc550490 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc542cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 --2- 172.21.15.26:0/2194456483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 -- 172.21.15.26:0/2194456483 >> 172.21.15.26:0/2194456483 conn(0x5597bc4af260 msgr2=0x5597bc536450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 -- 172.21.15.26:0/2194456483 shutdown_connections 2023-10-06T21:18:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.412+0000 7f586070bb80 1 -- 172.21.15.26:0/2194456483 wait complete. 2023-10-06T21:18:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.413+0000 7f586070bb80 1 -- start start 2023-10-06T21:18:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.413+0000 7f586070bb80 1 Processor -- start 2023-10-06T21:18:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc54af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc54b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc48de30 con 0x5597bc5428d0 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc42a710 con 0x5597bc5431f0 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc42a3b0 con 0x5597bc54e0a0 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f586070bb80 10 client.0 _ll_register_callbacks cb 0x5597bc287800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f5845d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f5845d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33364/0 (socket says 172.21.15.26:33364) 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.414+0000 7f5845d9b700 1 -- 172.21.15.26:0/1220921588 learned_addr learned my addr 172.21.15.26:0/1220921588 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f584659c700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc54af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 -- 172.21.15.26:0/1220921588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 msgr2=0x5597bc54b9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc54b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 -- 172.21.15.26:0/1220921588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 msgr2=0x5597bc54af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc54af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 -- 172.21.15.26:0/1220921588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848073600 con 0x5597bc5431f0 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f5845d9b700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f583c0119b0 tx=0x7f583c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194d6b676e575e06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.415+0000 7f582dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.416+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c01f840 con 0x5597bc5431f0 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.416+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f583c017070 con 0x5597bc5431f0 2023-10-06T21:18:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.416+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597bc52df10 con 0x5597bc5431f0 2023-10-06T21:18:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.417+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f583c01f9a0 con 0x5597bc5431f0 2023-10-06T21:18:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.417+0000 7f586070bb80 10 client.26597 fetch_fsmap learned FSMap version 45 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.417+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5597bc2a1c60 con 0x5597bc5431f0 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.418+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597bc52df10 con 0x5597bc5431f0 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.419+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f583c028d20 con 0x5597bc5431f0 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.419+0000 7f586070bb80 10 client.26597 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.419+0000 7f586070bb80 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.419+0000 7f586070bb80 20 client.26597 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.420+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc55dd30 0x5597bc560150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.420+0000 7f586070bb80 4 client.26597 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:33.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.420+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597bc486e10 con 0x5597bc55dd30 2023-10-06T21:18:33.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.420+0000 7f584659c700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc55dd30 0x5597bc560150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.421+0000 7f584659c700 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc55dd30 0x5597bc560150 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d931c47ebff338fd server_cookie=e68ff7ba51658866 in_seq=0 out_seq=0 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.421+0000 7f582dffb700 10 client.26597 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.421+0000 7f582dffb700 1 -- 172.21.15.26:0/1220921588 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5597bc486e10 con 0x5597bc55dd30 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.421+0000 7f582dffb700 10 client.26597 handle_command_reply: tid=0 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.421+0000 7f586070bb80 1 client.26597 shutdown 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 msgr2=0x5597bc54b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f583c0119b0 tx=0x7f583c0047c0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 shutdown_connections 2023-10-06T21:18:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc54b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc54af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc55dd30 0x5597bc560150 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 --2- 172.21.15.26:0/1220921588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc54b470 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 >> 172.21.15.26:0/1220921588 conn(0x5597bc4af260 msgr2=0x5597bc536040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f582dffb700 0 client.26597 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 shutdown_connections 2023-10-06T21:18:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 1 -- 172.21.15.26:0/1220921588 wait complete. 2023-10-06T21:18:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.422+0000 7f586070bb80 20 client.26597 trim_cache size 0 max 16384 2023-10-06T21:18:33.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.453+0000 7f586070bb80 1 Processor -- start 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.453+0000 7f586070bb80 1 -- start start 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc548200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc30e920 con 0x5597bc5428d0 2023-10-06T21:18:33.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc52dde0 con 0x5597bc54e0a0 2023-10-06T21:18:33.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.454+0000 7f586070bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc5405a0 con 0x5597bc5431f0 2023-10-06T21:18:33.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.455+0000 7f584659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.455+0000 7f584659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35020/0 (socket says 172.21.15.26:35020) 2023-10-06T21:18:33.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.455+0000 7f584659c700 1 -- 172.21.15.26:0/3315994134 learned_addr learned my addr 172.21.15.26:0/3315994134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:33.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.455+0000 7f5846d9d700 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc548200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 -- 172.21.15.26:0/3315994134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 msgr2=0x5597bc54b7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:18:33.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 -- 172.21.15.26:0/3315994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 msgr2=0x5597bc548200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc548200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 -- 172.21.15.26:0/3315994134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597bc2df250 con 0x5597bc5428d0 2023-10-06T21:18:33.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.456+0000 7f584659c700 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f583800d010 tx=0x7f58380099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cfb2fc8ca2b28d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.466+0000 7f58357fa700 1 -- 172.21.15.26:0/3315994134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583800e040 con 0x5597bc5428d0 2023-10-06T21:18:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.466+0000 7f58357fa700 1 -- 172.21.15.26:0/3315994134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5838003db0 con 0x5597bc5428d0 2023-10-06T21:18:33.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.466+0000 7f58357fa700 1 -- 172.21.15.26:0/3315994134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838023a90 con 0x5597bc5428d0 2023-10-06T21:18:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.469+0000 7f586070bb80 1 -- 172.21.15.26:0/3315994134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 msgr2=0x5597bc554be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.469+0000 7f586070bb80 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f583800d010 tx=0x7f58380099e0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.480+0000 7f586070bb80 1 -- 172.21.15.26:0/3315994134 shutdown_connections 2023-10-06T21:18:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.480+0000 7f586070bb80 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc54e0a0 0x5597bc548200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.480+0000 7f586070bb80 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc554be0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.480+0000 7f586070bb80 1 --2- 172.21.15.26:0/3315994134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc5431f0 0x5597bc54b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.480+0000 7f586070bb80 1 -- 172.21.15.26:0/3315994134 >> 172.21.15.26:0/3315994134 conn(0x5597bc4af260 msgr2=0x5597bc5462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.485+0000 7f586070bb80 1 -- 172.21.15.26:0/3315994134 shutdown_connections 2023-10-06T21:18:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.485+0000 7f586070bb80 1 -- 172.21.15.26:0/3315994134 wait complete. 2023-10-06T21:18:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.486+0000 7f586070bb80 1 -- start start 2023-10-06T21:18:33.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.486+0000 7f586070bb80 1 Processor -- start 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc5a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc5a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc5aa440 con 0x5597bc5428d0 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc5aa5b0 con 0x5597bc5431f0 2023-10-06T21:18:33.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597bc5aa6f0 con 0x5597bc54e0a0 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f586070bb80 10 client.0 _ll_register_callbacks cb 0x5597bc287800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f584659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f5845d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc5a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.487+0000 7f584659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35026/0 (socket says 172.21.15.26:35026) 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 -- 172.21.15.26:0/2089646776 learned_addr learned my addr 172.21.15.26:0/2089646776 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 -- 172.21.15.26:0/2089646776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 msgr2=0x5597bc5a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc5a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 -- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 msgr2=0x5597bc5a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:33.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc5a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.488+0000 7f584659c700 1 -- 172.21.15.26:0/2089646776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584810e5d0 con 0x5597bc5428d0 2023-10-06T21:18:33.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.489+0000 7f584659c700 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f583800b070 tx=0x7f58380090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103eadc3c509d8f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.489+0000 7f582dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.489+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583800e070 con 0x5597bc5428d0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.489+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f58380076b0 con 0x5597bc5428d0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.489+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58400a1cb0 con 0x5597bc5428d0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.490+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5838007870 con 0x5597bc5428d0 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.490+0000 7f586070bb80 10 client.16332 fetch_fsmap learned FSMap version 45 2023-10-06T21:18:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.490+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5597bc2b72b0 con 0x5597bc5428d0 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.490+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5838039020 con 0x5597bc5428d0 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 45) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f583802a030 con 0x5597bc5428d0 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 10 client.16332 fetch_fsmap finished waiting for FSMap version 45 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 20 client.16332 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc538e10 0x5597bc53b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 4 client.16332 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:18:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597bc486b30 con 0x5597bc538e10 2023-10-06T21:18:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.491+0000 7f5845d9b700 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc538e10 0x5597bc53b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.494+0000 7f5845d9b700 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc538e10 0x5597bc53b230 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbecc3983a8b05a9 server_cookie=8298fe4c8a5cf91e in_seq=0 out_seq=0 2023-10-06T21:18:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.494+0000 7f582dffb700 10 client.16332 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.495+0000 7f582dffb700 1 -- 172.21.15.26:0/2089646776 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5597bc486b30 con 0x5597bc538e10 2023-10-06T21:18:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.495+0000 7f582dffb700 10 client.16332 handle_command_reply: tid=0 2023-10-06T21:18:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.495+0000 7f586070bb80 1 client.16332 shutdown 2023-10-06T21:18:33.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 msgr2=0x5597bc5ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f583800b070 tx=0x7f58380090f0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 shutdown_connections 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597bc54e0a0 0x5597bc5a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597bc5428d0 0x5597bc5ab1c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5597bc538e10 0x5597bc53b230 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.496+0000 7f586070bb80 1 --2- 172.21.15.26:0/2089646776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597bc5431f0 0x5597bc5a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.497+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 >> 172.21.15.26:0/2089646776 conn(0x5597bc4af260 msgr2=0x5597bc54b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.497+0000 7f582dffb700 0 client.16332 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.497+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 shutdown_connections 2023-10-06T21:18:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.497+0000 7f586070bb80 1 -- 172.21.15.26:0/2089646776 wait complete. 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.499+0000 7f586070bb80 20 client.16332 trim_cache size 0 max 16384 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:18:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:18:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:18:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:18:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:18:33.503 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:18:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.502+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 msgr2=0x7f5830005ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.502+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 0x7f5830005ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5848198570 tx=0x7f583c007480 comp rx=0 tx=0).stop 2023-10-06T21:18:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.502+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 msgr2=0x7f584819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.502+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 0x7f584819bd80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f584000b3f0 tx=0x7f584000b700 comp rx=0 tx=0).stop 2023-10-06T21:18:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 shutdown_connections 2023-10-06T21:18:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5830003a20 0x7f5830005ee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f584819b900 0x7f584819bd80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58481010c0 0x7f5848196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 --2- 172.21.15.26:0/318531817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58481061e0 0x7f58481974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.505+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 >> 172.21.15.26:0/318531817 conn(0x7f58480fc960 msgr2=0x7f5848104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.506+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 shutdown_connections 2023-10-06T21:18:33.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:18:33.507+0000 7f584cdff700 1 -- 172.21.15.26:0/318531817 wait complete. 2023-10-06T21:18:33.538 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:18:34.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:34 smithi078 ceph-mon[134896]: pgmap v255: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:34.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:34 smithi167 ceph-mon[134494]: pgmap v255: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:34.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:34 smithi026 ceph-mon[98490]: pgmap v255: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:36.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:36 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:36.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:36 smithi167 ceph-mon[134494]: pgmap v256: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:36 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:36.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:36 smithi026 ceph-mon[98490]: pgmap v256: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:36.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:36 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:36.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:36 smithi078 ceph-mon[134896]: pgmap v256: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 -- 172.21.15.26:0/3059719500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f810b670 msgr2=0x7f57f8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 --2- 172.21.15.26:0/3059719500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f810b670 0x7f57f8107eb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b0a0 tx=0x7f57f400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 -- 172.21.15.26:0/3059719500 shutdown_connections 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 --2- 172.21.15.26:0/3059719500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 --2- 172.21.15.26:0/3059719500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f810b670 0x7f57f8107eb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 --2- 172.21.15.26:0/3059719500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 -- 172.21.15.26:0/3059719500 >> 172.21.15.26:0/3059719500 conn(0x7f57f806d3e0 msgr2=0x7f57f806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 -- 172.21.15.26:0/3059719500 shutdown_connections 2023-10-06T21:18:36.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.838+0000 7f58003ff700 1 -- 172.21.15.26:0/3059719500 wait complete. 2023-10-06T21:18:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.839+0000 7f58003ff700 1 Processor -- start 2023-10-06T21:18:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 -- start start 2023-10-06T21:18:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f81a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81a8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f81a1980 0x7f57f81a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81a2400 con 0x7f57f8115fe0 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81a2540 con 0x7f57f81a1980 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.840+0000 7f58003ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81a2700 con 0x7f57f8108480 2023-10-06T21:18:36.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fd99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fd99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81a8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35034/0 (socket says 172.21.15.26:35034) 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fd99a700 1 -- 172.21.15.26:0/670589198 learned_addr learned my addr 172.21.15.26:0/670589198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe99c700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f81a1980 0x7f57f81a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f81a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 -- 172.21.15.26:0/670589198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f81a1980 msgr2=0x7f57f81a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f81a1980 0x7f57f81a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 -- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 msgr2=0x7f57f81a8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81a8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.841+0000 7f57fe19b700 1 -- 172.21.15.26:0/670589198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009d00 con 0x7f57f8108480 2023-10-06T21:18:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f57fe19b700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f81a7b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b070 tx=0x7f57f4007b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f57ef7fe700 1 -- 172.21.15.26:0/670589198 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4020070 con 0x7f57f8108480 2023-10-06T21:18:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f81ae930 con 0x7f57f8108480 2023-10-06T21:18:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f57ef7fe700 1 -- 172.21.15.26:0/670589198 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f4004d20 con 0x7f57f8108480 2023-10-06T21:18:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f57ef7fe700 1 -- 172.21.15.26:0/670589198 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400e040 con 0x7f57f8108480 2023-10-06T21:18:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.842+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f81aee80 con 0x7f57f8108480 2023-10-06T21:18:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.849+0000 7f5813d39b80 1 Processor -- start 2023-10-06T21:18:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.849+0000 7f57ef7fe700 1 -- 172.21.15.26:0/670589198 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f57f4005030 con 0x7f57f8108480 2023-10-06T21:18:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.852+0000 7f57ef7fe700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 0x7f57e40a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.852+0000 7f57ef7fe700 1 -- 172.21.15.26:0/670589198 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f57f40cda80 con 0x7f57f8108480 2023-10-06T21:18:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.853+0000 7f5813d39b80 1 -- start start 2023-10-06T21:18:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.853+0000 7f57fe99c700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 0x7f57e40a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.853+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.853+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.853+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f5813d39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560578c4400 con 0x556057963d30 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f5813d39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560578b5bc0 con 0x556057963410 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f5813d39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560578b34e0 con 0x55605796ebe0 2023-10-06T21:18:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51234/0 (socket says 172.21.15.26:51234) 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe99c700 1 -- 172.21.15.26:0/3596233509 learned_addr learned my addr 172.21.15.26:0/3596233509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fd99a700 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33404/0 (socket says 172.21.15.26:33404) 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe99c700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 0x7f57e40a3bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57f806a250 tx=0x7f57f0009110 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 -- 172.21.15.26:0/3596233509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 msgr2=0x556057970fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 -- 172.21.15.26:0/3596233509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 msgr2=0x55605796e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe19b700 1 -- 172.21.15.26:0/3596233509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560576d9d80 con 0x556057963410 2023-10-06T21:18:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.854+0000 7f57fe99c700 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.855+0000 7f57fe19b700 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cb1f0 tx=0x7f57f400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5df3a5bed674e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.855+0000 7f57ecbfe700 1 -- 172.21.15.26:0/3596233509 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f409b210 con 0x556057963410 2023-10-06T21:18:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.855+0000 7f57ecbfe700 1 -- 172.21.15.26:0/3596233509 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f409ab10 con 0x556057963410 2023-10-06T21:18:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.855+0000 7f57ecbfe700 1 -- 172.21.15.26:0/3596233509 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019570 con 0x556057963410 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- 172.21.15.26:0/3596233509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 msgr2=0x5560579637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cb1f0 tx=0x7f57f400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- 172.21.15.26:0/3596233509 shutdown_connections 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057970fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x5560579637f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 --2- 172.21.15.26:0/3596233509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- 172.21.15.26:0/3596233509 >> 172.21.15.26:0/3596233509 conn(0x5560578d0200 msgr2=0x556057956ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- 172.21.15.26:0/3596233509 shutdown_connections 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- 172.21.15.26:0/3596233509 wait complete. 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.856+0000 7f5813d39b80 1 -- start start 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 Processor -- start 2023-10-06T21:18:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560578aedd0 con 0x556057963d30 2023-10-06T21:18:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605784b6b0 con 0x55605796ebe0 2023-10-06T21:18:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.857+0000 7f5813d39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605784b350 con 0x556057963410 2023-10-06T21:18:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f5813d39b80 10 client.0 _ll_register_callbacks cb 0x5560576a87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51242/0 (socket says 172.21.15.26:51242) 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 -- 172.21.15.26:0/358274945 learned_addr learned my addr 172.21.15.26:0/358274945 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fd99a700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe99c700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 -- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 msgr2=0x55605796fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 -- 172.21.15.26:0/358274945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 msgr2=0x55605796f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 -- 172.21.15.26:0/358274945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f000e700 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57fe19b700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57f40a28e0 tx=0x7f57f4019580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b5a3332507af64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f409ec60 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f57f409be40 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.858+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55605794ed50 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f40a2e10 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 10 client.26527 fetch_fsmap learned FSMap version 46 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f57f81aee80 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f407f080 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 46) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f57f409a100 con 0x556057963410 2023-10-06T21:18:36.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 10 client.26527 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 20 client.26527 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55605797dee0 0x556057980300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 4 client.26527 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560578a7db0 con 0x55605797dee0 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.859+0000 7f57fd99a700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55605797dee0 0x556057980300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.860+0000 7f57fd99a700 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55605797dee0 0x556057980300 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f63bbd67c2a41c server_cookie=2c6b77a1d5492bb1 in_seq=0 out_seq=0 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.860+0000 7f57e1ffb700 10 client.26527 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f57e1ffb700 1 -- 172.21.15.26:0/358274945 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560578a7db0 con 0x55605797dee0 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f57e1ffb700 10 client.26527 handle_command_reply: tid=0 2023-10-06T21:18:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 client.26527 shutdown 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 msgr2=0x55605796efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57f40a28e0 tx=0x7f57f4019580 comp rx=0 tx=0).stop 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 shutdown_connections 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55605797dee0 0x556057980300 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x55605796efc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 --2- 172.21.15.26:0/358274945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f57e1ffb700 0 client.26527 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 >> 172.21.15.26:0/358274945 conn(0x5560578d0200 msgr2=0x556057956be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 shutdown_connections 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.861+0000 7f5813d39b80 1 -- 172.21.15.26:0/358274945 wait complete. 2023-10-06T21:18:36.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.862+0000 7f5813d39b80 20 client.26527 trim_cache size 0 max 16384 2023-10-06T21:18:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 Processor -- start 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 -- start start 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x556057979730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605772f8c0 con 0x556057963d30 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605796e0c0 con 0x55605796ebe0 2023-10-06T21:18:36.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.877+0000 7f5813d39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55605796e2c0 con 0x556057963410 2023-10-06T21:18:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fd99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33422/0 (socket says 172.21.15.26:33422) 2023-10-06T21:18:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 -- 172.21.15.26:0/1015987830 learned_addr learned my addr 172.21.15.26:0/1015987830 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:36.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe19b700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x556057979730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 -- 172.21.15.26:0/1015987830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 msgr2=0x556057979730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x556057979730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 -- 172.21.15.26:0/1015987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 msgr2=0x55605796aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe99c700 1 -- 172.21.15.26:0/1015987830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560576c2c00 con 0x55605796ebe0 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fd99a700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.878+0000 7f57fe19b700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x556057979730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.879+0000 7f57fe99c700 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f57f000ec60 tx=0x7f57f0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5484ea3af2494359 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.879+0000 7f57ed3ff700 1 -- 172.21.15.26:0/1015987830 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0015710 con 0x55605796ebe0 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.879+0000 7f57ed3ff700 1 -- 172.21.15.26:0/1015987830 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f0015d30 con 0x55605796ebe0 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.879+0000 7f57ed3ff700 1 -- 172.21.15.26:0/1015987830 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0020430 con 0x55605796ebe0 2023-10-06T21:18:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 -- 172.21.15.26:0/1015987830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 msgr2=0x55605796b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f57f000ec60 tx=0x7f57f0019000 comp rx=0 tx=0).stop 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 -- 172.21.15.26:0/1015987830 shutdown_connections 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55605796ebe0 0x55605796b3e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556057963410 0x556057979730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 --2- 172.21.15.26:0/1015987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x55605796aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 -- 172.21.15.26:0/1015987830 >> 172.21.15.26:0/1015987830 conn(0x5560578d0200 msgr2=0x556057968f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 -- 172.21.15.26:0/1015987830 shutdown_connections 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.880+0000 7f5813d39b80 1 -- 172.21.15.26:0/1015987830 wait complete. 2023-10-06T21:18:36.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 -- start start 2023-10-06T21:18:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 Processor -- start 2023-10-06T21:18:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x556057973da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057971d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556057972320 con 0x556057963d30 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556057972490 con 0x556057963410 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556057972650 con 0x55605796ebe0 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f5813d39b80 10 client.0 _ll_register_callbacks cb 0x5560576a87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.881+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33424/0 (socket says 172.21.15.26:33424) 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 -- 172.21.15.26:0/739046599 learned_addr learned my addr 172.21.15.26:0/739046599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe99c700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057971d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fd99a700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x556057973da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 -- 172.21.15.26:0/739046599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 msgr2=0x556057971d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057971d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 -- 172.21.15.26:0/739046599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 msgr2=0x556057973da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x556057973da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 -- 172.21.15.26:0/739046599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f81aee80 con 0x556057963410 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fd99a700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x556057973da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe99c700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057971d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57fe19b700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f57f409a810 tx=0x7f57f4004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f79424f85fd3199 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4009960 con 0x556057963410 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57f409ab10 con 0x556057963410 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.882+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f57f409be40 con 0x556057963410 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f40a2bf0 con 0x556057963410 2023-10-06T21:18:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 10 client.26621 fetch_fsmap learned FSMap version 46 2023-10-06T21:18:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f57e8009940 con 0x556057963410 2023-10-06T21:18:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f409ab10 con 0x556057963410 2023-10-06T21:18:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f57f409a090 con 0x556057963410 2023-10-06T21:18:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 10 client.26621 fetch_fsmap finished waiting for FSMap version 46 2023-10-06T21:18:36.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 20 client.26621 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560579d2980 0x5560579d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 4 client.26621 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.883+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560578a7ad0 con 0x5560579d2980 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.884+0000 7f57fd99a700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560579d2980 0x5560579d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.884+0000 7f57fd99a700 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560579d2980 0x5560579d4da0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d640fa72aef4aa81 server_cookie=a28f44814d7a8439 in_seq=0 out_seq=0 2023-10-06T21:18:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.884+0000 7f57e1ffb700 10 client.26621 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.898+0000 7f57e1ffb700 1 -- 172.21.15.26:0/739046599 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560578a7ad0 con 0x5560579d2980 2023-10-06T21:18:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.898+0000 7f57e1ffb700 10 client.26621 handle_command_reply: tid=0 2023-10-06T21:18:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.898+0000 7f5813d39b80 1 client.26621 shutdown 2023-10-06T21:18:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.898+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 msgr2=0x556057973860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f57f409a810 tx=0x7f57f4004e90 comp rx=0 tx=0).stop 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 shutdown_connections 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55605796ebe0 0x556057971d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560579d2980 0x5560579d4da0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556057963410 0x556057973860 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 --2- 172.21.15.26:0/739046599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556057963d30 0x556057973da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 >> 172.21.15.26:0/739046599 conn(0x5560578d0200 msgr2=0x55605796c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f57e1ffb700 0 client.26621 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 shutdown_connections 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 1 -- 172.21.15.26:0/739046599 wait complete. 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.899+0000 7f5813d39b80 20 client.26621 trim_cache size 0 max 16384 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:18:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 msgr2=0x7f57e40a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 0x7f57e40a3bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57f806a250 tx=0x7f57f0009110 comp rx=0 tx=0).stop 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 msgr2=0x7f57f81a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f81a7b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f57f400b070 tx=0x7f57f4007b60 comp rx=0 tx=0).stop 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 shutdown_connections 2023-10-06T21:18:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57f81a1980 0x7f57f81a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57f8108480 0x7f57f81a7b20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f57e40a1730 0x7f57e40a3bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 --2- 172.21.15.26:0/670589198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f8115fe0 0x7f57f81a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 >> 172.21.15.26:0/670589198 conn(0x7f57f806d3e0 msgr2=0x7f57f8117580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 shutdown_connections 2023-10-06T21:18:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:36.900+0000 7f58003ff700 1 -- 172.21.15.26:0/670589198 wait complete. 2023-10-06T21:18:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 475.0 s, 1152.0 tps, lat 83.748 ms stddev 94.992 2023-10-06T21:18:38.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:38 smithi078 ceph-mon[134896]: pgmap v257: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-10-06T21:18:38.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:38 smithi167 ceph-mon[134494]: pgmap v257: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-10-06T21:18:38.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:38 smithi026 ceph-mon[98490]: pgmap v257: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2023-10-06T21:18:39.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:39 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:39.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:39 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:39.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:39 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:18:40.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:40 smithi078 ceph-mon[134896]: pgmap v258: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-06T21:18:40.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:40 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:40.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:40 smithi167 ceph-mon[134494]: pgmap v258: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-06T21:18:40.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:40 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:40.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:40 smithi026 ceph-mon[98490]: pgmap v258: 97 pgs: 97 active+clean; 6.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-06T21:18:40.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:40 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:18:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:41 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:41 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:41.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:41 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:18:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:41 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:41 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:41 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:18:41.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:41 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:18:41.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:41 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:18:41.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:41 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:18:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 480.0 s, 1010.4 tps, lat 102.161 ms stddev 118.245 2023-10-06T21:18:42.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:42 smithi078 ceph-mon[134896]: pgmap v259: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-10-06T21:18:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:42 smithi167 ceph-mon[134494]: pgmap v259: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-10-06T21:18:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:42 smithi026 ceph-mon[98490]: pgmap v259: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 862 op/s 2023-10-06T21:18:44.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:44 smithi167 ceph-mon[134494]: pgmap v260: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 690 op/s 2023-10-06T21:18:44.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:44 smithi026 ceph-mon[98490]: pgmap v260: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 690 op/s 2023-10-06T21:18:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:44 smithi078 ceph-mon[134896]: pgmap v260: 97 pgs: 97 active+clean; 6.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 690 op/s 2023-10-06T21:18:46.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:46 smithi078 ceph-mon[134896]: pgmap v261: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-06T21:18:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:46 smithi167 ceph-mon[134494]: pgmap v261: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-06T21:18:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:46 smithi026 ceph-mon[98490]: pgmap v261: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 699 op/s 2023-10-06T21:18:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 485.0 s, 1389.2 tps, lat 72.361 ms stddev 93.147 2023-10-06T21:18:48.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:48 smithi167 ceph-mon[134494]: pgmap v262: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 947 op/s 2023-10-06T21:18:48.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:48 smithi026 ceph-mon[98490]: pgmap v262: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 947 op/s 2023-10-06T21:18:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:48 smithi078 ceph-mon[134896]: pgmap v262: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 947 op/s 2023-10-06T21:18:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.111+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/520463796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68105e20 msgr2=0x7f6e680682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.111+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/520463796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68105e20 0x7f6e680682d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6e580098f0 tx=0x7f6e58009c00 comp rx=0 tx=0).stop 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/520463796 shutdown_connections 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/520463796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 0x7f6e68103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/520463796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68105e20 0x7f6e680682d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/520463796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e68068810 0x7f6e68068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/520463796 >> 172.21.15.26:0/520463796 conn(0x7f6e68076b70 msgr2=0x7f6e68076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/520463796 shutdown_connections 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.112+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/520463796 wait complete. 2023-10-06T21:18:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.113+0000 7f6e6e7ff700 1 Processor -- start 2023-10-06T21:18:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.113+0000 7f6e6e7ff700 1 -- start start 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68068810 0x7f6e681a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 0x7f6e681a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6819e070 con 0x7f6e681038b0 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6819e1b0 con 0x7f6e6819d5f0 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6e7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6819e370 con 0x7f6e68068810 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6cd9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6cd9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45054/0 (socket says 172.21.15.26:45054) 2023-10-06T21:18:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e6cd9c700 1 -- 172.21.15.26:0/1575819453 learned_addr learned my addr 172.21.15.26:0/1575819453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68068810 0x7f6e681a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.114+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 0x7f6e681a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 -- 172.21.15.26:0/1575819453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68068810 msgr2=0x7f6e681a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68068810 0x7f6e681a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 -- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 msgr2=0x7f6e681a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 0x7f6e681a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 -- 172.21.15.26:0/1575819453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e58009520 con 0x7f6e6819d5f0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6cd9c700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e657fa700 1 -- 172.21.15.26:0/1575819453 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50004a30 con 0x7f6e6819d5f0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e657fa700 1 -- 172.21.15.26:0/1575819453 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e500054e0 con 0x7f6e6819d5f0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e657fa700 1 -- 172.21.15.26:0/1575819453 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50010bc0 con 0x7f6e6819d5f0 2023-10-06T21:18:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.115+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e681aa590 con 0x7f6e6819d5f0 2023-10-06T21:18:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.116+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e681aaa50 con 0x7f6e6819d5f0 2023-10-06T21:18:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.125+0000 7f6e82035b80 1 Processor -- start 2023-10-06T21:18:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.126+0000 7f6e657fa700 1 -- 172.21.15.26:0/1575819453 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6e50005000 con 0x7f6e6819d5f0 2023-10-06T21:18:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.126+0000 7f6e82035b80 1 -- start start 2023-10-06T21:18:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fab8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fa22400 con 0x56063fac43d0 2023-10-06T21:18:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fa13bc0 con 0x56063fab8c00 2023-10-06T21:18:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.127+0000 7f6e82035b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fa114e0 con 0x56063fab9520 2023-10-06T21:18:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42964/0 (socket says 172.21.15.26:42964) 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 -- 172.21.15.26:0/234780854 learned_addr learned my addr 172.21.15.26:0/234780854 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e677fe700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fab8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e677fe700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 -- 172.21.15.26:0/234780854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 msgr2=0x56063fab8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fab8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 -- 172.21.15.26:0/234780854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 msgr2=0x56063fac67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 -- 172.21.15.26:0/234780854 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56063f837d80 con 0x56063fab9520 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e677fe700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.128+0000 7f6e67fff700 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009810 tx=0x7f6e58005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5541dd6fd95bb06d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e5e7fc700 1 -- 172.21.15.26:0/234780854 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800f660 con 0x56063fab9520 2023-10-06T21:18:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e657fa700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 0x7f6e540a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e5e7fc700 1 -- 172.21.15.26:0/234780854 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e5801d040 con 0x56063fab9520 2023-10-06T21:18:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e5e7fc700 1 -- 172.21.15.26:0/234780854 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58005430 con 0x56063fab9520 2023-10-06T21:18:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e657fa700 1 -- 172.21.15.26:0/1575819453 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e50014070 con 0x7f6e6819d5f0 2023-10-06T21:18:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.129+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 0x7f6e540a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- 172.21.15.26:0/234780854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 msgr2=0x56063fac3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009810 tx=0x7f6e58005dc0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 0x7f6e540a3ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e6819ea60 tx=0x7f6e6000b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- 172.21.15.26:0/234780854 shutdown_connections 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fab8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 --2- 172.21.15.26:0/234780854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac3e90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- 172.21.15.26:0/234780854 >> 172.21.15.26:0/234780854 conn(0x56063fa2e200 msgr2=0x56063fab5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- 172.21.15.26:0/234780854 shutdown_connections 2023-10-06T21:18:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- 172.21.15.26:0/234780854 wait complete. 2023-10-06T21:18:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.130+0000 7f6e82035b80 1 -- start start 2023-10-06T21:18:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.131+0000 7f6e82035b80 1 Processor -- start 2023-10-06T21:18:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fac3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fa0cdd0 con 0x56063fac43d0 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f9a96b0 con 0x56063fab8c00 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f9a9350 con 0x56063fab9520 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e82035b80 10 client.0 _ll_register_callbacks cb 0x56063f8067a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e67fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fac3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42976/0 (socket says 172.21.15.26:42976) 2023-10-06T21:18:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e677fe700 1 -- 172.21.15.26:0/544216478 learned_addr learned my addr 172.21.15.26:0/544216478 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.132+0000 7f6e6cd9c700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 -- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 msgr2=0x56063fac3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fac3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 -- 172.21.15.26:0/544216478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 msgr2=0x56063fac15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 -- 172.21.15.26:0/544216478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e68114dd0 con 0x56063fab9520 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e677fe700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e60003fb0 tx=0x7f6e60004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb6def436f71273 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.133+0000 7f6e477fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000e040 con 0x56063fab9520 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56063faacd50 con 0x56063fab9520 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e60008e40 con 0x56063fab9520 2023-10-06T21:18:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e60018ee0 con 0x56063fab9520 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e82035b80 10 client.26545 fetch_fsmap learned FSMap version 47 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.134+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56063f820c00 con 0x56063fab9520 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.135+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56063faacd50 con 0x56063fab9520 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.135+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e600257c0 con 0x56063fab9520 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.135+0000 7f6e82035b80 10 client.26545 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.135+0000 7f6e82035b80 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e82035b80 20 client.26545 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fad3f10 0x56063fad6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e82035b80 4 client.26545 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56063fa05db0 con 0x56063fad3f10 2023-10-06T21:18:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e67fff700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fad3f10 0x56063fad6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e67fff700 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fad3f10 0x56063fad6330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fdde670e336532 server_cookie=e3622872751603f9 in_seq=0 out_seq=0 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.136+0000 7f6e477fe700 10 client.26545 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.137+0000 7f6e477fe700 1 -- 172.21.15.26:0/544216478 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56063fa05db0 con 0x56063fad3f10 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.137+0000 7f6e477fe700 10 client.26545 handle_command_reply: tid=0 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.137+0000 7f6e82035b80 1 client.26545 shutdown 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 msgr2=0x56063fac0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e60003fb0 tx=0x7f6e60004b40 comp rx=0 tx=0).stop 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 shutdown_connections 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fad3f10 0x56063fad6330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab8c00 0x56063fac3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 --2- 172.21.15.26:0/544216478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab9520 0x56063fac0fd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 >> 172.21.15.26:0/544216478 conn(0x56063fa2e200 msgr2=0x56063fab4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e477fe700 0 client.26545 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 shutdown_connections 2023-10-06T21:18:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 1 -- 172.21.15.26:0/544216478 wait complete. 2023-10-06T21:18:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.138+0000 7f6e82035b80 20 client.26545 trim_cache size 0 max 16384 2023-10-06T21:18:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.165+0000 7f6e82035b80 1 Processor -- start 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 -- start start 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063fac67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fac07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063f88d8c0 con 0x56063fac43d0 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fab9b60 con 0x56063fab9520 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.166+0000 7f6e82035b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063facfd80 con 0x56063fab8c00 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e6cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e6cd9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33316/0 (socket says 172.21.15.26:33316) 2023-10-06T21:18:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e6cd9c700 1 -- 172.21.15.26:0/1485684096 learned_addr learned my addr 172.21.15.26:0/1485684096 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fac07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063fac67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e67fff700 1 -- 172.21.15.26:0/1485684096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 msgr2=0x56063fac07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.167+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fac07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e67fff700 1 -- 172.21.15.26:0/1485684096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 msgr2=0x56063fac38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e67fff700 1 -- 172.21.15.26:0/1485684096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56063f85e1f0 con 0x56063fab8c00 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e6cd9c700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fac07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:18:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.168+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063fac67c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c010 tx=0x7f6e58009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634b12d31e571816 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.169+0000 7f6e5effd700 1 -- 172.21.15.26:0/1485684096 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800f5c0 con 0x56063fab8c00 2023-10-06T21:18:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.169+0000 7f6e5effd700 1 -- 172.21.15.26:0/1485684096 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e5802e110 con 0x56063fab8c00 2023-10-06T21:18:50.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.169+0000 7f6e5effd700 1 -- 172.21.15.26:0/1485684096 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58005430 con 0x56063fab8c00 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 -- 172.21.15.26:0/1485684096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 msgr2=0x56063fac67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063fac67c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c010 tx=0x7f6e58009f50 comp rx=0 tx=0).stop 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 -- 172.21.15.26:0/1485684096 shutdown_connections 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fac38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063fac67c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1485684096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fac07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 -- 172.21.15.26:0/1485684096 >> 172.21.15.26:0/1485684096 conn(0x56063fa2e200 msgr2=0x56063facf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 -- 172.21.15.26:0/1485684096 shutdown_connections 2023-10-06T21:18:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.171+0000 7f6e82035b80 1 -- 172.21.15.26:0/1485684096 wait complete. 2023-10-06T21:18:50.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.172+0000 7f6e82035b80 1 -- start start 2023-10-06T21:18:50.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.172+0000 7f6e82035b80 1 Processor -- start 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063facaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fb237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fb1feb0 con 0x56063fac43d0 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fb20020 con 0x56063fab9520 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56063fb201e0 con 0x56063fab8c00 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e82035b80 10 client.0 _ll_register_callbacks cb 0x7f6e60008d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:18:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45086/0 (socket says 172.21.15.26:45086) 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e677fe700 1 -- 172.21.15.26:0/1656355662 learned_addr learned my addr 172.21.15.26:0/1656355662 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.173+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063facaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.174+0000 7f6e677fe700 1 -- 172.21.15.26:0/1656355662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 msgr2=0x56063facaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.174+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063facaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.174+0000 7f6e677fe700 1 -- 172.21.15.26:0/1656355662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 msgr2=0x56063fb237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.174+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fb237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.174+0000 7f6e677fe700 1 -- 172.21.15.26:0/1656355662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56063f820c00 con 0x56063fab9520 2023-10-06T21:18:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.175+0000 7f6e677fe700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009540 tx=0x7f6e600186c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4f25e9234a9476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:18:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.175+0000 7f6e477fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:18:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.176+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000e050 con 0x56063fab9520 2023-10-06T21:18:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.176+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e60018e70 con 0x56063fab9520 2023-10-06T21:18:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.177+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e5801d040 con 0x56063fab9520 2023-10-06T21:18:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.178+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e60025850 con 0x56063fab9520 2023-10-06T21:18:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.178+0000 7f6e82035b80 10 client.26639 fetch_fsmap learned FSMap version 47 2023-10-06T21:18:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.178+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f6e681aaa50 con 0x56063fab9520 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e60048030 con 0x56063fab9520 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6e600040b0 con 0x56063fab9520 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 10 client.26639 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 20 client.26639 populate_metadata read hostname 'smithi026' 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fb39930 0x56063fb3bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 4 client.26639 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:18:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.179+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56063fa05ad0 con 0x56063fb39930 2023-10-06T21:18:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.181+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fb39930 0x56063fb3bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:18:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.181+0000 7f6e67fff700 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fb39930 0x56063fb3bd50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16f432199863ef7 server_cookie=bae2cbfc13f3b7bb in_seq=0 out_seq=0 2023-10-06T21:18:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.181+0000 7f6e477fe700 10 client.26639 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.278+0000 7f6e477fe700 1 -- 172.21.15.26:0/1656355662 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56063fa05ad0 con 0x56063fb39930 2023-10-06T21:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.278+0000 7f6e477fe700 10 client.26639 handle_command_reply: tid=0 2023-10-06T21:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.278+0000 7f6e82035b80 1 client.26639 shutdown 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 msgr2=0x56063fb23290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009540 tx=0x7f6e600186c0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 shutdown_connections 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56063fac43d0 0x56063fb237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56063fab8c00 0x56063facaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56063fb39930 0x56063fb3bd50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 --2- 172.21.15.26:0/1656355662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063fab9520 0x56063fb23290 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 >> 172.21.15.26:0/1656355662 conn(0x56063fa2e200 msgr2=0x56063fabc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e477fe700 0 client.26639 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 shutdown_connections 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 1 -- 172.21.15.26:0/1656355662 wait complete. 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.280+0000 7f6e82035b80 20 client.26639 trim_cache size 0 max 16384 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:18:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:18:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 msgr2=0x7f6e540a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 0x7f6e540a3ab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e6819ea60 tx=0x7f6e6000b040 comp rx=0 tx=0).stop 2023-10-06T21:18:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 msgr2=0x7f6e6819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:18:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).stop 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 shutdown_connections 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e6819d5f0 0x7f6e6819daa0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e68068810 0x7f6e681a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e540a15f0 0x7f6e540a3ab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 --2- 172.21.15.26:0/1575819453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e681038b0 0x7f6e681a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 >> 172.21.15.26:0/1575819453 conn(0x7f6e68076b70 msgr2=0x7f6e680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 shutdown_connections 2023-10-06T21:18:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:18:50.281+0000 7f6e6e7ff700 1 -- 172.21.15.26:0/1575819453 wait complete. 2023-10-06T21:18:50.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:50 smithi167 ceph-mon[134494]: pgmap v263: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-10-06T21:18:50.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:50 smithi026 ceph-mon[98490]: pgmap v263: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-10-06T21:18:50.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:50 smithi078 ceph-mon[134896]: pgmap v263: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 672 op/s 2023-10-06T21:18:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 490.0 s, 1453.2 tps, lat 68.290 ms stddev 78.593 2023-10-06T21:18:52.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:52 smithi167 ceph-mon[134494]: pgmap v264: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 900 op/s 2023-10-06T21:18:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:52 smithi026 ceph-mon[98490]: pgmap v264: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 900 op/s 2023-10-06T21:18:52.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:52 smithi078 ceph-mon[134896]: pgmap v264: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 900 op/s 2023-10-06T21:18:54.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:54 smithi167 ceph-mon[134494]: pgmap v265: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 660 op/s 2023-10-06T21:18:54.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:54 smithi026 ceph-mon[98490]: pgmap v265: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 660 op/s 2023-10-06T21:18:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:54 smithi078 ceph-mon[134896]: pgmap v265: 97 pgs: 97 active+clean; 6.6 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 660 op/s 2023-10-06T21:18:56.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:56 smithi078 ceph-mon[134896]: pgmap v266: 97 pgs: 97 active+clean; 6.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 647 op/s 2023-10-06T21:18:56.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:56 smithi167 ceph-mon[134494]: pgmap v266: 97 pgs: 97 active+clean; 6.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 647 op/s 2023-10-06T21:18:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:56 smithi026 ceph-mon[98490]: pgmap v266: 97 pgs: 97 active+clean; 6.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 MiB/s wr, 647 op/s 2023-10-06T21:18:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 495.0 s, 1396.9 tps, lat 71.784 ms stddev 77.595 2023-10-06T21:18:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:18:58 smithi167 ceph-mon[134494]: pgmap v267: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 876 op/s 2023-10-06T21:18:58.833 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:18:58 smithi026 ceph-mon[98490]: pgmap v267: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 876 op/s 2023-10-06T21:18:58.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:18:58 smithi078 ceph-mon[134896]: pgmap v267: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 876 op/s 2023-10-06T21:19:00.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:00 smithi167 ceph-mon[134494]: pgmap v268: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-06T21:19:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:00 smithi026 ceph-mon[98490]: pgmap v268: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-06T21:19:00.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:00 smithi078 ceph-mon[134896]: pgmap v268: 97 pgs: 97 active+clean; 6.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 581 op/s 2023-10-06T21:19:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 500.0 s, 1129.7 tps, lat 88.366 ms stddev 144.750 2023-10-06T21:19:02.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:02 smithi167 ceph-mon[134494]: pgmap v269: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.64k op/s 2023-10-06T21:19:02.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:02 smithi026 ceph-mon[98490]: pgmap v269: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.64k op/s 2023-10-06T21:19:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:02 smithi078 ceph-mon[134896]: pgmap v269: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 28 MiB/s wr, 1.64k op/s 2023-10-06T21:19:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 -- 172.21.15.26:0/1280589706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0fef30 msgr2=0x7f675c0ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 --2- 172.21.15.26:0/1280589706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0fef30 0x7f675c0ff310 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6758009990 tx=0x7f6758009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 -- 172.21.15.26:0/1280589706 shutdown_connections 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 --2- 172.21.15.26:0/1280589706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 0x7f675c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 --2- 172.21.15.26:0/1280589706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0fef30 0x7f675c0ff310 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 --2- 172.21.15.26:0/1280589706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0ff850 0x7f675c111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.510+0000 7f67641ff700 1 -- 172.21.15.26:0/1280589706 >> 172.21.15.26:0/1280589706 conn(0x7f675c076e70 msgr2=0x7f675c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.511+0000 7f67641ff700 1 -- 172.21.15.26:0/1280589706 shutdown_connections 2023-10-06T21:19:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.511+0000 7f67641ff700 1 -- 172.21.15.26:0/1280589706 wait complete. 2023-10-06T21:19:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.512+0000 7f67641ff700 1 Processor -- start 2023-10-06T21:19:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.512+0000 7f67641ff700 1 -- start start 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.512+0000 7f67641ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0fef30 0x7f675c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f67641ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f67641ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 0x7f675c072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f67641ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c072910 con 0x7f675c0fef30 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f67641ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c072a80 con 0x7f675c111bc0 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f67641ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c072c40 con 0x7f675c0ff850 2023-10-06T21:19:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676279c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 0x7f675c072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42950/0 (socket says 172.21.15.26:42950) 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f6761f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0fef30 0x7f675c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 -- 172.21.15.26:0/145369703 learned_addr learned my addr 172.21.15.26:0/145369703 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 -- 172.21.15.26:0/145369703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 msgr2=0x7f675c072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 0x7f675c072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 -- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0fef30 msgr2=0x7f675c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0fef30 0x7f675c06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.513+0000 7f676179a700 1 -- 172.21.15.26:0/145369703 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580095f0 con 0x7f675c0ff850 2023-10-06T21:19:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f676179a700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9a0 tx=0x7f674c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f6752ffd700 1 -- 172.21.15.26:0/145369703 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c009790 con 0x7f675c0ff850 2023-10-06T21:19:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f6752ffd700 1 -- 172.21.15.26:0/145369703 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c010430 con 0x7f675c0ff850 2023-10-06T21:19:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f6752ffd700 1 -- 172.21.15.26:0/145369703 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c018680 con 0x7f675c0ff850 2023-10-06T21:19:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c072f20 con 0x7f675c0ff850 2023-10-06T21:19:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.514+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1b2c80 con 0x7f675c0ff850 2023-10-06T21:19:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.523+0000 7f6752ffd700 1 -- 172.21.15.26:0/145369703 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f674c010880 con 0x7f675c0ff850 2023-10-06T21:19:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.524+0000 7f6777be0b80 1 Processor -- start 2023-10-06T21:19:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.524+0000 7f6777be0b80 1 -- start start 2023-10-06T21:19:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cbf0400 con 0x557b5cc87520 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cbe1bc0 con 0x557b5cc86c00 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6777be0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cbdf4e0 con 0x557b5cc923d0 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51638/0 (socket says 172.21.15.26:51638) 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 -- 172.21.15.26:0/2810456690 learned_addr learned my addr 172.21.15.26:0/2810456690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f6761f9b700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 -- 172.21.15.26:0/2810456690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 msgr2=0x557b5cc947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676179a700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc947c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.525+0000 7f676279c700 1 -- 172.21.15.26:0/2810456690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 msgr2=0x557b5cc91e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f676279c700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc91e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f676279c700 1 -- 172.21.15.26:0/2810456690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b5ca05d80 con 0x557b5cc86c00 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6761f9b700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f676179a700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f676279c700 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f67540115f0 tx=0x7f6754009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b1287c0aeff27b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6747fff700 1 -- 172.21.15.26:0/2810456690 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400e040 con 0x557b5cc86c00 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6752ffd700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 0x7f67480a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6747fff700 1 -- 172.21.15.26:0/2810456690 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675400f040 con 0x557b5cc86c00 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6747fff700 1 -- 172.21.15.26:0/2810456690 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540048a0 con 0x557b5cc86c00 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.526+0000 7f6752ffd700 1 -- 172.21.15.26:0/145369703 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f674c014070 con 0x7f675c0ff850 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6761f9b700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 0x7f67480a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 -- 172.21.15.26:0/2810456690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 msgr2=0x557b5cc86fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f67540115f0 tx=0x7f6754009920 comp rx=0 tx=0).stop 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 -- 172.21.15.26:0/2810456690 shutdown_connections 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc86fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 --2- 172.21.15.26:0/2810456690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc91e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.527+0000 7f6777be0b80 1 -- 172.21.15.26:0/2810456690 >> 172.21.15.26:0/2810456690 conn(0x557b5cbfc200 msgr2=0x557b5cc83080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.528+0000 7f6761f9b700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 0x7f67480a3bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f675800b2e0 tx=0x7f6758005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.528+0000 7f6777be0b80 1 -- 172.21.15.26:0/2810456690 shutdown_connections 2023-10-06T21:19:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.532+0000 7f6777be0b80 1 -- 172.21.15.26:0/2810456690 wait complete. 2023-10-06T21:19:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.533+0000 7f6777be0b80 1 -- start start 2023-10-06T21:19:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.533+0000 7f6777be0b80 1 Processor -- start 2023-10-06T21:19:03.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:19:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc91d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cbdadd0 con 0x557b5cc87520 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cb776b0 con 0x557b5cc86c00 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cb77350 con 0x557b5cc923d0 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.534+0000 7f6777be0b80 10 client.0 _ll_register_callbacks cb 0x557b5c9d47a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.539+0000 7f676279c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.539+0000 7f676279c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42974/0 (socket says 172.21.15.26:42974) 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.540+0000 7f676279c700 1 -- 172.21.15.26:0/539933169 learned_addr learned my addr 172.21.15.26:0/539933169 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.543+0000 7f676279c700 1 -- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 msgr2=0x557b5cc91d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.543+0000 7f676279c700 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc91d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.543+0000 7f676279c700 1 -- 172.21.15.26:0/539933169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 msgr2=0x557b5cc8f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.543+0000 7f676279c700 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.543+0000 7f676279c700 1 -- 172.21.15.26:0/539933169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67540112d0 con 0x557b5cc923d0 2023-10-06T21:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.544+0000 7f676279c700 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6754004a00 tx=0x7f6754009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f214318ccf354435 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.544+0000 7f67457fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.550+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400e070 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.550+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6754003c90 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.550+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b5cc7ad50 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.551+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6754003e50 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.553+0000 7f6777be0b80 10 client.26560 fetch_fsmap learned FSMap version 47 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.553+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557b5c9eec00 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.553+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b5cc7ad50 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.553+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6754015610 con 0x557b5cc923d0 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 20 client.26560 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5cca1d80 0x557b5cca41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 4 client.26560 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.554+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b5cbd3db0 con 0x557b5cca1d80 2023-10-06T21:19:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.556+0000 7f6761f9b700 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5cca1d80 0x557b5cca41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.573+0000 7f6761f9b700 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5cca1d80 0x557b5cca41c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e475311c022a71a server_cookie=ec85319706e45f65 in_seq=0 out_seq=0 2023-10-06T21:19:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.573+0000 7f67457fa700 10 client.26560 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.575+0000 7f67457fa700 1 -- 172.21.15.26:0/539933169 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b5cbd3db0 con 0x557b5cca1d80 2023-10-06T21:19:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.575+0000 7f67457fa700 10 client.26560 handle_command_reply: tid=0 2023-10-06T21:19:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.576+0000 7f6777be0b80 1 client.26560 shutdown 2023-10-06T21:19:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.576+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 msgr2=0x557b5cc8f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.576+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6754004a00 tx=0x7f6754009d00 comp rx=0 tx=0).stop 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 shutdown_connections 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5cc8f660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5cca1d80 0x557b5cca41c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc91d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 --2- 172.21.15.26:0/539933169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc8f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 >> 172.21.15.26:0/539933169 conn(0x557b5cbfc200 msgr2=0x557b5cc82c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f67457fa700 0 client.26560 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 shutdown_connections 2023-10-06T21:19:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.577+0000 7f6777be0b80 1 -- 172.21.15.26:0/539933169 wait complete. 2023-10-06T21:19:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.578+0000 7f6777be0b80 20 client.26560 trim_cache size 0 max 16384 2023-10-06T21:19:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.603+0000 7f6777be0b80 1 Processor -- start 2023-10-06T21:19:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.603+0000 7f6777be0b80 1 -- start start 2023-10-06T21:19:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc89b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 0x557b5cc98f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc923d0 0x557b5cc8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ca5b8c0 con 0x557b5cc923d0 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cc83dc0 con 0x557b5cc86c00 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6777be0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cc7f150 con 0x557b5cc87520 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6761f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc89b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6761f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc89b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51664/0 (socket says 172.21.15.26:51664) 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f6761f9b700 1 -- 172.21.15.26:0/3467798863 learned_addr learned my addr 172.21.15.26:0/3467798863 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 0x557b5cc98f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676279c700 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc923d0 0x557b5cc8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 -- 172.21.15.26:0/3467798863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 msgr2=0x557b5cc89b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc89b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 -- 172.21.15.26:0/3467798863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc923d0 msgr2=0x557b5cc8e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc923d0 0x557b5cc8e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.604+0000 7f676179a700 1 -- 172.21.15.26:0/3467798863 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b5cc949d0 con 0x557b5cc87520 2023-10-06T21:19:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.605+0000 7f676179a700 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 0x557b5cc98f10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f674c0ca9e0 tx=0x7f674c099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ca045deca52b6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.605+0000 7f6750ff9700 1 -- 172.21.15.26:0/3467798863 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0cc050 con 0x557b5cc87520 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.605+0000 7f6750ff9700 1 -- 172.21.15.26:0/3467798863 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f674c09a270 con 0x557b5cc87520 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.605+0000 7f6750ff9700 1 -- 172.21.15.26:0/3467798863 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0cf370 con 0x557b5cc87520 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 -- 172.21.15.26:0/3467798863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 msgr2=0x557b5cc98f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 0x557b5cc98f10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f674c0ca9e0 tx=0x7f674c099e30 comp rx=0 tx=0).stop 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 -- 172.21.15.26:0/3467798863 shutdown_connections 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc923d0 0x557b5cc8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc89b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 --2- 172.21.15.26:0/3467798863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc87520 0x557b5cc98f10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 -- 172.21.15.26:0/3467798863 >> 172.21.15.26:0/3467798863 conn(0x557b5cbfc200 msgr2=0x557b5cca3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 -- 172.21.15.26:0/3467798863 shutdown_connections 2023-10-06T21:19:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.606+0000 7f6777be0b80 1 -- 172.21.15.26:0/3467798863 wait complete. 2023-10-06T21:19:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.607+0000 7f6777be0b80 1 -- start start 2023-10-06T21:19:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.607+0000 7f6777be0b80 1 Processor -- start 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5ccf4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ccf5140 con 0x557b5cc87520 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ccf5280 con 0x557b5cc86c00 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ccf53c0 con 0x557b5cc923d0 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.608+0000 7f6777be0b80 10 client.0 _ll_register_callbacks cb 0x7f67540114f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51668/0 (socket says 172.21.15.26:51668) 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 -- 172.21.15.26:0/1166662969 learned_addr learned my addr 172.21.15.26:0/1166662969 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f676279c700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5ccf4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 -- 172.21.15.26:0/1166662969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 msgr2=0x557b5ccf4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5ccf4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 -- 172.21.15.26:0/1166662969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 msgr2=0x557b5cc95620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.609+0000 7f6761f9b700 1 -- 172.21.15.26:0/1166662969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1b2c80 con 0x557b5cc86c00 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.610+0000 7f6761f9b700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f67580319e0 tx=0x7f6758031da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a24bd6b9ba4d89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.610+0000 7f67457fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.610+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675801d050 con 0x557b5cc86c00 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.610+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6758036030 con 0x557b5cc86c00 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.611+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675400f040 con 0x557b5cc86c00 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.611+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67580054d0 con 0x557b5cc86c00 2023-10-06T21:19:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.611+0000 7f6777be0b80 10 client.26654 fetch_fsmap learned FSMap version 47 2023-10-06T21:19:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.611+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557b5ca2c1f0 con 0x557b5cc86c00 2023-10-06T21:19:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675803f030 con 0x557b5cc86c00 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f675800f460 con 0x557b5cc86c00 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 10 client.26654 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 20 client.26654 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5ccf6b50 0x557b5ccf8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 4 client.26654 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b5cbd3ad0 con 0x557b5ccf6b50 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.612+0000 7f676179a700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5ccf6b50 0x557b5ccf8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.613+0000 7f676179a700 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5ccf6b50 0x557b5ccf8f90 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0094857c2b70543 server_cookie=21113167db40e3a2 in_seq=0 out_seq=0 2023-10-06T21:19:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.613+0000 7f67457fa700 10 client.26654 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.636+0000 7f67457fa700 1 -- 172.21.15.26:0/1166662969 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f674c0b1040 con 0x557b5ccf6b50 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.636+0000 7f67457fa700 10 client.26654 handle_command_reply: tid=0 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.636+0000 7f6777be0b80 1 client.26654 shutdown 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.636+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 msgr2=0x557b5cc950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.636+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f67580319e0 tx=0x7f6758031da0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 shutdown_connections 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b5cc923d0 0x557b5ccf4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557b5ccf6b50 0x557b5ccf8f90 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557b5cc86c00 0x557b5cc950e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f67457fa700 0 client.26654 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 --2- 172.21.15.26:0/1166662969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557b5cc87520 0x557b5cc95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 >> 172.21.15.26:0/1166662969 conn(0x557b5cbfc200 msgr2=0x557b5cc8a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 shutdown_connections 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 1 -- 172.21.15.26:0/1166662969 wait complete. 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.637+0000 7f6777be0b80 20 client.26654 trim_cache size 0 max 16384 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:19:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:19:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 msgr2=0x7f67480a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 0x7f67480a3bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f675800b2e0 tx=0x7f6758005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 msgr2=0x7f675c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.641 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9a0 tx=0x7f674c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 shutdown_connections 2023-10-06T21:19:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f675c111bc0 0x7f675c072140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67480a1730 0x7f67480a3bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675c0fef30 0x7f675c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 --2- 172.21.15.26:0/145369703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f675c0ff850 0x7f675c06cb30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 >> 172.21.15.26:0/145369703 conn(0x7f675c076e70 msgr2=0x7f675c10fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 shutdown_connections 2023-10-06T21:19:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:03.638+0000 7f67641ff700 1 -- 172.21.15.26:0/145369703 wait complete. 2023-10-06T21:19:03.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 -- 172.21.15.26:0/3450285893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 msgr2=0x7f4ffc10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 --2- 172.21.15.26:0/3450285893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc10adf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098f0 tx=0x7f4ff0009c00 comp rx=0 tx=0).stop 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 -- 172.21.15.26:0/3450285893 shutdown_connections 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 --2- 172.21.15.26:0/3450285893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 --2- 172.21.15.26:0/3450285893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 0x7f4ffc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 --2- 172.21.15.26:0/3450285893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc10adf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 -- 172.21.15.26:0/3450285893 >> 172.21.15.26:0/3450285893 conn(0x7f4ffc0fb830 msgr2=0x7f4ffc0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 -- 172.21.15.26:0/3450285893 shutdown_connections 2023-10-06T21:19:03.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.879+0000 7f50011ff700 1 -- 172.21.15.26:0/3450285893 wait complete. 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.880+0000 7f50011ff700 1 Processor -- start 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.880+0000 7f50011ff700 1 -- start start 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 0x7f4ffc0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1009b0 con 0x7f4ffc101ff0 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1a56e0 con 0x7f4ffc10b410 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f50011ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1a5820 con 0x7f4ffc102910 2023-10-06T21:19:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51684/0 (socket says 172.21.15.26:51684) 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f4ffb59e700 1 -- 172.21.15.26:0/1670697350 learned_addr learned my addr 172.21.15.26:0/1670697350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.881+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 0x7f4ffc0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 -- 172.21.15.26:0/1670697350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 msgr2=0x7f4ffc100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 -- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 msgr2=0x7f4ffc0ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 0x7f4ffc0ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 -- 172.21.15.26:0/1670697350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009520 con 0x7f4ffc10b410 2023-10-06T21:19:03.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c110 tx=0x7f4ff400c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4fe3fff700 1 -- 172.21.15.26:0/1670697350 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4017070 con 0x7f4ffc10b410 2023-10-06T21:19:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc1a5a20 con 0x7f4ffc10b410 2023-10-06T21:19:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f4fe3fff700 1 -- 172.21.15.26:0/1670697350 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff400e050 con 0x7f4ffc10b410 2023-10-06T21:19:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.882+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc1a5eb0 con 0x7f4ffc10b410 2023-10-06T21:19:03.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.883+0000 7f4fe3fff700 1 -- 172.21.15.26:0/1670697350 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4007980 con 0x7f4ffc10b410 2023-10-06T21:19:03.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.888+0000 7f4fe3fff700 1 -- 172.21.15.26:0/1670697350 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4ff4004ad0 con 0x7f4ffc10b410 2023-10-06T21:19:03.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.891+0000 7f5014b06b80 1 Processor -- start 2023-10-06T21:19:03.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.891+0000 7f4fe3fff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 0x7f4fe40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.891+0000 7f5014b06b80 1 -- start start 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.891+0000 7f4fe3fff700 1 -- 172.21.15.26:0/1670697350 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4ff40ccc80 con 0x7f4ffc10b410 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.891+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 0x7f4fe40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c770c0 0x559560c774a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c8ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560be0460 con 0x559560c770c0 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560bd1c20 con 0x559560c779e0 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f5014b06b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560bcf540 con 0x559560c8b0a0 2023-10-06T21:19:03.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43030/0 (socket says 172.21.15.26:43030) 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 -- 172.21.15.26:0/2032964737 learned_addr learned my addr 172.21.15.26:0/2032964737 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c8ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c770c0 0x559560c774a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 -- 172.21.15.26:0/2032964737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 msgr2=0x559560c8ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c8ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 -- 172.21.15.26:0/2032964737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c770c0 msgr2=0x559560c774a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c770c0 0x559560c774a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.892+0000 7f4ffb59e700 1 -- 172.21.15.26:0/2032964737 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595609f5de0 con 0x559560c8b0a0 2023-10-06T21:19:03.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.893+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 0x7f4fe40a3b30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005e90 tx=0x7f4ff0019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.893+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c820 tx=0x7f4ff4099af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d96d11fbac8907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.894+0000 7f4fe17fa700 1 -- 172.21.15.26:0/2032964737 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff409a2f0 con 0x559560c8b0a0 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.894+0000 7f4fe17fa700 1 -- 172.21.15.26:0/2032964737 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff400f040 con 0x559560c8b0a0 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.894+0000 7f4fe17fa700 1 -- 172.21.15.26:0/2032964737 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff409a0b0 con 0x559560c8b0a0 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 -- 172.21.15.26:0/2032964737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 msgr2=0x559560c8d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c820 tx=0x7f4ff4099af0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 -- 172.21.15.26:0/2032964737 shutdown_connections 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c8b0a0 0x559560c8d490 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c770c0 0x559560c774a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 --2- 172.21.15.26:0/2032964737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c8ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 -- 172.21.15.26:0/2032964737 >> 172.21.15.26:0/2032964737 conn(0x559560bec260 msgr2=0x559560c73450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 -- 172.21.15.26:0/2032964737 shutdown_connections 2023-10-06T21:19:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.895+0000 7f5014b06b80 1 -- 172.21.15.26:0/2032964737 wait complete. 2023-10-06T21:19:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.896+0000 7f5014b06b80 1 -- start start 2023-10-06T21:19:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.896+0000 7f5014b06b80 1 Processor -- start 2023-10-06T21:19:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560bcae30 con 0x559560c8b0a0 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560b67710 con 0x559560c779e0 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560b673b0 con 0x559560c770c0 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f5014b06b80 10 client.0 _ll_register_callbacks cb 0x5595609c4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f4ffad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f4ffa59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f4ffad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43036/0 (socket says 172.21.15.26:43036) 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f4ffad9d700 1 -- 172.21.15.26:0/620959511 learned_addr learned my addr 172.21.15.26:0/620959511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.897+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 -- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 msgr2=0x559560c88400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c88400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 -- 172.21.15.26:0/620959511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 msgr2=0x559560c88940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c88940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 -- 172.21.15.26:0/620959511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc1a5eb0 con 0x559560c770c0 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4fec013710 tx=0x7f4fec013a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b687a9d02f89fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4fdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec009870 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559560c6af10 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4fec011970 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.898+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fec011b30 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 10 client.26575 fetch_fsmap learned FSMap version 47 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5595609dec60 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559560c6af10 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fec01d3f0 con 0x559560c770c0 2023-10-06T21:19:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 10 client.26575 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 20 client.26575 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560c9ad60 0x559560c9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 4 client.26575 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559560bc3e10 con 0x559560c9ad60 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.899+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560c9ad60 0x559560c9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.900+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560c9ad60 0x559560c9d180 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2665bfb92a85c129 server_cookie=af74efd98e8c6289 in_seq=0 out_seq=0 2023-10-06T21:19:03.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.900+0000 7f4fdeff5700 10 client.26575 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.900+0000 7f4fdeff5700 1 -- 172.21.15.26:0/620959511 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559560bc3e10 con 0x559560c9ad60 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.900+0000 7f4fdeff5700 10 client.26575 handle_command_reply: tid=0 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 client.26575 shutdown 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 msgr2=0x559560c87ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4fec013710 tx=0x7f4fec013a20 comp rx=0 tx=0).stop 2023-10-06T21:19:03.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 shutdown_connections 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c88940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c770c0 0x559560c87ec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560c9ad60 0x559560c9d180 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 --2- 172.21.15.26:0/620959511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c779e0 0x559560c88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 >> 172.21.15.26:0/620959511 conn(0x559560bec260 msgr2=0x559560c72fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f4fdeff5700 0 client.26575 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.901+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 shutdown_connections 2023-10-06T21:19:03.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.902+0000 7f5014b06b80 1 -- 172.21.15.26:0/620959511 wait complete. 2023-10-06T21:19:03.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.903+0000 7f5014b06b80 20 client.26575 trim_cache size 0 max 16384 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 Processor -- start 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 -- start start 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560c85130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c85670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560a4b920 con 0x559560c8b0a0 2023-10-06T21:19:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560c6ade0 con 0x559560c770c0 2023-10-06T21:19:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.929+0000 7f5014b06b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560c74dc0 con 0x559560c779e0 2023-10-06T21:19:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51708/0 (socket says 172.21.15.26:51708) 2023-10-06T21:19:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 -- 172.21.15.26:0/3867082840 learned_addr learned my addr 172.21.15.26:0/3867082840 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffa59c700 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560c85130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c85670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 -- 172.21.15.26:0/3867082840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 msgr2=0x559560c85130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560c85130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 -- 172.21.15.26:0/3867082840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 msgr2=0x559560c85670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c85670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 -- 172.21.15.26:0/3867082840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559560c6f140 con 0x559560c770c0 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.930+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0136e0 tx=0x7f4fec005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26391c6ac60d60ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f4fe1ffb700 1 -- 172.21.15.26:0/3867082840 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec01dbc0 con 0x559560c770c0 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f4fe1ffb700 1 -- 172.21.15.26:0/3867082840 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fec0055d0 con 0x559560c770c0 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f4fe1ffb700 1 -- 172.21.15.26:0/3867082840 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec011dd0 con 0x559560c770c0 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 -- 172.21.15.26:0/3867082840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 msgr2=0x559560c91bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0136e0 tx=0x7f4fec005990 comp rx=0 tx=0).stop 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 -- 172.21.15.26:0/3867082840 shutdown_connections 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560c85670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560c91bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.931+0000 7f5014b06b80 1 --2- 172.21.15.26:0/3867082840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560c85130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.932+0000 7f5014b06b80 1 -- 172.21.15.26:0/3867082840 >> 172.21.15.26:0/3867082840 conn(0x559560bec260 msgr2=0x559560c831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.932+0000 7f5014b06b80 1 -- 172.21.15.26:0/3867082840 shutdown_connections 2023-10-06T21:19:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.932+0000 7f5014b06b80 1 -- 172.21.15.26:0/3867082840 wait complete. 2023-10-06T21:19:03.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.933+0000 7f5014b06b80 1 -- start start 2023-10-06T21:19:03.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.933+0000 7f5014b06b80 1 Processor -- start 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.934+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560ce8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.934+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560ce87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.934+0000 7f5014b06b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.934+0000 7f5014b06b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560ce3150 con 0x559560c8b0a0 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.934+0000 7f5014b06b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560ce32c0 con 0x559560c770c0 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.935+0000 7f5014b06b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559560ce3480 con 0x559560c779e0 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.935+0000 7f5014b06b80 10 client.0 _ll_register_callbacks cb 0x5595609c4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.936+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.936+0000 7f4ffb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37186/0 (socket says 172.21.15.26:37186) 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.936+0000 7f4ffb59e700 1 -- 172.21.15.26:0/441995652 learned_addr learned my addr 172.21.15.26:0/441995652 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffb59e700 1 -- 172.21.15.26:0/441995652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 msgr2=0x559560ce87b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560ce87b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffb59e700 1 -- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 msgr2=0x559560ce8270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560ce8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560ce8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.937+0000 7f4ffb59e700 1 -- 172.21.15.26:0/441995652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595609dec60 con 0x559560c8b0a0 2023-10-06T21:19:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.955+0000 7f4ffb59e700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40a1a50 tx=0x7f4ff4019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f778df50671f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.955+0000 7f4fdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:19:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.965+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff409a050 con 0x559560c8b0a0 2023-10-06T21:19:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.965+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4ff4099bf0 con 0x559560c8b0a0 2023-10-06T21:19:03.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.967+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff400f040 con 0x559560c8b0a0 2023-10-06T21:19:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.968+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff40a1f80 con 0x559560c8b0a0 2023-10-06T21:19:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.968+0000 7f5014b06b80 10 client.16389 fetch_fsmap learned FSMap version 47 2023-10-06T21:19:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.968+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559560a1c250 con 0x559560c8b0a0 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff400f040 con 0x559560c8b0a0 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 47) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4ff4007ba0 con 0x559560c8b0a0 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 20 client.16389 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560ce9220 0x559560ceb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 4 client.16389 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:19:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.969+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559560bc3b30 con 0x559560ce9220 2023-10-06T21:19:03.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.973+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560ce9220 0x559560ceb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.974+0000 7f4ffad9d700 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560ce9220 0x559560ceb640 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33b7fae618c4b0b server_cookie=97321ec15a3c4f11 in_seq=0 out_seq=0 2023-10-06T21:19:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.974+0000 7f4fdeff5700 10 client.16389 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f4fdeff5700 1 -- 172.21.15.26:0/441995652 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559560bc3b30 con 0x559560ce9220 2023-10-06T21:19:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f4fdeff5700 10 client.16389 handle_command_reply: tid=0 2023-10-06T21:19:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 client.16389 shutdown 2023-10-06T21:19:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 msgr2=0x559560ce2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40a1a50 tx=0x7f4ff4019430 comp rx=0 tx=0).stop 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 shutdown_connections 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559560c8b0a0 0x559560ce2b00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x559560ce9220 0x559560ceb640 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559560c770c0 0x559560ce8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f4fdeff5700 0 client.16389 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 --2- 172.21.15.26:0/441995652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559560c779e0 0x559560ce87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.976+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 >> 172.21.15.26:0/441995652 conn(0x559560bec260 msgr2=0x559560c78fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.977+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 shutdown_connections 2023-10-06T21:19:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.977+0000 7f5014b06b80 1 -- 172.21.15.26:0/441995652 wait complete. 2023-10-06T21:19:03.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.977+0000 7f5014b06b80 20 client.16389 trim_cache size 0 max 16384 2023-10-06T21:19:03.980 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:19:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "3b50318a-9d97-4aef-9984-7b1b05be7adf": { 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "3b50318a-9d97-4aef-9984-7b1b05be7adf", 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:19:03.981 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:19:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 msgr2=0x7f4fe40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 0x7f4fe40a3b30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005e90 tx=0x7f4ff0019040 comp rx=0 tx=0).stop 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 msgr2=0x7f4ffc1a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c110 tx=0x7f4ff400c420 comp rx=0 tx=0).stop 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 shutdown_connections 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ffc10b410 0x7f4ffc1a51a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4fe40a1670 0x7f4fe40a3b30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ffc101ff0 0x7f4ffc0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 --2- 172.21.15.26:0/1670697350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ffc102910 0x7f4ffc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:03.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 >> 172.21.15.26:0/1670697350 conn(0x7f4ffc0fb830 msgr2=0x7f4ffc0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:03.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 shutdown_connections 2023-10-06T21:19:03.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:03.978+0000 7f50011ff700 1 -- 172.21.15.26:0/1670697350 wait complete. 2023-10-06T21:19:04.005 INFO:tasks.cephfs.filesystem:scrub status for tag:3b50318a-9d97-4aef-9984-7b1b05be7adf - {'path': '/', 'tag': '3b50318a-9d97-4aef-9984-7b1b05be7adf', 'options': 'recursive,force'} 2023-10-06T21:19:04.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:04 smithi026 ceph-mon[98490]: pgmap v270: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-10-06T21:19:04.374 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:04 smithi078 ceph-mon[134896]: pgmap v270: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-10-06T21:19:04.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:04 smithi167 ceph-mon[134494]: pgmap v270: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2023-10-06T21:19:06.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:06 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:06.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:06 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:06.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:06 smithi167 ceph-mon[134494]: pgmap v271: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-06T21:19:06.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:06 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:06.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:06 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:06.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:06 smithi026 ceph-mon[98490]: pgmap v271: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-06T21:19:06.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:06 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:06.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:06 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:06.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:06 smithi078 ceph-mon[134896]: pgmap v271: 97 pgs: 97 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-06T21:19:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 505.0 s, 1266.6 tps, lat 78.552 ms stddev 97.807 2023-10-06T21:19:07.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:07 smithi167 ceph-mon[134494]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:19:07.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:07 smithi026 ceph-mon[98490]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:19:07.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:07 smithi078 ceph-mon[134896]: osdmap e94: 12 total, 12 up, 12 in 2023-10-06T21:19:08.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:08 smithi167 ceph-mon[134494]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:08.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:08 smithi026 ceph-mon[98490]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:08.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:08 smithi078 ceph-mon[134896]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:10.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:10 smithi167 ceph-mon[134494]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:10.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:10 smithi026 ceph-mon[98490]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:10.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:10 smithi078 ceph-mon[134896]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 6.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 29 MiB/s wr, 1.95k op/s 2023-10-06T21:19:11.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:11.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:11.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:11 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:11.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:11 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:11 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:11.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:11 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:11.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:11.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:11.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:11 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:11.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:11 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:19:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 510.0 s, 1592.4 tps, lat 63.491 ms stddev 74.952 2023-10-06T21:19:12.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:12 smithi167 ceph-mon[134494]: pgmap v275: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 914 op/s 2023-10-06T21:19:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:12 smithi026 ceph-mon[98490]: pgmap v275: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 914 op/s 2023-10-06T21:19:12.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:12 smithi078 ceph-mon[134896]: pgmap v275: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 5.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 914 op/s 2023-10-06T21:19:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:14 smithi167 ceph-mon[134494]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-06T21:19:14.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:14 smithi026 ceph-mon[98490]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-06T21:19:14.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:14 smithi078 ceph-mon[134896]: pgmap v276: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 12 MiB/s wr, 699 op/s 2023-10-06T21:19:16.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:16 smithi167 ceph-mon[134494]: pgmap v277: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 680 op/s 2023-10-06T21:19:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:16 smithi026 ceph-mon[98490]: pgmap v277: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 680 op/s 2023-10-06T21:19:16.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:16 smithi078 ceph-mon[134896]: pgmap v277: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 680 op/s 2023-10-06T21:19:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.903+0000 7fa96d5ff700 1 -- 172.21.15.26:0/3858177216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 msgr2=0x7fa9680682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.903+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/3858177216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9680682d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0098f0 tx=0x7fa95c009c00 comp rx=0 tx=0).stop 2023-10-06T21:19:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.904+0000 7fa96d5ff700 1 -- 172.21.15.26:0/3858177216 shutdown_connections 2023-10-06T21:19:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.904+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/3858177216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9681038b0 0x7fa968103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.904+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/3858177216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9680682d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.904+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/3858177216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa968068810 0x7fa968068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.904+0000 7fa96d5ff700 1 -- 172.21.15.26:0/3858177216 >> 172.21.15.26:0/3858177216 conn(0x7fa968076b70 msgr2=0x7fa968076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.905+0000 7fa96d5ff700 1 -- 172.21.15.26:0/3858177216 shutdown_connections 2023-10-06T21:19:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.905+0000 7fa96d5ff700 1 -- 172.21.15.26:0/3858177216 wait complete. 2023-10-06T21:19:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.906+0000 7fa96d5ff700 1 Processor -- start 2023-10-06T21:19:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.906+0000 7fa96d5ff700 1 -- start start 2023-10-06T21:19:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9681038b0 0x7fa9681103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9681a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a11f0 con 0x7fa968068810 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a1330 con 0x7fa9681038b0 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa96d5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a14f0 con 0x7fa968105e20 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52528/0 (socket says 172.21.15.26:52528) 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 -- 172.21.15.26:0/1868322941 learned_addr learned my addr 172.21.15.26:0/1868322941 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa9677fe700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9681a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa9667fc700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9681038b0 0x7fa9681103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 -- 172.21.15.26:0/1868322941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 msgr2=0x7fa9681a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9681a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 -- 172.21.15.26:0/1868322941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9681038b0 msgr2=0x7fa9681103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.907+0000 7fa966ffd700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9681038b0 0x7fa9681103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.908+0000 7fa966ffd700 1 -- 172.21.15.26:0/1868322941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c009520 con 0x7fa968068810 2023-10-06T21:19:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.908+0000 7fa966ffd700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009810 tx=0x7fa95c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.910+0000 7fa94ffff700 1 -- 172.21.15.26:0/1868322941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c01c070 con 0x7fa968068810 2023-10-06T21:19:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.911+0000 7fa94ffff700 1 -- 172.21.15.26:0/1868322941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95c005500 con 0x7fa968068810 2023-10-06T21:19:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.911+0000 7fa94ffff700 1 -- 172.21.15.26:0/1868322941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c004a40 con 0x7fa968068810 2023-10-06T21:19:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.911+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681a17d0 con 0x7fa968068810 2023-10-06T21:19:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.912+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681a1d10 con 0x7fa968068810 2023-10-06T21:19:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.921+0000 7fa980f01b80 1 Processor -- start 2023-10-06T21:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.921+0000 7fa94ffff700 1 -- 172.21.15.26:0/1868322941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa95c020070 con 0x7fa968068810 2023-10-06T21:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 -- start start 2023-10-06T21:19:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d057e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d05a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cfb6400 con 0x561e0d0583d0 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.922+0000 7fa980f01b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cfa7bc0 con 0x561e0d04cc00 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.923+0000 7fa980f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cfa54e0 con 0x561e0d04d520 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.923+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.923+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33970/0 (socket says 172.21.15.26:33970) 2023-10-06T21:19:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.923+0000 7fa9677fe700 1 -- 172.21.15.26:0/3150699039 learned_addr learned my addr 172.21.15.26:0/3150699039 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.923+0000 7fa966ffd700 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d057e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 -- 172.21.15.26:0/3150699039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 msgr2=0x561e0d057e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d057e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 -- 172.21.15.26:0/3150699039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 msgr2=0x561e0d05a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d05a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 -- 172.21.15.26:0/3150699039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0cdcbd80 con 0x561e0d04cc00 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa958007790 tx=0x7fa958007aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d67cd21bdfb28a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.924+0000 7fa94ffff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 0x7fa9500a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.925+0000 7fa94d7fa700 1 -- 172.21.15.26:0/3150699039 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580125f0 con 0x561e0d04cc00 2023-10-06T21:19:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.925+0000 7fa94ffff700 1 -- 172.21.15.26:0/1868322941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa95c0cddd0 con 0x7fa968068810 2023-10-06T21:19:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.925+0000 7fa94d7fa700 1 -- 172.21.15.26:0/3150699039 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958012c10 con 0x561e0d04cc00 2023-10-06T21:19:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.925+0000 7fa94d7fa700 1 -- 172.21.15.26:0/3150699039 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958013d40 con 0x561e0d04cc00 2023-10-06T21:19:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.925+0000 7fa966ffd700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 0x7fa9500a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.927+0000 7fa980f01b80 1 -- 172.21.15.26:0/3150699039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 msgr2=0x561e0d04cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.927+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa958007790 tx=0x7fa958007aa0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.928+0000 7fa966ffd700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 0x7fa9500a3bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa95c097810 tx=0x7fa95c0a1040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 -- 172.21.15.26:0/3150699039 shutdown_connections 2023-10-06T21:19:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d05a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d04cfe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3150699039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d057e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 -- 172.21.15.26:0/3150699039 >> 172.21.15.26:0/3150699039 conn(0x561e0cfc2200 msgr2=0x561e0d049080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 -- 172.21.15.26:0/3150699039 shutdown_connections 2023-10-06T21:19:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.929+0000 7fa980f01b80 1 -- 172.21.15.26:0/3150699039 wait complete. 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.930+0000 7fa980f01b80 1 -- start start 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.930+0000 7fa980f01b80 1 Processor -- start 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d055070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d055710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cfa0dd0 con 0x561e0d0583d0 2023-10-06T21:19:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cf3d6b0 con 0x561e0d04cc00 2023-10-06T21:19:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0cf3d350 con 0x561e0d04d520 2023-10-06T21:19:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.931+0000 7fa980f01b80 10 client.0 _ll_register_callbacks cb 0x561e0cd9a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.932+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.932+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33982/0 (socket says 172.21.15.26:33982) 2023-10-06T21:19:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.932+0000 7fa9677fe700 1 -- 172.21.15.26:0/2510616116 learned_addr learned my addr 172.21.15.26:0/2510616116 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.932+0000 7fa9667fc700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d055070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d055710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 -- 172.21.15.26:0/2510616116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 msgr2=0x561e0d055070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d055070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 -- 172.21.15.26:0/2510616116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 msgr2=0x561e0d055710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d055710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.934+0000 7fa9677fe700 1 -- 172.21.15.26:0/2510616116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95800e040 con 0x561e0d04cc00 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.935+0000 7fa9677fe700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa958006b10 tx=0x7fa958013880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74957cd9b21b64d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.935+0000 7fa94aff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.935+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958025070 con 0x561e0d04cc00 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.935+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e0d040d50 con 0x561e0d04cc00 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.935+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa9580129e0 con 0x561e0d04cc00 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa958013920 con 0x561e0d04cc00 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 10 client.26681 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561e0cdb4c00 con 0x561e0d04cc00 2023-10-06T21:19:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e0d040d50 con 0x561e0d04cc00 2023-10-06T21:19:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa95801e9e0 con 0x561e0d04cc00 2023-10-06T21:19:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 20 client.26681 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d067ee0 0x561e0d06a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 4 client.26681 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.936+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e0cf99db0 con 0x561e0d067ee0 2023-10-06T21:19:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.937+0000 7fa9667fc700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d067ee0 0x561e0d06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.939+0000 7fa9667fc700 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d067ee0 0x561e0d06a300 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16a5e3b16fa20d1 server_cookie=6739fde203afbb75 in_seq=0 out_seq=0 2023-10-06T21:19:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.940+0000 7fa94aff5700 10 client.26681 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.940+0000 7fa94aff5700 1 -- 172.21.15.26:0/2510616116 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561e0cf99db0 con 0x561e0d067ee0 2023-10-06T21:19:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.940+0000 7fa94aff5700 10 client.26681 handle_command_reply: tid=0 2023-10-06T21:19:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.940+0000 7fa980f01b80 1 client.26681 shutdown 2023-10-06T21:19:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 msgr2=0x561e0d057df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa958006b10 tx=0x7fa958013880 comp rx=0 tx=0).stop 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 shutdown_connections 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d055710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d067ee0 0x561e0d06a300 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04cc00 0x561e0d057df0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 --2- 172.21.15.26:0/2510616116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04d520 0x561e0d055070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 >> 172.21.15.26:0/2510616116 conn(0x561e0cfc2200 msgr2=0x561e0d048c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.941+0000 7fa94aff5700 0 client.26681 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.942+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 shutdown_connections 2023-10-06T21:19:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.942+0000 7fa980f01b80 1 -- 172.21.15.26:0/2510616116 wait complete. 2023-10-06T21:19:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.942+0000 7fa980f01b80 20 client.26681 trim_cache size 0 max 16384 2023-10-06T21:19:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 Processor -- start 2023-10-06T21:19:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 -- start start 2023-10-06T21:19:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d057990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d050330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0ce218c0 con 0x561e0d0583d0 2023-10-06T21:19:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0d0554d0 con 0x561e0d04d520 2023-10-06T21:19:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.970+0000 7fa980f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0d049dc0 con 0x561e0d04cc00 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.971+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.971+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59052/0 (socket says 172.21.15.26:59052) 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.971+0000 7fa9677fe700 1 -- 172.21.15.26:0/3564282567 learned_addr learned my addr 172.21.15.26:0/3564282567 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.971+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d050330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.971+0000 7fa9667fc700 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d057990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 -- 172.21.15.26:0/3564282567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 msgr2=0x561e0d057990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d057990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 -- 172.21.15.26:0/3564282567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 msgr2=0x561e0d050330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d050330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 -- 172.21.15.26:0/3564282567 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0cdf21f0 con 0x561e0d04cc00 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.972+0000 7fa9677fe700 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9580127a0 tx=0x7fa958021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8b97f02f1a5bf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.973+0000 7fa94dffb700 1 -- 172.21.15.26:0/3564282567 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958025040 con 0x561e0d04cc00 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.973+0000 7fa94dffb700 1 -- 172.21.15.26:0/3564282567 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958026040 con 0x561e0d04cc00 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.973+0000 7fa94dffb700 1 -- 172.21.15.26:0/3564282567 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958013e90 con 0x561e0d04cc00 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 -- 172.21.15.26:0/3564282567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 msgr2=0x561e0d05ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9580127a0 tx=0x7fa958021040 comp rx=0 tx=0).stop 2023-10-06T21:19:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 -- 172.21.15.26:0/3564282567 shutdown_connections 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d050330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d05ef10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 --2- 172.21.15.26:0/3564282567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d057990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.974+0000 7fa980f01b80 1 -- 172.21.15.26:0/3564282567 >> 172.21.15.26:0/3564282567 conn(0x561e0cfc2200 msgr2=0x561e0d063670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.976+0000 7fa980f01b80 1 -- 172.21.15.26:0/3564282567 shutdown_connections 2023-10-06T21:19:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.976+0000 7fa980f01b80 1 -- 172.21.15.26:0/3564282567 wait complete. 2023-10-06T21:19:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.976+0000 7fa980f01b80 1 -- start start 2023-10-06T21:19:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.977+0000 7fa980f01b80 1 Processor -- start 2023-10-06T21:19:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.978+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.978+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.978+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59054/0 (socket says 172.21.15.26:59054) 2023-10-06T21:19:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.978+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d051810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa980f01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d051d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa980f01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0d052430 con 0x561e0d0583d0 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa980f01b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0d0525a0 con 0x561e0d04d520 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa980f01b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0d052760 con 0x561e0d04cc00 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa980f01b80 10 client.0 _ll_register_callbacks cb 0x7fa9580075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa9677fe700 1 -- 172.21.15.26:0/535308784 learned_addr learned my addr 172.21.15.26:0/535308784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa9667fc700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d051810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.979+0000 7fa9677fe700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d051d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 -- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 msgr2=0x561e0d051810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d051810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 -- 172.21.15.26:0/535308784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 msgr2=0x561e0d051d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d051d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 -- 172.21.15.26:0/535308784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e0cdb4c00 con 0x561e0d04cc00 2023-10-06T21:19:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa9677fe700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa95801eda0 tx=0x7fa958013990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de57fed34cd95832 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.980+0000 7fa94aff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958025050 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa958000e40 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa958012c10 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa958005d20 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa980f01b80 10 client.26596 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.981+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561e0cdca250 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa958012c10 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa958046070 con 0x561e0d04cc00 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 10 client.26596 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 10 client.26596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 20 client.26596 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d0bc4b0 0x561e0d0be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 4 client.26596 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:19:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e0cf99ad0 con 0x561e0d0bc4b0 2023-10-06T21:19:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.982+0000 7fa9667fc700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d0bc4b0 0x561e0d0be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.983+0000 7fa9667fc700 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d0bc4b0 0x561e0d0be8d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837c9e1e46f16116 server_cookie=b314db64f04edf14 in_seq=0 out_seq=0 2023-10-06T21:19:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:16.984+0000 7fa94aff5700 10 client.26596 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.006+0000 7fa94aff5700 1 -- 172.21.15.26:0/535308784 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e0cf99ad0 con 0x561e0d0bc4b0 2023-10-06T21:19:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.006+0000 7fa94aff5700 10 client.26596 handle_command_reply: tid=0 2023-10-06T21:19:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.006+0000 7fa980f01b80 1 client.26596 shutdown 2023-10-06T21:19:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 msgr2=0x561e0d0512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa95801eda0 tx=0x7fa958013990 comp rx=0 tx=0).stop 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 shutdown_connections 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e0d0583d0 0x561e0d051d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e0d04cc00 0x561e0d0512d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x561e0d0bc4b0 0x561e0d0be8d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 --2- 172.21.15.26:0/535308784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e0d04d520 0x561e0d051810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 >> 172.21.15.26:0/535308784 conn(0x561e0cfc2200 msgr2=0x561e0d050330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.008+0000 7fa94aff5700 0 client.26596 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.009+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 shutdown_connections 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.009+0000 7fa980f01b80 1 -- 172.21.15.26:0/535308784 wait complete. 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.009+0000 7fa980f01b80 20 client.26596 trim_cache size 0 max 16384 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:19:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:19:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:19:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 msgr2=0x7fa9500a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 0x7fa9500a3bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa95c097810 tx=0x7fa95c0a1040 comp rx=0 tx=0).stop 2023-10-06T21:19:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 msgr2=0x7fa96810fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009810 tx=0x7fa95c00b790 comp rx=0 tx=0).stop 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 shutdown_connections 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa968105e20 0x7fa9681a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa9500a1730 0x7fa9500a3bf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa968068810 0x7fa96810fe90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 --2- 172.21.15.26:0/1868322941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9681038b0 0x7fa9681103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 >> 172.21.15.26:0/1868322941 conn(0x7fa968076b70 msgr2=0x7fa968102890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 shutdown_connections 2023-10-06T21:19:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:17.010+0000 7fa96d5ff700 1 -- 172.21.15.26:0/1868322941 wait complete. 2023-10-06T21:19:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 515.0 s, 1529.9 tps, lat 65.448 ms stddev 68.254 2023-10-06T21:19:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:18 smithi167 ceph-mon[134494]: pgmap v278: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:19:18.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:18 smithi167 ceph-mon[134494]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:19:18.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:18 smithi026 ceph-mon[98490]: pgmap v278: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:19:18.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:18 smithi026 ceph-mon[98490]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:19:18.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:18 smithi078 ceph-mon[134896]: pgmap v278: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:19:18.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:18 smithi078 ceph-mon[134896]: osdmap e95: 12 total, 12 up, 12 in 2023-10-06T21:19:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:20 smithi167 ceph-mon[134494]: pgmap v280: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 684 op/s 2023-10-06T21:19:20.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:20 smithi026 ceph-mon[98490]: pgmap v280: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 684 op/s 2023-10-06T21:19:20.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:20 smithi078 ceph-mon[134896]: pgmap v280: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 684 op/s 2023-10-06T21:19:22.130 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 520.0 s, 1518.4 tps, lat 65.154 ms stddev 70.445 2023-10-06T21:19:22.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:22 smithi167 ceph-mon[134494]: pgmap v281: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-06T21:19:22.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:22 smithi026 ceph-mon[98490]: pgmap v281: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-06T21:19:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:22 smithi078 ceph-mon[134896]: pgmap v281: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 712 op/s 2023-10-06T21:19:24.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:24 smithi167 ceph-mon[134494]: pgmap v282: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-10-06T21:19:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:24 smithi026 ceph-mon[98490]: pgmap v282: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-10-06T21:19:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:24 smithi078 ceph-mon[134896]: pgmap v282: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-10-06T21:19:26.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:26 smithi167 ceph-mon[134494]: pgmap v283: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 733 op/s 2023-10-06T21:19:26.841 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:26 smithi026 ceph-mon[98490]: pgmap v283: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 733 op/s 2023-10-06T21:19:26.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:26 smithi078 ceph-mon[134896]: pgmap v283: 97 pgs: 97 active+clean; 5.9 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 733 op/s 2023-10-06T21:19:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 525.0 s, 1749.1 tps, lat 57.694 ms stddev 61.084 2023-10-06T21:19:28.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:28 smithi167 ceph-mon[134494]: pgmap v284: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 679 op/s 2023-10-06T21:19:28.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:28 smithi026 ceph-mon[98490]: pgmap v284: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 679 op/s 2023-10-06T21:19:28.883 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:28 smithi078 ceph-mon[134896]: pgmap v284: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 679 op/s 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.236+0000 7fde353ff700 1 -- 172.21.15.26:0/4115021848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3010d5a0 msgr2=0x7fde3010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.236+0000 7fde353ff700 1 --2- 172.21.15.26:0/4115021848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3010d5a0 0x7fde3010d980 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fde20009920 tx=0x7fde20009c30 comp rx=0 tx=0).stop 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 -- 172.21.15.26:0/4115021848 shutdown_connections 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 --2- 172.21.15.26:0/4115021848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde30107ea0 0x7fde30114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 --2- 172.21.15.26:0/4115021848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3010d5a0 0x7fde3010d980 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 --2- 172.21.15.26:0/4115021848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 -- 172.21.15.26:0/4115021848 >> 172.21.15.26:0/4115021848 conn(0x7fde30076cd0 msgr2=0x7fde30105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 -- 172.21.15.26:0/4115021848 shutdown_connections 2023-10-06T21:19:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.237+0000 7fde353ff700 1 -- 172.21.15.26:0/4115021848 wait complete. 2023-10-06T21:19:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.238+0000 7fde353ff700 1 Processor -- start 2023-10-06T21:19:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 -- start start 2023-10-06T21:19:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde30107ea0 0x7fde30072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3010d5a0 0x7fde3006b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006ba00 con 0x7fde3010d5a0 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006bb70 con 0x7fde30107ea0 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.240+0000 7fde353ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006bd30 con 0x7fde301074e0 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38756/0 (socket says 172.21.15.26:38756) 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 -- 172.21.15.26:0/258360886 learned_addr learned my addr 172.21.15.26:0/258360886 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde30107ea0 0x7fde30072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 -- 172.21.15.26:0/258360886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde30107ea0 msgr2=0x7fde30072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde30107ea0 0x7fde30072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 -- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3010d5a0 msgr2=0x7fde3006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3010d5a0 0x7fde3006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 -- 172.21.15.26:0/258360886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240095f0 con 0x7fde301074e0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.241+0000 7fde2effd700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde20009840 tx=0x7fde2000b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.242+0000 7fde17fff700 1 -- 172.21.15.26:0/258360886 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2001c070 con 0x7fde301074e0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.242+0000 7fde17fff700 1 -- 172.21.15.26:0/258360886 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde20005040 con 0x7fde301074e0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.242+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde20009520 con 0x7fde301074e0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.242+0000 7fde17fff700 1 -- 172.21.15.26:0/258360886 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20003aa0 con 0x7fde301074e0 2023-10-06T21:19:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.242+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3006ffa0 con 0x7fde301074e0 2023-10-06T21:19:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.246+0000 7fde17fff700 1 -- 172.21.15.26:0/258360886 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fde20020030 con 0x7fde301074e0 2023-10-06T21:19:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.248+0000 7fde48c4cb80 1 Processor -- start 2023-10-06T21:19:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.250+0000 7fde17fff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 0x7fde180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.250+0000 7fde17fff700 1 -- 172.21.15.26:0/258360886 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fde200ce1e0 con 0x7fde301074e0 2023-10-06T21:19:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 -- start start 2023-10-06T21:19:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 0x7fde180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acce15400 con 0x555acceac520 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acce06bc0 con 0x555acceb73d0 2023-10-06T21:19:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.252+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acce044e0 con 0x555acceabc00 2023-10-06T21:19:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42376/0 (socket says 172.21.15.26:42376) 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2588698441 learned_addr learned my addr 172.21.15.26:0/2588698441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2effd700 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2588698441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 msgr2=0x555acceabfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceabfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2588698441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 msgr2=0x555acceb6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.253+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2588698441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555accc2ad80 con 0x555acceb73d0 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 0x7fde180a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde28007550 tx=0x7fde28007910 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fde280060b0 tx=0x7fde2801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedf24517c1998bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde157fa700 1 -- 172.21.15.26:0/2588698441 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2801ae30 con 0x555acceb73d0 2023-10-06T21:19:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde157fa700 1 -- 172.21.15.26:0/2588698441 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde2800f050 con 0x555acceb73d0 2023-10-06T21:19:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.254+0000 7fde157fa700 1 -- 172.21.15.26:0/2588698441 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28018890 con 0x555acceb73d0 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.255+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2588698441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 msgr2=0x555acceb97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.255+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fde280060b0 tx=0x7fde2801b000 comp rx=0 tx=0).stop 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2588698441 shutdown_connections 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb97c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceabfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2588698441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2588698441 >> 172.21.15.26:0/2588698441 conn(0x555acce21200 msgr2=0x555accea8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2588698441 shutdown_connections 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.256+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2588698441 wait complete. 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.257+0000 7fde48c4cb80 1 -- start start 2023-10-06T21:19:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.257+0000 7fde48c4cb80 1 Processor -- start 2023-10-06T21:19:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceabc00 0x555acceb41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accdffdd0 con 0x555acceac520 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accd9c6b0 con 0x555acceabc00 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accd9c350 con 0x555acceb73d0 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde48c4cb80 10 client.0 _ll_register_callbacks cb 0x555accbf97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38774/0 (socket says 172.21.15.26:38774) 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2814855533 learned_addr learned my addr 172.21.15.26:0/2814855533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.258+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceabc00 msgr2=0x555acceb41a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceabc00 0x555acceb41a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2814855533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 msgr2=0x555acceb46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 -- 172.21.15.26:0/2814855533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde24009ad0 con 0x555acceb73d0 2023-10-06T21:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde2801aee0 tx=0x7fde28023d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e098f08d2f516846 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.259+0000 7fde12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28027070 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555acce9fd50 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fde28023680 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde28023dc0 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde48c4cb80 10 client.26608 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fde2801a430 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.260+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555acce9fd50 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fde2802d460 con 0x555acceb73d0 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 10 client.26608 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 20 client.26608 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555acced7f70 0x555acceda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 4 client.26608 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555accdf8db0 con 0x555acced7f70 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.261+0000 7fde2effd700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555acced7f70 0x555acceda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.262+0000 7fde2effd700 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555acced7f70 0x555acceda390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc758593dd6cd93 server_cookie=8f7826317448bd84 in_seq=0 out_seq=0 2023-10-06T21:19:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.262+0000 7fde12ff5700 10 client.26608 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.262+0000 7fde12ff5700 1 -- 172.21.15.26:0/2814855533 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555accdf8db0 con 0x555acced7f70 2023-10-06T21:19:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.262+0000 7fde12ff5700 10 client.26608 handle_command_reply: tid=0 2023-10-06T21:19:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.262+0000 7fde48c4cb80 1 client.26608 shutdown 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 msgr2=0x555acceb4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde2801aee0 tx=0x7fde28023d20 comp rx=0 tx=0).stop 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 shutdown_connections 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceb73d0 0x555acceb4c20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555acced7f70 0x555acceda390 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceabc00 0x555acceb41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2814855533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 >> 172.21.15.26:0/2814855533 conn(0x555acce21200 msgr2=0x555accea7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde12ff5700 0 client.26608 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 shutdown_connections 2023-10-06T21:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2814855533 wait complete. 2023-10-06T21:19:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.263+0000 7fde48c4cb80 20 client.26608 trim_cache size 0 max 16384 2023-10-06T21:19:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.285+0000 7fde48c4cb80 1 Processor -- start 2023-10-06T21:19:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.285+0000 7fde48c4cb80 1 -- start start 2023-10-06T21:19:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceaec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accc808c0 con 0x555acceac520 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acceb0010 con 0x555acceb73d0 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555acceafd60 con 0x555acceabc00 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42392/0 (socket says 172.21.15.26:42392) 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2f7fe700 1 -- 172.21.15.26:0/1359955105 learned_addr learned my addr 172.21.15.26:0/1359955105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceaec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 -- 172.21.15.26:0/1359955105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 msgr2=0x555acceb4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 -- 172.21.15.26:0/1359955105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 msgr2=0x555acceb3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.286+0000 7fde2effd700 1 -- 172.21.15.26:0/1359955105 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555accc13c00 con 0x555acceabc00 2023-10-06T21:19:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.287+0000 7fde2effd700 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceaec50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde200cb8c0 tx=0x7fde20097520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1296ec3e9ac1a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.287+0000 7fde15ffb700 1 -- 172.21.15.26:0/1359955105 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2009f160 con 0x555acceabc00 2023-10-06T21:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.287+0000 7fde15ffb700 1 -- 172.21.15.26:0/1359955105 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde20021380 con 0x555acceabc00 2023-10-06T21:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.287+0000 7fde15ffb700 1 -- 172.21.15.26:0/1359955105 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20099580 con 0x555acceabc00 2023-10-06T21:19:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 -- 172.21.15.26:0/1359955105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 msgr2=0x555acceaec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceaec50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde200cb8c0 tx=0x7fde20097520 comp rx=0 tx=0).stop 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 -- 172.21.15.26:0/1359955105 shutdown_connections 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceb73d0 0x555acceb4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555acceaec50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/1359955105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceac520 0x555acceb3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.288+0000 7fde48c4cb80 1 -- 172.21.15.26:0/1359955105 >> 172.21.15.26:0/1359955105 conn(0x555acce21200 msgr2=0x555acceb4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.289+0000 7fde48c4cb80 1 -- 172.21.15.26:0/1359955105 shutdown_connections 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.289+0000 7fde48c4cb80 1 -- 172.21.15.26:0/1359955105 wait complete. 2023-10-06T21:19:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.289+0000 7fde48c4cb80 1 -- start start 2023-10-06T21:19:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 Processor -- start 2023-10-06T21:19:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555accece940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceb73d0 0x555acceccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accecd620 con 0x555acceb73d0 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accecd790 con 0x555acceac520 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555accecd950 con 0x555acceabc00 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.290+0000 7fde48c4cb80 10 client.0 _ll_register_callbacks cb 0x7fde240098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42396/0 (socket says 172.21.15.26:42396) 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 -- 172.21.15.26:0/2432429964 learned_addr learned my addr 172.21.15.26:0/2432429964 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2f7fe700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceb73d0 0x555acceccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 -- 172.21.15.26:0/2432429964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 msgr2=0x555accece940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555accece940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 -- 172.21.15.26:0/2432429964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceb73d0 msgr2=0x555acceccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceb73d0 0x555acceccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.291+0000 7fde2e7fc700 1 -- 172.21.15.26:0/2432429964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3006ffa0 con 0x555acceac520 2023-10-06T21:19:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.292+0000 7fde2e7fc700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde24013d00 tx=0x7fde24011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be00e64fe2205336 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.292+0000 7fde12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.292+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24021070 con 0x555acceac520 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.292+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fde24011cb0 con 0x555acceac520 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.293+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde2800f050 con 0x555acceac520 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.293+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde24011e70 con 0x555acceac520 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.293+0000 7fde48c4cb80 10 client.26702 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.293+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555accc511f0 con 0x555acceac520 2023-10-06T21:19:30.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde2800f050 con 0x555acceac520 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fde24024020 con 0x555acceac520 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 10 client.26702 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 10 client.26702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 20 client.26702 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555accf2d160 0x555accf2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 4 client.26702 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:19:30.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.294+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555accdf8ad0 con 0x555accf2d160 2023-10-06T21:19:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.295+0000 7fde2effd700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555accf2d160 0x555accf2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.308+0000 7fde2effd700 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555accf2d160 0x555accf2f580 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0497b42cf545082 server_cookie=b2cefb139a5252bd in_seq=0 out_seq=0 2023-10-06T21:19:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.308+0000 7fde12ff5700 10 client.26702 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.406+0000 7fde12ff5700 1 -- 172.21.15.26:0/2432429964 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555accdf8ad0 con 0x555accf2d160 2023-10-06T21:19:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.406+0000 7fde12ff5700 10 client.26702 handle_command_reply: tid=0 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.406+0000 7fde48c4cb80 1 client.26702 shutdown 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.406+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 msgr2=0x555accecee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.406+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde24013d00 tx=0x7fde24011720 comp rx=0 tx=0).stop 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 shutdown_connections 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555acceb73d0 0x555acceccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555acceabc00 0x555accece940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555accf2d160 0x555accf2f580 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 --2- 172.21.15.26:0/2432429964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555acceac520 0x555accecee80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 >> 172.21.15.26:0/2432429964 conn(0x555acce21200 msgr2=0x555acceb49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde12ff5700 0 client.26702 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 shutdown_connections 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 1 -- 172.21.15.26:0/2432429964 wait complete. 2023-10-06T21:19:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.407+0000 7fde48c4cb80 20 client.26702 trim_cache size 0 max 16384 2023-10-06T21:19:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:19:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:19:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:19:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 msgr2=0x7fde180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 0x7fde180a3b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde28007550 tx=0x7fde28007910 comp rx=0 tx=0).stop 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 msgr2=0x7fde30072410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde20009840 tx=0x7fde2000b560 comp rx=0 tx=0).stop 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 shutdown_connections 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fde180a1670 0x7fde180a3b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3010d5a0 0x7fde3006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde301074e0 0x7fde30072410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 --2- 172.21.15.26:0/258360886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde30107ea0 0x7fde30072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 >> 172.21.15.26:0/258360886 conn(0x7fde30076cd0 msgr2=0x7fde30077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 shutdown_connections 2023-10-06T21:19:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:30.408+0000 7fde353ff700 1 -- 172.21.15.26:0/258360886 wait complete. 2023-10-06T21:19:30.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:30 smithi167 ceph-mon[134494]: pgmap v285: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 607 op/s 2023-10-06T21:19:30.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:30 smithi026 ceph-mon[98490]: pgmap v285: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 607 op/s 2023-10-06T21:19:30.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:30 smithi078 ceph-mon[134896]: pgmap v285: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 607 op/s 2023-10-06T21:19:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 530.0 s, 1790.6 tps, lat 55.465 ms stddev 66.152 2023-10-06T21:19:32.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:32 smithi167 ceph-mon[134494]: pgmap v286: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 833 op/s 2023-10-06T21:19:32.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:32 smithi026 ceph-mon[98490]: pgmap v286: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 833 op/s 2023-10-06T21:19:32.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:32 smithi078 ceph-mon[134896]: pgmap v286: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 833 op/s 2023-10-06T21:19:34.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:19:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.355+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/2047572911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa14068c60 msgr2=0x7ffa14105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.355+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/2047572911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa14068c60 0x7ffa14105920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009920 tx=0x7ffa04009c30 comp rx=0 tx=0).stop 2023-10-06T21:19:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.357+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/2047572911 shutdown_connections 2023-10-06T21:19:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.357+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/2047572911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 0x7ffa1410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.357+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/2047572911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140682b0 0x7ffa14068690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.357+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/2047572911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa14068c60 0x7ffa14105920 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.357+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/2047572911 >> 172.21.15.26:0/2047572911 conn(0x7ffa140769d0 msgr2=0x7ffa14076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.358+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/2047572911 shutdown_connections 2023-10-06T21:19:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.358+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/2047572911 wait complete. 2023-10-06T21:19:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.359+0000 7ffa1ab9d700 1 Processor -- start 2023-10-06T21:19:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.359+0000 7ffa1ab9d700 1 -- start start 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.359+0000 7ffa1ab9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.359+0000 7ffa1ab9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 0x7ffa14100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1ab9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 0x7ffa141a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1ab9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a5600 con 0x7ffa140682b0 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1ab9d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a5770 con 0x7ffa14100980 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1ab9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a5930 con 0x7ffa14106050 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56620/0 (socket says 172.21.15.26:56620) 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 -- 172.21.15.26:0/3913399295 learned_addr learned my addr 172.21.15.26:0/3913399295 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1939a700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 0x7ffa14100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 0x7ffa141a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 -- 172.21.15.26:0/3913399295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 msgr2=0x7ffa14100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 0x7ffa14100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 -- 172.21.15.26:0/3913399295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 msgr2=0x7ffa141a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 0x7ffa141a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa19b9b700 1 -- 172.21.15.26:0/3913399295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009520 con 0x7ffa140682b0 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.360+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 0x7ffa141a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa19b9b700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa0affd700 1 -- 172.21.15.26:0/3913399295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004a30 con 0x7ffa140682b0 2023-10-06T21:19:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa0affd700 1 -- 172.21.15.26:0/3913399295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa1000cc80 con 0x7ffa140682b0 2023-10-06T21:19:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa141a5c10 con 0x7ffa140682b0 2023-10-06T21:19:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa0affd700 1 -- 172.21.15.26:0/3913399295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa100054e0 con 0x7ffa140682b0 2023-10-06T21:19:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.361+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141a6120 con 0x7ffa140682b0 2023-10-06T21:19:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.368+0000 7ffa0affd700 1 -- 172.21.15.26:0/3913399295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffa10004b90 con 0x7ffa140682b0 2023-10-06T21:19:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.369+0000 7ffa2f7f8b80 1 Processor -- start 2023-10-06T21:19:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.369+0000 7ffa2f7f8b80 1 -- start start 2023-10-06T21:19:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.369+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 0x55ed3facd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.369+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 0x55ed3fad8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3fad8a70 0x55ed3fadae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fa36460 con 0x55ed3fad8a70 2023-10-06T21:19:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fa27c20 con 0x55ed3facdbc0 2023-10-06T21:19:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fa25540 con 0x55ed3facd2a0 2023-10-06T21:19:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa19b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 0x55ed3fad8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa19b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 0x55ed3fad8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48564/0 (socket says 172.21.15.26:48564) 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa19b9b700 1 -- 172.21.15.26:0/487336230 learned_addr learned my addr 172.21.15.26:0/487336230 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.370+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 0x55ed3facd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 -- 172.21.15.26:0/487336230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 msgr2=0x55ed3fad8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 0x55ed3fad8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 -- 172.21.15.26:0/487336230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3fad8a70 msgr2=0x55ed3fadae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3fad8a70 0x55ed3fadae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 -- 172.21.15.26:0/487336230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed3f84bde0 con 0x55ed3facd2a0 2023-10-06T21:19:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.371+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 0x55ed3facd680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c003db0 tx=0x7ffa0c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0ee38386bc67ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.372+0000 7ff9fffff700 1 -- 172.21.15.26:0/487336230 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004bb0 con 0x55ed3facd2a0 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.372+0000 7ff9fffff700 1 -- 172.21.15.26:0/487336230 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c00f040 con 0x55ed3facd2a0 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.372+0000 7ff9fffff700 1 -- 172.21.15.26:0/487336230 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007c20 con 0x55ed3facd2a0 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.372+0000 7ffa0affd700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 0x7ffa000a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa0affd700 1 -- 172.21.15.26:0/3913399295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffa10014070 con 0x7ffa140682b0 2023-10-06T21:19:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/487336230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 msgr2=0x55ed3facd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 0x55ed3facd680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c003db0 tx=0x7ffa0c009920 comp rx=0 tx=0).stop 2023-10-06T21:19:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/487336230 shutdown_connections 2023-10-06T21:19:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3fad8a70 0x55ed3fadae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facd2a0 0x55ed3facd680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/487336230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3facdbc0 0x55ed3fad8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.373+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/487336230 >> 172.21.15.26:0/487336230 conn(0x55ed3fa42260 msgr2=0x55ed3fac9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.374+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/487336230 shutdown_connections 2023-10-06T21:19:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.374+0000 7ffa19b9b700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 0x7ffa000a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.374+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/487336230 wait complete. 2023-10-06T21:19:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 -- start start 2023-10-06T21:19:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa19b9b700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 0x7ffa000a3b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa100997c0 tx=0x7ffa100cdf60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 Processor -- start 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fad57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fa20e30 con 0x55ed3facd2a0 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3f9bd710 con 0x55ed3fad8a70 2023-10-06T21:19:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.375+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3f9bd3b0 con 0x55ed3facdbc0 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.376+0000 7ffa2f7f8b80 10 client.0 _ll_register_callbacks cb 0x55ed3f81a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.376+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fad57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.376+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fad57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56632/0 (socket says 172.21.15.26:56632) 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.376+0000 7ffa1a39c700 1 -- 172.21.15.26:0/614212372 learned_addr learned my addr 172.21.15.26:0/614212372 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.376+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 -- 172.21.15.26:0/614212372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 msgr2=0x55ed3fad5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 -- 172.21.15.26:0/614212372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 msgr2=0x55ed3fad57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fad57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 -- 172.21.15.26:0/614212372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c003a60 con 0x55ed3fad8a70 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009b00 tx=0x7ffa0c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98a17152bc0fef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.377+0000 7ff9fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.378+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007bc0 con 0x55ed3fad8a70 2023-10-06T21:19:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.378+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffa0c007d20 con 0x55ed3fad8a70 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.380+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed3fac1160 con 0x55ed3fad8a70 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.380+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0c00e4f0 con 0x55ed3fad8a70 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.380+0000 7ffa2f7f8b80 10 client.26714 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.380+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ed3f834c60 con 0x55ed3fad8a70 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed3fac1160 con 0x55ed3fad8a70 2023-10-06T21:19:34.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ffa0c015450 con 0x55ed3fad8a70 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 10 client.26714 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 20 client.26714 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3faf0c10 0x55ed3faf3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 4 client.26714 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed3fa19e10 con 0x55ed3faf0c10 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.381+0000 7ffa1939a700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3faf0c10 0x55ed3faf3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.382+0000 7ffa1939a700 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3faf0c10 0x55ed3faf3030 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50887f7320ae0972 server_cookie=f70ef29710793434 in_seq=0 out_seq=0 2023-10-06T21:19:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.382+0000 7ff9fd7fa700 10 client.26714 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:34.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.384+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/614212372 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed3fa19e10 con 0x55ed3faf0c10 2023-10-06T21:19:34.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.384+0000 7ff9fd7fa700 10 client.26714 handle_command_reply: tid=0 2023-10-06T21:19:34.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.385+0000 7ffa2f7f8b80 1 client.26714 shutdown 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 msgr2=0x55ed3fad6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009b00 tx=0x7ffa0c00b040 comp rx=0 tx=0).stop 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 shutdown_connections 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3faf0c10 0x55ed3faf3030 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6250 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fad57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/614212372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 >> 172.21.15.26:0/614212372 conn(0x55ed3fa42260 msgr2=0x55ed3fac91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ff9fd7fa700 0 client.26714 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:34.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 shutdown_connections 2023-10-06T21:19:34.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/614212372 wait complete. 2023-10-06T21:19:34.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.386+0000 7ffa2f7f8b80 20 client.26714 trim_cache size 0 max 16384 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.419+0000 7ffa2f7f8b80 1 Processor -- start 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.419+0000 7ffa2f7f8b80 1 -- start start 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fae7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3f8a1920 con 0x55ed3facd2a0 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3facdfa0 con 0x55ed3fad8a70 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.420+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3face4c0 con 0x55ed3facdbc0 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.421+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.421+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48580/0 (socket says 172.21.15.26:48580) 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.421+0000 7ffa1a39c700 1 -- 172.21.15.26:0/3045918700 learned_addr learned my addr 172.21.15.26:0/3045918700 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.421+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fae7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1939a700 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1a39c700 1 -- 172.21.15.26:0/3045918700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 msgr2=0x55ed3fad5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1a39c700 1 -- 172.21.15.26:0/3045918700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 msgr2=0x55ed3fad6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.422+0000 7ffa1a39c700 1 -- 172.21.15.26:0/3045918700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed3faf7c60 con 0x55ed3facd2a0 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.423+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fae7dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c015810 tx=0x7ffa0c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df10ee5024ee427 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.424+0000 7ffa08bff700 1 -- 172.21.15.26:0/3045918700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c015600 con 0x55ed3facd2a0 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.424+0000 7ffa08bff700 1 -- 172.21.15.26:0/3045918700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c028070 con 0x55ed3facd2a0 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.424+0000 7ffa08bff700 1 -- 172.21.15.26:0/3045918700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00e950 con 0x55ed3facd2a0 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.425+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/3045918700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 msgr2=0x55ed3fae7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.425+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fae7dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c015810 tx=0x7ffa0c000f40 comp rx=0 tx=0).stop 2023-10-06T21:19:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.431+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/3045918700 shutdown_connections 2023-10-06T21:19:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.431+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fad6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.431+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fae7dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.431+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/3045918700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fad5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.431+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/3045918700 >> 172.21.15.26:0/3045918700 conn(0x55ed3fa42260 msgr2=0x55ed3fad0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.435+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/3045918700 shutdown_connections 2023-10-06T21:19:34.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.436+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/3045918700 wait complete. 2023-10-06T21:19:34.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.437+0000 7ffa2f7f8b80 1 -- start start 2023-10-06T21:19:34.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.437+0000 7ffa2f7f8b80 1 Processor -- start 2023-10-06T21:19:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.440+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fb3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.440+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fb3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.440+0000 7ffa2f7f8b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.440+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fae79a0 con 0x55ed3facd2a0 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.440+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fae7b10 con 0x55ed3fad8a70 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.441+0000 7ffa2f7f8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3fae7cd0 con 0x55ed3facdbc0 2023-10-06T21:19:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.441+0000 7ffa2f7f8b80 10 client.0 _ll_register_callbacks cb 0x7ffa0c004ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.441+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.441+0000 7ffa1a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48584/0 (socket says 172.21.15.26:48584) 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.441+0000 7ffa1a39c700 1 -- 172.21.15.26:0/771584275 learned_addr learned my addr 172.21.15.26:0/771584275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 -- 172.21.15.26:0/771584275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 msgr2=0x55ed3fb3da70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fb3da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 -- 172.21.15.26:0/771584275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 msgr2=0x55ed3fb3d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fb3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 -- 172.21.15.26:0/771584275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed3f834c60 con 0x55ed3fad8a70 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.442+0000 7ffa1a39c700 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c022040 tx=0x7ffa0c0154e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1bf6c8b961a465 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.443+0000 7ff9fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.443+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007c80 con 0x55ed3fad8a70 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.443+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffa0c007de0 con 0x55ed3fad8a70 2023-10-06T21:19:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.443+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa0c00f040 con 0x55ed3fad8a70 2023-10-06T21:19:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.444+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa0c0090d0 con 0x55ed3fad8a70 2023-10-06T21:19:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.444+0000 7ffa2f7f8b80 10 client.26720 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.444+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ed3f872250 con 0x55ed3fad8a70 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c00f040 con 0x55ed3fad8a70 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7ffa0c03e070 con 0x55ed3fad8a70 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 10 client.26720 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 20 client.26720 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:34.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3fadc080 0x55ed3fade4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 4 client.26720 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:19:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.445+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed3fa19b30 con 0x55ed3fadc080 2023-10-06T21:19:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.446+0000 7ffa1939a700 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3fadc080 0x55ed3fade4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:34.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.448+0000 7ffa1939a700 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3fadc080 0x55ed3fade4a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d395a5f6bff89b server_cookie=cf3a079698ab9a0e in_seq=0 out_seq=0 2023-10-06T21:19:34.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.448+0000 7ff9fd7fa700 10 client.26720 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:34.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.449+0000 7ff9fd7fa700 1 -- 172.21.15.26:0/771584275 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ed3fa19b30 con 0x55ed3fadc080 2023-10-06T21:19:34.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.449+0000 7ff9fd7fa700 10 client.26720 handle_command_reply: tid=0 2023-10-06T21:19:34.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.449+0000 7ffa2f7f8b80 1 client.26720 shutdown 2023-10-06T21:19:34.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 msgr2=0x55ed3fae73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c022040 tx=0x7ffa0c0154e0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 shutdown_connections 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ed3fadc080 0x55ed3fade4a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed3fad8a70 0x55ed3fae73c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ed3facd2a0 0x55ed3fb3d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 --2- 172.21.15.26:0/771584275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed3facdbc0 0x55ed3fb3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 >> 172.21.15.26:0/771584275 conn(0x55ed3fa42260 msgr2=0x55ed3fad5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.450+0000 7ff9fd7fa700 0 client.26720 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.452+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 shutdown_connections 2023-10-06T21:19:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.452+0000 7ffa2f7f8b80 1 -- 172.21.15.26:0/771584275 wait complete. 2023-10-06T21:19:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.453+0000 7ffa2f7f8b80 20 client.26720 trim_cache size 0 max 16384 2023-10-06T21:19:34.456 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:19:34.456 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:19:34.456 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:19:34.456 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.453+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 msgr2=0x7ffa000a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 0x7ffa000a3b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa100997c0 tx=0x7ffa100cdf60 comp rx=0 tx=0).stop 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 msgr2=0x7ffa140fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9c0 tx=0x7ffa1000ecd0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 shutdown_connections 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa14100980 0x7ffa141a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffa000a1690 0x7ffa000a3b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa140682b0 0x7ffa140fff00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 --2- 172.21.15.26:0/3913399295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa14106050 0x7ffa14100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 >> 172.21.15.26:0/3913399295 conn(0x7ffa140769d0 msgr2=0x7ffa140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 shutdown_connections 2023-10-06T21:19:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:34.454+0000 7ffa1ab9d700 1 -- 172.21.15.26:0/3913399295 wait complete. 2023-10-06T21:19:34.485 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:34 smithi026 ceph-mon[98490]: pgmap v287: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-10-06T21:19:34.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:19:34.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:19:34.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:34 smithi167 ceph-mon[134494]: pgmap v287: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-10-06T21:19:34.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:34 smithi078 ceph-mon[134896]: pgmap v287: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 646 op/s 2023-10-06T21:19:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 -- 172.21.15.26:0/2285515164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c11a690 msgr2=0x7ff65c11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 --2- 172.21.15.26:0/2285515164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c11a690 0x7ff65c11ca80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009990 tx=0x7ff64c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 -- 172.21.15.26:0/2285515164 shutdown_connections 2023-10-06T21:19:35.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 --2- 172.21.15.26:0/2285515164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c11a690 0x7ff65c11ca80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 --2- 172.21.15.26:0/2285515164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c10f640 0x7ff65c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 --2- 172.21.15.26:0/2285515164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c1083f0 0x7ff65c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.152+0000 7ff6617ff700 1 -- 172.21.15.26:0/2285515164 >> 172.21.15.26:0/2285515164 conn(0x7ff65c06bf20 msgr2=0x7ff65c06c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.165+0000 7ff6617ff700 1 -- 172.21.15.26:0/2285515164 shutdown_connections 2023-10-06T21:19:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.174+0000 7ff6617ff700 1 -- 172.21.15.26:0/2285515164 wait complete. 2023-10-06T21:19:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.176+0000 7ff6617ff700 1 Processor -- start 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.185+0000 7ff6617ff700 1 -- start start 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c11a690 0x7ff65c113d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c1142c0 0x7ff65c114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1bae30 con 0x7ff65c1142c0 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1baf70 con 0x7ff65c10f640 2023-10-06T21:19:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.186+0000 7ff6617ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1bb0e0 con 0x7ff65c11a690 2023-10-06T21:19:35.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.192+0000 7ff65affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:35.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.192+0000 7ff65affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48598/0 (socket says 172.21.15.26:48598) 2023-10-06T21:19:35.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.192+0000 7ff65affd700 1 -- 172.21.15.26:0/3565313556 learned_addr learned my addr 172.21.15.26:0/3565313556 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.194+0000 7ff65a7fc700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c11a690 0x7ff65c113d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.196+0000 7ff65affd700 1 -- 172.21.15.26:0/3565313556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c11a690 msgr2=0x7ff65c113d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.196+0000 7ff65affd700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c11a690 0x7ff65c113d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.196+0000 7ff65affd700 1 -- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c1142c0 msgr2=0x7ff65c114770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.196+0000 7ff65affd700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c1142c0 0x7ff65c114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.196+0000 7ff65affd700 1 -- 172.21.15.26:0/3565313556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0095f0 con 0x7ff65c10f640 2023-10-06T21:19:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.201+0000 7ff65affd700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6540060b0 tx=0x7ff65400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.202+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400ddd0 con 0x7ff65c10f640 2023-10-06T21:19:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.202+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff65400f040 con 0x7ff65c10f640 2023-10-06T21:19:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.202+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654014600 con 0x7ff65c10f640 2023-10-06T21:19:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.202+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c1bb3c0 con 0x7ff65c10f640 2023-10-06T21:19:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.202+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c1bb880 con 0x7ff65c10f640 2023-10-06T21:19:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.212+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c005320 con 0x7ff65c10f640 2023-10-06T21:19:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.223+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ff654004ad0 con 0x7ff65c10f640 2023-10-06T21:19:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.225+0000 7ff653fff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 0x7ff6480a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.225+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff6540cc980 con 0x7ff65c10f640 2023-10-06T21:19:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.226+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6540f91a0 con 0x7ff65c10f640 2023-10-06T21:19:35.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.276+0000 7ff65a7fc700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 0x7ff6480a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.315+0000 7ff65a7fc700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 0x7ff6480a3d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff65c115570 tx=0x7ff64400b390 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:35.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.692+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff63c005190 con 0x7ff65c10f640 2023-10-06T21:19:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.693+0000 7ff653fff700 1 -- 172.21.15.26:0/3565313556 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11314 (secure 0 0 0) 0x7ff65401a030 con 0x7ff65c10f640 2023-10-06T21:19:35.696 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:19:35.696 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:19:09.459935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:19:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.695+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 msgr2=0x7ff6480a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:35.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.695+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 0x7ff6480a3d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff65c115570 tx=0x7ff64400b390 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.695+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 msgr2=0x7ff65c119da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.695+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6540060b0 tx=0x7ff65400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 shutdown_connections 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ff6480a1850 0x7ff6480a3d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c1142c0 0x7ff65c114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c10f640 0x7ff65c119da0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 --2- 172.21.15.26:0/3565313556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff65c11a690 0x7ff65c113d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 >> 172.21.15.26:0/3565313556 conn(0x7ff65c06bf20 msgr2=0x7ff65c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 shutdown_connections 2023-10-06T21:19:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:35.696+0000 7ff6617ff700 1 -- 172.21.15.26:0/3565313556 wait complete. 2023-10-06T21:19:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-10-06T21:19:35.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:19:36.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.048+0000 7f63a55ff700 1 -- 172.21.15.26:0/1413926619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a010f640 msgr2=0x7f63a0107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.048+0000 7f63a55ff700 1 --2- 172.21.15.26:0/1413926619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a010f640 0x7f63a0107eb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f639001a9a0 tx=0x7f639001acb0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 -- 172.21.15.26:0/1413926619 shutdown_connections 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 --2- 172.21.15.26:0/1413926619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 --2- 172.21.15.26:0/1413926619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a010f640 0x7f63a0107eb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 --2- 172.21.15.26:0/1413926619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a01083f0 0x7f63a0108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 -- 172.21.15.26:0/1413926619 >> 172.21.15.26:0/1413926619 conn(0x7f63a006d1a0 msgr2=0x7f63a006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.051+0000 7f63a55ff700 1 -- 172.21.15.26:0/1413926619 shutdown_connections 2023-10-06T21:19:36.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.054+0000 7f63a55ff700 1 -- 172.21.15.26:0/1413926619 wait complete. 2023-10-06T21:19:36.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.056+0000 7f63a55ff700 1 Processor -- start 2023-10-06T21:19:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 -- start start 2023-10-06T21:19:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a01083f0 0x7f63a0115930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a010f640 0x7f63a0115e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a9910 con 0x7f63a01083f0 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a9a50 con 0x7f63a011a690 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.058+0000 7f63a55ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a9be0 con 0x7f63a010f640 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.059+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.059+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48604/0 (socket says 172.21.15.26:48604) 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.059+0000 7f639f7fe700 1 -- 172.21.15.26:0/4195272029 learned_addr learned my addr 172.21.15.26:0/4195272029 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.060+0000 7f639f7fe700 1 -- 172.21.15.26:0/4195272029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a010f640 msgr2=0x7f63a0115e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.060+0000 7f639f7fe700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a010f640 0x7f63a0115e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.060+0000 7f639f7fe700 1 -- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a01083f0 msgr2=0x7f63a0115930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.060+0000 7f639f7fe700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a01083f0 0x7f63a0115930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.060+0000 7f639f7fe700 1 -- 172.21.15.26:0/4195272029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639001a600 con 0x7f63a011a690 2023-10-06T21:19:36.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.064+0000 7f639f7fe700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6398007ee0 tx=0x7f639800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.069+0000 7f6387fff700 1 -- 172.21.15.26:0/4195272029 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800ddd0 con 0x7f63a011a690 2023-10-06T21:19:36.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.070+0000 7f6387fff700 1 -- 172.21.15.26:0/4195272029 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639800f040 con 0x7f63a011a690 2023-10-06T21:19:36.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.070+0000 7f6387fff700 1 -- 172.21.15.26:0/4195272029 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398005a20 con 0x7f63a011a690 2023-10-06T21:19:36.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.072+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a01a9ec0 con 0x7f63a011a690 2023-10-06T21:19:36.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.072+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a01aa320 con 0x7f63a011a690 2023-10-06T21:19:36.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.077+0000 7f63b8f95b80 1 Processor -- start 2023-10-06T21:19:36.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.080+0000 7f6387fff700 1 -- 172.21.15.26:0/4195272029 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6398004ad0 con 0x7f63a011a690 2023-10-06T21:19:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.080+0000 7f63b8f95b80 1 -- start start 2023-10-06T21:19:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434470c0 0x560b434474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434479e0 0x560b43452350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b433b0460 con 0x560b434470c0 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b433a1c20 con 0x560b434479e0 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.081+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b4339f540 con 0x560b43452890 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38498/0 (socket says 172.21.15.26:38498) 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 -- 172.21.15.26:0/1929603679 learned_addr learned my addr 172.21.15.26:0/1929603679 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434470c0 0x560b434474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 -- 172.21.15.26:0/1929603679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434479e0 msgr2=0x560b43452350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434479e0 0x560b43452350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 -- 172.21.15.26:0/1929603679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434470c0 msgr2=0x560b434474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434470c0 0x560b434474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.082+0000 7f639e7fc700 1 -- 172.21.15.26:0/1929603679 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b431c5de0 con 0x560b43452890 2023-10-06T21:19:36.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.083+0000 7f6387fff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 0x7f63880a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.083+0000 7f6387fff700 1 -- 172.21.15.26:0/4195272029 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63980cc970 con 0x7f63a011a690 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.083+0000 7f639e7fc700 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6394013480 tx=0x7f639400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b223c3d5cb8fe1c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.084+0000 7f63857fa700 1 -- 172.21.15.26:0/1929603679 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394013c90 con 0x560b43452890 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.084+0000 7f63857fa700 1 -- 172.21.15.26:0/1929603679 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63940044e0 con 0x560b43452890 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.084+0000 7f63857fa700 1 -- 172.21.15.26:0/1929603679 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394010400 con 0x560b43452890 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.085+0000 7f639effd700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 0x7f63880a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.085+0000 7f63b8f95b80 1 -- 172.21.15.26:0/1929603679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 msgr2=0x560b43454c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.085+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6394013480 tx=0x7f639400c2d0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.086+0000 7f63b8f95b80 1 -- 172.21.15.26:0/1929603679 shutdown_connections 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.086+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43454c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.086+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434470c0 0x560b434474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.086+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/1929603679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434479e0 0x560b43452350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.086+0000 7f63b8f95b80 1 -- 172.21.15.26:0/1929603679 >> 172.21.15.26:0/1929603679 conn(0x560b433bc260 msgr2=0x560b43443450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.087+0000 7f639effd700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 0x7f63880a3c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f639001a8f0 tx=0x7f639002a040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.087+0000 7f63b8f95b80 1 -- 172.21.15.26:0/1929603679 shutdown_connections 2023-10-06T21:19:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.087+0000 7f63b8f95b80 1 -- 172.21.15.26:0/1929603679 wait complete. 2023-10-06T21:19:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.087+0000 7f63b8f95b80 1 -- start start 2023-10-06T21:19:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 Processor -- start 2023-10-06T21:19:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b43452890 0x560b434500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b4339ae30 con 0x560b434479e0 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43337710 con 0x560b43452890 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.088+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b433373b0 con 0x560b434470c0 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f63b8f95b80 10 client.0 _ll_register_callbacks cb 0x560b43194800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38508/0 (socket says 172.21.15.26:38508) 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 -- 172.21.15.26:0/2246019956 learned_addr learned my addr 172.21.15.26:0/2246019956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639f7fe700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b43452890 0x560b434500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 -- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b43452890 msgr2=0x560b434500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b43452890 0x560b434500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 -- 172.21.15.26:0/2246019956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 msgr2=0x560b4344fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.089+0000 7f639effd700 1 -- 172.21.15.26:0/2246019956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639400cd10 con 0x560b434470c0 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.090+0000 7f639effd700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f639001ce50 tx=0x7f639001a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7782db0a81ecfd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.090+0000 7f6382ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:36.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390004870 con 0x560b434470c0 2023-10-06T21:19:36.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63900049d0 con 0x560b434470c0 2023-10-06T21:19:36.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b4343af10 con 0x560b434470c0 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6390004b90 con 0x560b434470c0 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f63b8f95b80 10 client.26635 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560b431aec60 con 0x560b434470c0 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.091+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639004a080 con 0x560b434470c0 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6390033480 con 0x560b434470c0 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 10 client.26635 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 10 client.26635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 20 client.26635 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434624f0 0x560b43464910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 4 client.26635 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b43393e10 con 0x560b434624f0 2023-10-06T21:19:36.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.092+0000 7f639e7fc700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434624f0 0x560b43464910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.093+0000 7f639e7fc700 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434624f0 0x560b43464910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4acc7bd3427a1750 server_cookie=41e2702feade0668 in_seq=0 out_seq=0 2023-10-06T21:19:36.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.102+0000 7f6382ff5700 10 client.26635 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:19:36.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.102+0000 7f6382ff5700 1 -- 172.21.15.26:0/2246019956 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b43393e10 con 0x560b434624f0 2023-10-06T21:19:36.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.102+0000 7f6382ff5700 10 client.26635 handle_command_reply: tid=0 2023-10-06T21:19:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.102+0000 7f63b8f95b80 1 client.26635 shutdown 2023-10-06T21:19:36.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 msgr2=0x560b4344f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f639001ce50 tx=0x7f639001a320 comp rx=0 tx=0).stop 2023-10-06T21:19:36.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 shutdown_connections 2023-10-06T21:19:36.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434624f0 0x560b43464910 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b43452890 0x560b434500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b434470c0 0x560b4344f670 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/2246019956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.105+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 >> 172.21.15.26:0/2246019956 conn(0x560b433bc260 msgr2=0x560b43443040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.106+0000 7f6382ff5700 0 client.26635 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.106+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 shutdown_connections 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.106+0000 7f63b8f95b80 1 -- 172.21.15.26:0/2246019956 wait complete. 2023-10-06T21:19:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.106+0000 7f63b8f95b80 20 client.26635 trim_cache size 0 max 16384 2023-10-06T21:19:36.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.127+0000 7f63b8f95b80 1 Processor -- start 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.127+0000 7f63b8f95b80 1 -- start start 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b4344e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b4321b920 con 0x560b434479e0 2023-10-06T21:19:36.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b4345fab0 con 0x560b434470c0 2023-10-06T21:19:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b43448330 con 0x560b43452890 2023-10-06T21:19:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f639effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f639effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48630/0 (socket says 172.21.15.26:48630) 2023-10-06T21:19:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.128+0000 7f639effd700 1 -- 172.21.15.26:0/377472912 learned_addr learned my addr 172.21.15.26:0/377472912 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.129+0000 7f639effd700 1 -- 172.21.15.26:0/377472912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 msgr2=0x560b4344e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.129+0000 7f639effd700 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b4344e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.129+0000 7f639effd700 1 -- 172.21.15.26:0/377472912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 msgr2=0x560b4344ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.129+0000 7f639effd700 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.129+0000 7f639effd700 1 -- 172.21.15.26:0/377472912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b43469560 con 0x560b434470c0 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.130+0000 7f639effd700 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f639001c5d0 tx=0x7f639001cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3faae648dc1c41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.131+0000 7f6385ffb700 1 -- 172.21.15.26:0/377472912 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390033680 con 0x560b434470c0 2023-10-06T21:19:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.131+0000 7f6385ffb700 1 -- 172.21.15.26:0/377472912 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b4343af10 con 0x560b434470c0 2023-10-06T21:19:36.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.131+0000 7f6385ffb700 1 -- 172.21.15.26:0/377472912 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390003d70 con 0x560b434470c0 2023-10-06T21:19:36.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.133+0000 7f63b8f95b80 1 -- 172.21.15.26:0/377472912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 msgr2=0x560b43449fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.133+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f639001c5d0 tx=0x7f639001cfb0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 -- 172.21.15.26:0/377472912 shutdown_connections 2023-10-06T21:19:36.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b4344e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b43449fa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/377472912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b4344ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 -- 172.21.15.26:0/377472912 >> 172.21.15.26:0/377472912 conn(0x560b433bc260 msgr2=0x560b4344de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 -- 172.21.15.26:0/377472912 shutdown_connections 2023-10-06T21:19:36.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.134+0000 7f63b8f95b80 1 -- 172.21.15.26:0/377472912 wait complete. 2023-10-06T21:19:36.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.135+0000 7f63b8f95b80 1 -- start start 2023-10-06T21:19:36.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.135+0000 7f63b8f95b80 1 Processor -- start 2023-10-06T21:19:36.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b43457900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43457e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b434b5340 con 0x560b434479e0 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b434b5480 con 0x560b434470c0 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b434b55c0 con 0x560b43452890 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f63b8f95b80 10 client.0 _ll_register_callbacks cb 0x7f639001ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f639effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f639effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48636/0 (socket says 172.21.15.26:48636) 2023-10-06T21:19:36.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.136+0000 7f639effd700 1 -- 172.21.15.26:0/3775331009 learned_addr learned my addr 172.21.15.26:0/3775331009 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639f7fe700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43457e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 -- 172.21.15.26:0/3775331009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 msgr2=0x560b43457e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43457e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 -- 172.21.15.26:0/3775331009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 msgr2=0x560b43457900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b43457900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 -- 172.21.15.26:0/3775331009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b431aec60 con 0x560b434470c0 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f639effd700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6390033830 tx=0x7f6390033c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11d794b3f9696bfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.137+0000 7f6382ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.138+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390004dc0 con 0x560b434470c0 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.138+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63900039f0 con 0x560b434470c0 2023-10-06T21:19:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.141+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63940044e0 con 0x560b434470c0 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.141+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6390003bb0 con 0x560b434470c0 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.142+0000 7f63b8f95b80 10 client.26741 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.142+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560b431ec250 con 0x560b434470c0 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63940044e0 con 0x560b434470c0 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f639003e900 con 0x560b434470c0 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 10 client.26741 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 10 client.26741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:19:36.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 20 client.26741 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434b6930 0x560b434b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 4 client.26741 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:19:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b43393b30 con 0x560b434b6930 2023-10-06T21:19:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.143+0000 7f639e7fc700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434b6930 0x560b434b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f639e7fc700 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434b6930 0x560b434b8d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a84fc5c279446b8f server_cookie=55c3be5e7500c163 in_seq=0 out_seq=0 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f6382ff5700 10 client.26741 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f6382ff5700 1 -- 172.21.15.26:0/3775331009 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b43393b30 con 0x560b434b6930 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f6382ff5700 10 client.26741 handle_command_reply: tid=0 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f63b8f95b80 1 client.26741 shutdown 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 msgr2=0x560b434573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.144+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6390033830 tx=0x7f6390033c60 comp rx=0 tx=0).stop 2023-10-06T21:19:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 shutdown_connections 2023-10-06T21:19:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x560b434b6930 0x560b434b8d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b43452890 0x560b43457e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b434470c0 0x560b434573c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 --2- 172.21.15.26:0/3775331009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b434479e0 0x560b43457900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 >> 172.21.15.26:0/3775331009 conn(0x560b433bc260 msgr2=0x560b4344e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f6382ff5700 0 client.26741 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 shutdown_connections 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.145+0000 7f63b8f95b80 1 -- 172.21.15.26:0/3775331009 wait complete. 2023-10-06T21:19:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.146+0000 7f63b8f95b80 20 client.26741 trim_cache size 0 max 16384 2023-10-06T21:19:36.152 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:19:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.146+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 msgr2=0x7f63880a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 0x7f63880a3c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f639001a8f0 tx=0x7f639002a040 comp rx=0 tx=0).stop 2023-10-06T21:19:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 msgr2=0x7f63a01a93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6398007ee0 tx=0x7f639800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 shutdown_connections 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63a011a690 0x7f63a01a93d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63880a1780 0x7f63880a3c40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63a01083f0 0x7f63a0115930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 --2- 172.21.15.26:0/4195272029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63a010f640 0x7f63a0115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 >> 172.21.15.26:0/4195272029 conn(0x7f63a006d1a0 msgr2=0x7f63a011aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 shutdown_connections 2023-10-06T21:19:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.147+0000 7f63a55ff700 1 -- 172.21.15.26:0/4195272029 wait complete. 2023-10-06T21:19:36.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:19:36.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:36 smithi026 ceph-mon[98490]: pgmap v288: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-06T21:19:36.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:36 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3565313556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:36.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.590+0000 7f9b127ff700 1 -- 172.21.15.26:0/248079396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c10b410 msgr2=0x7f9b0c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.590+0000 7f9b127ff700 1 --2- 172.21.15.26:0/248079396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c10b410 0x7f9b0c10d8d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400b0a0 tx=0x7f9b0400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.591+0000 7f9b127ff700 1 -- 172.21.15.26:0/248079396 shutdown_connections 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.591+0000 7f9b127ff700 1 --2- 172.21.15.26:0/248079396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c10b410 0x7f9b0c10d8d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.591+0000 7f9b127ff700 1 --2- 172.21.15.26:0/248079396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c101ff0 0x7f9b0c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.591+0000 7f9b127ff700 1 --2- 172.21.15.26:0/248079396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.591+0000 7f9b127ff700 1 -- 172.21.15.26:0/248079396 >> 172.21.15.26:0/248079396 conn(0x7f9b0c0fb830 msgr2=0x7f9b0c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.593+0000 7f9b127ff700 1 -- 172.21.15.26:0/248079396 shutdown_connections 2023-10-06T21:19:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.593+0000 7f9b127ff700 1 -- 172.21.15.26:0/248079396 wait complete. 2023-10-06T21:19:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.595+0000 7f9b127ff700 1 Processor -- start 2023-10-06T21:19:36.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.595+0000 7f9b127ff700 1 -- start start 2023-10-06T21:19:36.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c101ff0 0x7f9b0c0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c10b410 0x7f9b0c1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c1009b0 con 0x7f9b0c101ff0 2023-10-06T21:19:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c1a58b0 con 0x7f9b0c102910 2023-10-06T21:19:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b127ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c1a59f0 con 0x7f9b0c10b410 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48650/0 (socket says 172.21.15.26:48650) 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.596+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3078028246 learned_addr learned my addr 172.21.15.26:0/3078028246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c101ff0 0x7f9b0c0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c10b410 0x7f9b0c1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3078028246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c10b410 msgr2=0x7f9b0c1a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c10b410 0x7f9b0c1a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c101ff0 msgr2=0x7f9b0c0ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c101ff0 0x7f9b0c0ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3078028246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04009d00 con 0x7f9b0c102910 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b460 tx=0x7f9b0000b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b097fa700 1 -- 172.21.15.26:0/3078028246 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00011830 con 0x7f9b0c102910 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b097fa700 1 -- 172.21.15.26:0/3078028246 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b00011e50 con 0x7f9b0c102910 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b097fa700 1 -- 172.21.15.26:0/3078028246 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000f2e0 con 0x7f9b0c102910 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.597+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c196f50 con 0x7f9b0c102910 2023-10-06T21:19:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.598+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c1973e0 con 0x7f9b0c102910 2023-10-06T21:19:36.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.602+0000 7f9b097fa700 1 -- 172.21.15.26:0/3078028246 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9b00011990 con 0x7f9b0c102910 2023-10-06T21:19:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.604+0000 7f9b097fa700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 0x7f9af40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.604+0000 7f9b097fa700 1 -- 172.21.15.26:0/3078028246 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b000ccb00 con 0x7f9b0c102910 2023-10-06T21:19:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.604+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 0x7f9af40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.605+0000 7f9b2621cb80 1 Processor -- start 2023-10-06T21:19:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.605+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 0x7f9af40a3b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009910 tx=0x7f9afc008040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 -- start start 2023-10-06T21:19:36.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af8bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af81b460 con 0x55b5af8bd890 2023-10-06T21:19:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af80cc20 con 0x55b5af8b20c0 2023-10-06T21:19:36.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.606+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af80a540 con 0x55b5af8b29e0 2023-10-06T21:19:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38550/0 (socket says 172.21.15.26:38550) 2023-10-06T21:19:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3640195003 learned_addr learned my addr 172.21.15.26:0/3640195003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3640195003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 msgr2=0x55b5af8b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3640195003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 msgr2=0x55b5af8bfc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af8bfc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3640195003 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5af630de0 con 0x55b5af8b29e0 2023-10-06T21:19:36.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.607+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b00096080 tx=0x7f9b000994e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61da44e0fa8d4aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.611+0000 7f9af27fc700 1 -- 172.21.15.26:0/3640195003 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0009dcd0 con 0x55b5af8b29e0 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.611+0000 7f9af27fc700 1 -- 172.21.15.26:0/3640195003 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b000a1f10 con 0x55b5af8b29e0 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.611+0000 7f9af27fc700 1 -- 172.21.15.26:0/3640195003 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00017520 con 0x55b5af8b29e0 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3640195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 msgr2=0x55b5af8bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b00096080 tx=0x7f9b000994e0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3640195003 shutdown_connections 2023-10-06T21:19:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af8bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3640195003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bd350 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.612+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3640195003 >> 172.21.15.26:0/3640195003 conn(0x55b5af827260 msgr2=0x55b5af8ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.613+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3640195003 shutdown_connections 2023-10-06T21:19:36.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.613+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3640195003 wait complete. 2023-10-06T21:19:36.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.614+0000 7f9b2621cb80 1 -- start start 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.614+0000 7f9b2621cb80 1 Processor -- start 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b20c0 0x55b5af8ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8bd890 0x55b5af8bb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af805e30 con 0x55b5af8b20c0 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af7a2710 con 0x55b5af8bd890 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af7a23b0 con 0x55b5af8b29e0 2023-10-06T21:19:36.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b2621cb80 10 client.0 _ll_register_callbacks cb 0x55b5af5ff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38566/0 (socket says 172.21.15.26:38566) 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3876911702 learned_addr learned my addr 172.21.15.26:0/3876911702 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b20c0 0x55b5af8ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.615+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8bd890 0x55b5af8bb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3876911702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8bd890 msgr2=0x55b5af8bb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8bd890 0x55b5af8bb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b20c0 msgr2=0x55b5af8ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b20c0 0x55b5af8ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3876911702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c1973e0 con 0x55b5af8b29e0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b00099b10 tx=0x7f9b000ca260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1eb02221008b71b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9aeb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b000105f0 con 0x55b5af8b29e0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b000b22a0 con 0x55b5af8b29e0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.616+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5af8a5f10 con 0x55b5af8b29e0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.617+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b000b2460 con 0x55b5af8b29e0 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.617+0000 7f9b2621cb80 10 client.26647 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:36.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.617+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b5af619c60 con 0x55b5af8b29e0 2023-10-06T21:19:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.617+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5af8a5f10 con 0x55b5af8b29e0 2023-10-06T21:19:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b00099070 con 0x55b5af8b29e0 2023-10-06T21:19:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 10 client.26647 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 10 client.26647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 20 client.26647 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af8de030 0x55b5af8e0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 4 client.26647 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.618+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5af7fee10 con 0x55b5af8de030 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.619+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af8de030 0x55b5af8e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.620+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af8de030 0x55b5af8e0450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20208a417027de0a server_cookie=e3bca9775f449d22 in_seq=0 out_seq=0 2023-10-06T21:19:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.620+0000 7f9aeb7fe700 10 client.26647 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:19:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.621+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/3876911702 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b5af7fee10 con 0x55b5af8de030 2023-10-06T21:19:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.621+0000 7f9aeb7fe700 10 client.26647 handle_command_reply: tid=0 2023-10-06T21:19:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.621+0000 7f9b2621cb80 1 client.26647 shutdown 2023-10-06T21:19:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 msgr2=0x55b5af8bac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b00099b10 tx=0x7f9b000ca260 comp rx=0 tx=0).stop 2023-10-06T21:19:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 shutdown_connections 2023-10-06T21:19:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8bd890 0x55b5af8bb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b20c0 0x55b5af8ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af8de030 0x55b5af8e0450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3876911702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af8bac90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9aeb7fe700 0 client.26647 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 >> 172.21.15.26:0/3876911702 conn(0x55b5af827260 msgr2=0x55b5af8adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 shutdown_connections 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.622+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3876911702 wait complete. 2023-10-06T21:19:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.623+0000 7f9b2621cb80 20 client.26647 trim_cache size 0 max 16384 2023-10-06T21:19:36.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.656+0000 7f9b2621cb80 1 Processor -- start 2023-10-06T21:19:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 -- start start 2023-10-06T21:19:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 0x55b5af8c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 0x55b5af8b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af686920 con 0x55b5af8b29e0 2023-10-06T21:19:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af8afd40 con 0x55b5af8b20c0 2023-10-06T21:19:36.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.669+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af8b32e0 con 0x55b5af8bd890 2023-10-06T21:19:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.674+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 0x55b5af8c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.674+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 0x55b5af8c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56734/0 (socket says 172.21.15.26:56734) 2023-10-06T21:19:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.674+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/3005931877 learned_addr learned my addr 172.21.15.26:0/3005931877 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.676+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 0x55b5af8b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.678+0000 7f9b10d9c700 1 -- 172.21.15.26:0/3005931877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 msgr2=0x55b5af8b5130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.678+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.678+0000 7f9b10d9c700 1 -- 172.21.15.26:0/3005931877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 msgr2=0x55b5af8c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.678+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 0x55b5af8c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.678+0000 7f9b10d9c700 1 -- 172.21.15.26:0/3005931877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5af8aa140 con 0x55b5af8bd890 2023-10-06T21:19:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.683+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 0x55b5af8b79c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c101d50 tx=0x7f9b040092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1e68f4fc8315a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.691+0000 7f9af2ffd700 1 -- 172.21.15.26:0/3005931877 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400e040 con 0x55b5af8bd890 2023-10-06T21:19:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.691+0000 7f9af2ffd700 1 -- 172.21.15.26:0/3005931877 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b040094d0 con 0x55b5af8bd890 2023-10-06T21:19:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.691+0000 7f9af2ffd700 1 -- 172.21.15.26:0/3005931877 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0401c8d0 con 0x55b5af8bd890 2023-10-06T21:19:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.696+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3005931877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 msgr2=0x55b5af8b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.696+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 0x55b5af8b79c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c101d50 tx=0x7f9b040092b0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.697+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3005931877 shutdown_connections 2023-10-06T21:19:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.697+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8bd890 0x55b5af8b79c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.697+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af8b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.697+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/3005931877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8b29e0 0x55b5af8c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.697+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3005931877 >> 172.21.15.26:0/3005931877 conn(0x55b5af827260 msgr2=0x55b5af8b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.701+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3005931877 shutdown_connections 2023-10-06T21:19:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.702+0000 7f9b2621cb80 1 -- 172.21.15.26:0/3005931877 wait complete. 2023-10-06T21:19:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.702+0000 7f9b2621cb80 1 -- start start 2023-10-06T21:19:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.703+0000 7f9b2621cb80 1 Processor -- start 2023-10-06T21:19:36.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.703+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af92b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.703+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b2621cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af9298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af929e20 con 0x55b5af8bd890 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af929f90 con 0x55b5af8b20c0 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b2621cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5af92a150 con 0x55b5af8b29e0 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b2621cb80 10 client.0 _ll_register_callbacks cb 0x7f9afc0091a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b0b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38580/0 (socket says 172.21.15.26:38580) 2023-10-06T21:19:36.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.704+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/932417334 learned_addr learned my addr 172.21.15.26:0/932417334 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:36.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b10d9c700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af9298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/932417334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 msgr2=0x55b5af92b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:36.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af92b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 msgr2=0x55b5af9298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:36.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af9298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.708+0000 7f9b0b7fe700 1 -- 172.21.15.26:0/932417334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c10e590 con 0x55b5af8b29e0 2023-10-06T21:19:36.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.713+0000 7f9b0b7fe700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bc20 tx=0x7f9b000ca740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c61dea80956863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:36.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.713+0000 7f9aeb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.719+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0009a0a0 con 0x55b5af8b29e0 2023-10-06T21:19:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.719+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9b00099070 con 0x55b5af8b29e0 2023-10-06T21:19:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.719+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b000a1f10 con 0x55b5af8b29e0 2023-10-06T21:19:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.719+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b00095ec0 con 0x55b5af8b29e0 2023-10-06T21:19:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.720+0000 7f9b2621cb80 10 client.26653 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.720+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b5af657250 con 0x55b5af8b29e0 2023-10-06T21:19:36.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.721+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b000a1f10 con 0x55b5af8b29e0 2023-10-06T21:19:36.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.721+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9b000b90f0 con 0x55b5af8b29e0 2023-10-06T21:19:36.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.726+0000 7f9b2621cb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:36.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.726+0000 7f9b2621cb80 10 client.26653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:19:36.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.726+0000 7f9b2621cb80 20 client.26653 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:36.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.726+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af943760 0x55b5af945b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:36.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.727+0000 7f9b2621cb80 4 client.26653 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:19:36.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.727+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5af7feb30 con 0x55b5af943760 2023-10-06T21:19:36.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.728+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af943760 0x55b5af945b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:36.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.729+0000 7f9b0bfff700 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af943760 0x55b5af945b50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45ee0b8906b7beee server_cookie=ff1f3f743c84cd01 in_seq=0 out_seq=0 2023-10-06T21:19:36.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.730+0000 7f9aeb7fe700 10 client.26653 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:19:36.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.734+0000 7f9aeb7fe700 1 -- 172.21.15.26:0/932417334 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5af7feb30 con 0x55b5af943760 2023-10-06T21:19:36.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.734+0000 7f9aeb7fe700 10 client.26653 handle_command_reply: tid=0 2023-10-06T21:19:36.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.734+0000 7f9b2621cb80 1 client.26653 shutdown 2023-10-06T21:19:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 msgr2=0x55b5af92b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bc20 tx=0x7f9b000ca740 comp rx=0 tx=0).stop 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 shutdown_connections 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5af8bd890 0x55b5af9298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b5af8b20c0 0x55b5af92b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x55b5af943760 0x55b5af945b50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 --2- 172.21.15.26:0/932417334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5af8b29e0 0x55b5af92b620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 >> 172.21.15.26:0/932417334 conn(0x55b5af827260 msgr2=0x55b5af8ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.736+0000 7f9aeb7fe700 0 client.26653 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:19:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.741+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 shutdown_connections 2023-10-06T21:19:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.741+0000 7f9b2621cb80 1 -- 172.21.15.26:0/932417334 wait complete. 2023-10-06T21:19:36.747 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.743+0000 7f9b2621cb80 20 client.26653 trim_cache size 0 max 16384 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.743+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 msgr2=0x7f9af40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.743+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 0x7f9af40a3b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009910 tx=0x7f9afc008040 comp rx=0 tx=0).stop 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.743+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 msgr2=0x7f9b0c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.743+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000b460 tx=0x7f9b0000b820 comp rx=0 tx=0).stop 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 shutdown_connections 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b0c10b410 0x7f9b0c1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9af40a1670 0x7f9af40a3b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b0c101ff0 0x7f9b0c0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 --2- 172.21.15.26:0/3078028246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b0c102910 0x7f9b0c100180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.745+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 >> 172.21.15.26:0/3078028246 conn(0x7f9b0c0fb830 msgr2=0x7f9b0c10d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.746+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 shutdown_connections 2023-10-06T21:19:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:36.746+0000 7f9b127ff700 1 -- 172.21.15.26:0/3078028246 wait complete. 2023-10-06T21:19:36.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:19:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:36 smithi167 ceph-mon[134494]: pgmap v288: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-06T21:19:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:36 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3565313556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:36.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:36 smithi078 ceph-mon[134896]: pgmap v288: 97 pgs: 97 active+clean; 6.0 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-06T21:19:36.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:36 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3565313556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:19:37.131 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 535.0 s, 897.2 tps, lat 109.270 ms stddev 194.180 2023-10-06T21:19:37.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 -- 172.21.15.26:0/511802563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0811a690 msgr2=0x7fcf0811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/511802563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0811a690 0x7fcf0811ca80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcef801a9a0 tx=0x7fcef801acb0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 -- 172.21.15.26:0/511802563 shutdown_connections 2023-10-06T21:19:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/511802563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0811a690 0x7fcf0811ca80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/511802563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0810f640 0x7fcf08107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/511802563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf081083f0 0x7fcf08108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.135+0000 7fcf0cbff700 1 -- 172.21.15.26:0/511802563 >> 172.21.15.26:0/511802563 conn(0x7fcf0806d1a0 msgr2=0x7fcf0806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.136+0000 7fcf0cbff700 1 -- 172.21.15.26:0/511802563 shutdown_connections 2023-10-06T21:19:37.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.136+0000 7fcf0cbff700 1 -- 172.21.15.26:0/511802563 wait complete. 2023-10-06T21:19:37.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.138+0000 7fcf0cbff700 1 Processor -- start 2023-10-06T21:19:37.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.138+0000 7fcf0cbff700 1 -- start start 2023-10-06T21:19:37.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf081083f0 0x7fcf081159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 0x7fcf08115f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a99a0 con 0x7fcf0810f640 2023-10-06T21:19:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a9ae0 con 0x7fcf0811a690 2023-10-06T21:19:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf0cbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081a9ca0 con 0x7fcf081083f0 2023-10-06T21:19:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf06d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 0x7fcf08115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf06d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48714/0 (socket says 172.21.15.26:48714) 2023-10-06T21:19:37.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.139+0000 7fcf06d9d700 1 -- 172.21.15.26:0/563296391 learned_addr learned my addr 172.21.15.26:0/563296391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:37.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 -- 172.21.15.26:0/563296391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf081083f0 msgr2=0x7fcf081159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf081083f0 0x7fcf081159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 -- 172.21.15.26:0/563296391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 msgr2=0x7fcf081a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 -- 172.21.15.26:0/563296391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef801a600 con 0x7fcf0810f640 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.140+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 0x7fcf08115f30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcefc006fd0 tx=0x7fcefc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.141+0000 7fcef77fe700 1 -- 172.21.15.26:0/563296391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc011830 con 0x7fcf0810f640 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.141+0000 7fcef77fe700 1 -- 172.21.15.26:0/563296391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcefc011e50 con 0x7fcf0810f640 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.141+0000 7fcef77fe700 1 -- 172.21.15.26:0/563296391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc010dc0 con 0x7fcf0810f640 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.141+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf081a9f80 con 0x7fcf0810f640 2023-10-06T21:19:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.141+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081aa350 con 0x7fcf0810f640 2023-10-06T21:19:37.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.145+0000 7fcef77fe700 1 -- 172.21.15.26:0/563296391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcefc010380 con 0x7fcf0810f640 2023-10-06T21:19:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.147+0000 7fcf2060bb80 1 Processor -- start 2023-10-06T21:19:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.147+0000 7fcf2060bb80 1 -- start start 2023-10-06T21:19:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.147+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x5607358474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735854c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607357b0460 con 0x5607358479e0 2023-10-06T21:19:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607357a1c20 con 0x5607358470c0 2023-10-06T21:19:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073579f540 con 0x560735852890 2023-10-06T21:19:37.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcef77fe700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 0x7fcef00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56776/0 (socket says 172.21.15.26:56776) 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf0659c700 1 -- 172.21.15.26:0/2252643226 learned_addr learned my addr 172.21.15.26:0/2252643226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x5607358474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.148+0000 7fcef77fe700 1 -- 172.21.15.26:0/563296391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcefc0cc9a0 con 0x7fcf0810f640 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 0x7fcef00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 -- 172.21.15.26:0/2252643226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 msgr2=0x560735854c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735854c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 -- 172.21.15.26:0/2252643226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 msgr2=0x5607358474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x5607358474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 -- 172.21.15.26:0/2252643226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607355c5de0 con 0x5607358479e0 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.149+0000 7fcf0659c700 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009da0 tx=0x7fcf0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b5a9839442a4ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.150+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 0x7fcef00a3cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005850 tx=0x7fcef801acb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.150+0000 7fcef4ff9700 1 -- 172.21.15.26:0/2252643226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000e070 con 0x5607358479e0 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.150+0000 7fcef4ff9700 1 -- 172.21.15.26:0/2252643226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf000098d0 con 0x5607358479e0 2023-10-06T21:19:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.150+0000 7fcef4ff9700 1 -- 172.21.15.26:0/2252643226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00011390 con 0x5607358479e0 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 -- 172.21.15.26:0/2252643226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 msgr2=0x560735852350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009da0 tx=0x7fcf0000b330 comp rx=0 tx=0).stop 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 -- 172.21.15.26:0/2252643226 shutdown_connections 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735854c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x5607358474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/2252643226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735852350 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 -- 172.21.15.26:0/2252643226 >> 172.21.15.26:0/2252643226 conn(0x5607357bc260 msgr2=0x560735843450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 -- 172.21.15.26:0/2252643226 shutdown_connections 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.151+0000 7fcf2060bb80 1 -- 172.21.15.26:0/2252643226 wait complete. 2023-10-06T21:19:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.152+0000 7fcf2060bb80 1 -- start start 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.152+0000 7fcf2060bb80 1 Processor -- start 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358479e0 0x56073584fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x5607358500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073579ae30 con 0x560735852890 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560735737710 con 0x5607358470c0 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607357373b0 con 0x5607358479e0 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf2060bb80 10 client.0 _ll_register_callbacks cb 0x560735594800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48720/0 (socket says 172.21.15.26:48720) 2023-10-06T21:19:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 -- 172.21.15.26:0/34651436 learned_addr learned my addr 172.21.15.26:0/34651436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x5607358500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 -- 172.21.15.26:0/34651436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358479e0 msgr2=0x56073584fb80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358479e0 0x56073584fb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 -- 172.21.15.26:0/34651436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 msgr2=0x5607358500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x5607358500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf0659c700 1 -- 172.21.15.26:0/34651436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00009a50 con 0x5607358470c0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.153+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x5607358500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fcf0659c700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012de0 tx=0x7fcf00012ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4d3c871e505992 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fceedffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000e040 con 0x5607358470c0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560735793e10 con 0x5607358470c0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf0001b620 con 0x5607358470c0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf000127b0 con 0x5607358470c0 2023-10-06T21:19:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fcf2060bb80 10 client.26759 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.154+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5607355aec60 con 0x5607358470c0 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560735793e10 con 0x5607358470c0 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf0001a910 con 0x5607358470c0 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 10 client.26759 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 20 client.26759 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560735862330 0x560735864750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 4 client.26759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560735793b30 con 0x560735862330 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.155+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560735862330 0x560735864750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.156+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560735862330 0x560735864750 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9253818f0ad89f server_cookie=c20a4f4b5d91a434 in_seq=0 out_seq=0 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.156+0000 7fceedffb700 10 client.26759 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fceedffb700 1 -- 172.21.15.26:0/34651436 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560735793b30 con 0x560735862330 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fceedffb700 10 client.26759 handle_command_reply: tid=0 2023-10-06T21:19:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 client.26759 shutdown 2023-10-06T21:19:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 msgr2=0x56073584f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf00012de0 tx=0x7fcf00012ec0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 shutdown_connections 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x5607358500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560735862330 0x560735864750 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x56073584f640 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/34651436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358479e0 0x56073584fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 >> 172.21.15.26:0/34651436 conn(0x5607357bc260 msgr2=0x560735842fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.157+0000 7fceedffb700 0 client.26759 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.158+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 shutdown_connections 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.158+0000 7fcf2060bb80 1 -- 172.21.15.26:0/34651436 wait complete. 2023-10-06T21:19:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.158+0000 7fcf2060bb80 20 client.26759 trim_cache size 0 max 16384 2023-10-06T21:19:37.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 Processor -- start 2023-10-06T21:19:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 -- start start 2023-10-06T21:19:37.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358470c0 0x56073584a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x56073584c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073561b920 con 0x560735852890 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560735850c20 con 0x5607358479e0 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.175+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560735847dc0 con 0x5607358470c0 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48722/0 (socket says 172.21.15.26:48722) 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 -- 172.21.15.26:0/1423465288 learned_addr learned my addr 172.21.15.26:0/1423465288 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:37.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf06d9d700 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x56073584c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 -- 172.21.15.26:0/1423465288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358470c0 msgr2=0x56073584a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358470c0 0x56073584a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 -- 172.21.15.26:0/1423465288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 msgr2=0x56073584c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x56073584c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 -- 172.21.15.26:0/1423465288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073583f140 con 0x5607358479e0 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00bb30 tx=0x7fcefc01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=387480caef95d8e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.176+0000 7fcef57fa700 1 -- 172.21.15.26:0/1423465288 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc099a20 con 0x5607358479e0 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcef57fa700 1 -- 172.21.15.26:0/1423465288 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcefc09db70 con 0x5607358479e0 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcef57fa700 1 -- 172.21.15.26:0/1423465288 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc011bf0 con 0x5607358479e0 2023-10-06T21:19:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 -- 172.21.15.26:0/1423465288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 msgr2=0x5607358593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00bb30 tx=0x7fcefc01c040 comp rx=0 tx=0).stop 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 -- 172.21.15.26:0/1423465288 shutdown_connections 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560735852890 0x56073584c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607358470c0 0x56073584a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/1423465288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358479e0 0x5607358593d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 -- 172.21.15.26:0/1423465288 >> 172.21.15.26:0/1423465288 conn(0x5607357bc260 msgr2=0x56073584a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.177+0000 7fcf2060bb80 1 -- 172.21.15.26:0/1423465288 shutdown_connections 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.178+0000 7fcf2060bb80 1 -- 172.21.15.26:0/1423465288 wait complete. 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.178+0000 7fcf2060bb80 1 -- start start 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.178+0000 7fcf2060bb80 1 Processor -- start 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735859020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735856f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560735857640 con 0x5607358479e0 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607358577b0 con 0x5607358470c0 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560735857970 con 0x560735852890 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf2060bb80 10 client.0 _ll_register_callbacks cb 0x7fcf00009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf05d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735859020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48726/0 (socket says 172.21.15.26:48726) 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 -- 172.21.15.26:0/560551702 learned_addr learned my addr 172.21.15.26:0/560551702 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 -- 172.21.15.26:0/560551702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 msgr2=0x560735856f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735856f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 -- 172.21.15.26:0/560551702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 msgr2=0x560735859020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735859020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.179+0000 7fcf0659c700 1 -- 172.21.15.26:0/560551702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081aa350 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fcf0659c700 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf0002c040 tx=0x7fcf0001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7278d370eb4af8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fceedffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000e040 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56073583af10 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf000127b0 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0001ae60 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fcf2060bb80 10 client.26765 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5607355ec250 con 0x5607358470c0 2023-10-06T21:19:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.180+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56073583af10 con 0x5607358470c0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf00027440 con 0x5607358470c0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 10 client.26765 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 10 client.26765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 20 client.26765 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5607358b6db0 0x5607358b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 4 client.26765 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607355e6f70 con 0x5607358b6db0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5607358b6db0 0x5607358b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fcf05d9b700 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5607358b6db0 0x5607358b91a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ee41c5b0885cca server_cookie=ecd5270a089b2bf5 in_seq=0 out_seq=0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.181+0000 7fceedffb700 10 client.26765 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fceedffb700 1 -- 172.21.15.26:0/560551702 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607355e6f70 con 0x5607358b6db0 2023-10-06T21:19:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fceedffb700 10 client.26765 handle_command_reply: tid=0 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 client.26765 shutdown 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 msgr2=0x560735858ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf0002c040 tx=0x7fcf0001b000 comp rx=0 tx=0).stop 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 shutdown_connections 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560735852890 0x560735856f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5607358b6db0 0x5607358b91a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607358470c0 0x560735858ae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 --2- 172.21.15.26:0/560551702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607358479e0 0x560735859020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 >> 172.21.15.26:0/560551702 conn(0x5607357bc260 msgr2=0x56073584ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fceedffb700 0 client.26765 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 shutdown_connections 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 1 -- 172.21.15.26:0/560551702 wait complete. 2023-10-06T21:19:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.182+0000 7fcf2060bb80 20 client.26765 trim_cache size 0 max 16384 2023-10-06T21:19:37.193 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:19:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 msgr2=0x7fcef00a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 0x7fcef00a3cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005850 tx=0x7fcef801acb0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 msgr2=0x7fcf08115f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 0x7fcf08115f30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcefc006fd0 tx=0x7fcefc00b730 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 shutdown_connections 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0811a690 0x7fcf081a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf081083f0 0x7fcf081159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcef00a1800 0x7fcef00a3cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 --2- 172.21.15.26:0/563296391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf0810f640 0x7fcf08115f30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 >> 172.21.15.26:0/563296391 conn(0x7fcf0806d1a0 msgr2=0x7fcf0811aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 shutdown_connections 2023-10-06T21:19:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:37.183+0000 7fcf0cbff700 1 -- 172.21.15.26:0/563296391 wait complete. 2023-10-06T21:19:38.205 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:19:38.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:19:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 -- 172.21.15.26:0/2993567977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e010f640 msgr2=0x7f61e0107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 --2- 172.21.15.26:0/2993567977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e010f640 0x7f61e0107eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009990 tx=0x7f61dc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 -- 172.21.15.26:0/2993567977 shutdown_connections 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 --2- 172.21.15.26:0/2993567977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e011a690 0x7f61e011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 --2- 172.21.15.26:0/2993567977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e010f640 0x7f61e0107eb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 --2- 172.21.15.26:0/2993567977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.557+0000 7f61e87ff700 1 -- 172.21.15.26:0/2993567977 >> 172.21.15.26:0/2993567977 conn(0x7f61e006d1a0 msgr2=0x7f61e006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.562+0000 7f61e87ff700 1 -- 172.21.15.26:0/2993567977 shutdown_connections 2023-10-06T21:19:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.562+0000 7f61e87ff700 1 -- 172.21.15.26:0/2993567977 wait complete. 2023-10-06T21:19:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.563+0000 7f61e87ff700 1 Processor -- start 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.563+0000 7f61e87ff700 1 -- start start 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 0x7f61e0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 0x7f61e0118450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0118a50 con 0x7f61e01083f0 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0118bc0 con 0x7f61e011a690 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.564+0000 7f61e87ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0118d80 con 0x7f61e010f640 2023-10-06T21:19:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.565+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 0x7f61e0118450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.565+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 0x7f61e0118450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48742/0 (socket says 172.21.15.26:48742) 2023-10-06T21:19:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.565+0000 7f61e6d9c700 1 -- 172.21.15.26:0/1088780333 learned_addr learned my addr 172.21.15.26:0/1088780333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.565+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 0x7f61e0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 -- 172.21.15.26:0/1088780333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 msgr2=0x7f61e0113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 0x7f61e0113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 -- 172.21.15.26:0/1088780333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 msgr2=0x7f61e0118450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 0x7f61e0118450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.566+0000 7f61e659b700 1 -- 172.21.15.26:0/1088780333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0095f0 con 0x7f61e01083f0 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.567+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 0x7f61e0113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.567+0000 7f61e659b700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0113a40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.569+0000 7f61d77fe700 1 -- 172.21.15.26:0/1088780333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc01d070 con 0x7f61e01083f0 2023-10-06T21:19:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.569+0000 7f61d77fe700 1 -- 172.21.15.26:0/1088780333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61dc005580 con 0x7f61e01083f0 2023-10-06T21:19:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.569+0000 7f61d77fe700 1 -- 172.21.15.26:0/1088780333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00fe60 con 0x7f61e01083f0 2023-10-06T21:19:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.569+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e0119000 con 0x7f61e01083f0 2023-10-06T21:19:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.569+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e01bb410 con 0x7f61e01083f0 2023-10-06T21:19:38.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:38 smithi026 ceph-mon[98490]: pgmap v289: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:19:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.576+0000 7f61fc056b80 1 Processor -- start 2023-10-06T21:19:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.576+0000 7f61d77fe700 1 -- 172.21.15.26:0/1088780333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f61dc00f560 con 0x7f61e01083f0 2023-10-06T21:19:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 -- start start 2023-10-06T21:19:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce8e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8ea60 0x55b4bce993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcdf74e0 con 0x55b4bce8ea60 2023-10-06T21:19:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcde8ca0 con 0x55b4bce99910 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.577+0000 7f61fc056b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcde65c0 con 0x55b4bce8e140 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce8e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48750/0 (socket says 172.21.15.26:48750) 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce8e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38640/0 (socket says 172.21.15.26:38640) 2023-10-06T21:19:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 -- 172.21.15.26:0/2412702214 learned_addr learned my addr 172.21.15.26:0/2412702214 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 -- 172.21.15.26:0/2412702214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 msgr2=0x55b4bce8e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce8e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 -- 172.21.15.26:0/2412702214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8ea60 msgr2=0x55b4bce993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8ea60 0x55b4bce993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.578+0000 7f61e5d9a700 1 -- 172.21.15.26:0/2412702214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4bcc0ce60 con 0x55b4bce99910 2023-10-06T21:19:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.580+0000 7f61d77fe700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 0x7f61cc0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.580+0000 7f61d77fe700 1 -- 172.21.15.26:0/1088780333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61dc0ce100 con 0x7f61e01083f0 2023-10-06T21:19:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.580+0000 7f61e6d9c700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 0x7f61cc0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.581+0000 7f61e6d9c700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 0x7f61cc0a3cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007450 tx=0x7f61d8009830 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.582+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009e60 tx=0x7f61d000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d5933ae2167b74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.582+0000 7f61d4ff9700 1 -- 172.21.15.26:0/2412702214 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000ec20 con 0x55b4bce99910 2023-10-06T21:19:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.582+0000 7f61d4ff9700 1 -- 172.21.15.26:0/2412702214 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d000ed80 con 0x55b4bce99910 2023-10-06T21:19:38.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.582+0000 7f61d4ff9700 1 -- 172.21.15.26:0/2412702214 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0012700 con 0x55b4bce99910 2023-10-06T21:19:38.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.583+0000 7f61fc056b80 1 -- 172.21.15.26:0/2412702214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 msgr2=0x55b4bce9bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.583+0000 7f61fc056b80 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009e60 tx=0x7f61d000e4b0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 -- 172.21.15.26:0/2412702214 shutdown_connections 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bce9bd00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce8e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 --2- 172.21.15.26:0/2412702214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8ea60 0x55b4bce993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 -- 172.21.15.26:0/2412702214 >> 172.21.15.26:0/2412702214 conn(0x55b4bce032e0 msgr2=0x55b4bce8a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 -- 172.21.15.26:0/2412702214 shutdown_connections 2023-10-06T21:19:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 -- 172.21.15.26:0/2412702214 wait complete. 2023-10-06T21:19:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.584+0000 7f61fc056b80 1 -- start start 2023-10-06T21:19:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.585+0000 7f61fc056b80 1 Processor -- start 2023-10-06T21:19:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8ea60 0x55b4bce96c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcde1eb0 con 0x55b4bce99910 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcd7e790 con 0x55b4bce8ea60 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcd7e430 con 0x55b4bce8e140 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61fc056b80 10 client.0 _ll_register_callbacks cb 0x55b4bcbdb880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61e659b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61e659b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38650/0 (socket says 172.21.15.26:38650) 2023-10-06T21:19:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61e659b700 1 -- 172.21.15.26:0/838731311 learned_addr learned my addr 172.21.15.26:0/838731311 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.586+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8ea60 0x55b4bce96c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 -- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8ea60 msgr2=0x55b4bce96c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8ea60 0x55b4bce96c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 -- 172.21.15.26:0/838731311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 msgr2=0x55b4bce97160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 -- 172.21.15.26:0/838731311 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d0009ac0 con 0x55b4bce8e140 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61e659b700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a2ed0 tx=0x7f61dc09af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f6f0b5a09026d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:38.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc09b800 con 0x55b4bce8e140 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.587+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4bce81f90 con 0x55b4bce8e140 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.588+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61dc09b100 con 0x55b4bce8e140 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.588+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61dc09f250 con 0x55b4bce8e140 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.588+0000 7f61fc056b80 10 client.26662 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.588+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b4bcbf5ce0 con 0x55b4bce8e140 2023-10-06T21:19:38.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.588+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4bce81f90 con 0x55b4bce8e140 2023-10-06T21:19:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f61dc0a2f70 con 0x55b4bce8e140 2023-10-06T21:19:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 10 client.26662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 20 client.26662 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bcea8f60 0x55b4bceab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 4 client.26662 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.589+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4bcddae90 con 0x55b4bcea8f60 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.590+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bcea8f60 0x55b4bceab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.590+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bcea8f60 0x55b4bceab380 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97550781c16e185a server_cookie=b94ea43ccec97be7 in_seq=0 out_seq=0 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.590+0000 7f61c9ffb700 10 client.26662 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61c9ffb700 1 -- 172.21.15.26:0/838731311 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b4bcddae90 con 0x55b4bcea8f60 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61c9ffb700 10 client.26662 handle_command_reply: tid=0 2023-10-06T21:19:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 client.26662 shutdown 2023-10-06T21:19:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 msgr2=0x55b4bce966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a2ed0 tx=0x7f61dc09af80 comp rx=0 tx=0).stop 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 shutdown_connections 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8e140 0x55b4bce966e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bcea8f60 0x55b4bceab380 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 --2- 172.21.15.26:0/838731311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8ea60 0x55b4bce96c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 >> 172.21.15.26:0/838731311 conn(0x55b4bce032e0 msgr2=0x55b4bce8a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.591+0000 7f61c9ffb700 0 client.26662 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.592+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 shutdown_connections 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.592+0000 7f61fc056b80 1 -- 172.21.15.26:0/838731311 wait complete. 2023-10-06T21:19:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.592+0000 7f61fc056b80 20 client.26662 trim_cache size 0 max 16384 2023-10-06T21:19:38.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 Processor -- start 2023-10-06T21:19:38.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 -- start start 2023-10-06T21:19:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8e140 0x55b4bce98e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bce95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcc629a0 con 0x55b4bce8e140 2023-10-06T21:19:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bcea09a0 con 0x55b4bce99910 2023-10-06T21:19:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.624+0000 7f61fc056b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bce92550 con 0x55b4bce8ea60 2023-10-06T21:19:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.625+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.625+0000 7f61e6d9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48770/0 (socket says 172.21.15.26:48770) 2023-10-06T21:19:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.625+0000 7f61e6d9c700 1 -- 172.21.15.26:0/4138554025 learned_addr learned my addr 172.21.15.26:0/4138554025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e659b700 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8e140 0x55b4bce98e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bce95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e6d9c700 1 -- 172.21.15.26:0/4138554025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 msgr2=0x55b4bce95970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e6d9c700 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bce95970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e6d9c700 1 -- 172.21.15.26:0/4138554025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8e140 msgr2=0x55b4bce98e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e6d9c700 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8e140 0x55b4bce98e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.626+0000 7f61e6d9c700 1 -- 172.21.15.26:0/4138554025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4bce88f10 con 0x55b4bce99910 2023-10-06T21:19:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.627+0000 7f61e6d9c700 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007f30 tx=0x7f61d800b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a063797e43ef4a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.627+0000 7f61d57fa700 1 -- 172.21.15.26:0/4138554025 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8017df0 con 0x55b4bce99910 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.627+0000 7f61d57fa700 1 -- 172.21.15.26:0/4138554025 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d80228c0 con 0x55b4bce99910 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.627+0000 7f61d57fa700 1 -- 172.21.15.26:0/4138554025 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d80157d0 con 0x55b4bce99910 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 -- 172.21.15.26:0/4138554025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 msgr2=0x55b4bcea0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007f30 tx=0x7f61d800b050 comp rx=0 tx=0).stop 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 -- 172.21.15.26:0/4138554025 shutdown_connections 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce99910 0x55b4bcea0460 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce8e140 0x55b4bce98e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 --2- 172.21.15.26:0/4138554025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bce95970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 -- 172.21.15.26:0/4138554025 >> 172.21.15.26:0/4138554025 conn(0x55b4bce032e0 msgr2=0x55b4bce896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 -- 172.21.15.26:0/4138554025 shutdown_connections 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.628+0000 7f61fc056b80 1 -- 172.21.15.26:0/4138554025 wait complete. 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.629+0000 7f61fc056b80 1 -- start start 2023-10-06T21:19:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.629+0000 7f61fc056b80 1 Processor -- start 2023-10-06T21:19:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8e140 0x55b4bcef7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bce9fa60 con 0x55b4bce99910 2023-10-06T21:19:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bce9fbd0 con 0x55b4bce8e140 2023-10-06T21:19:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4bce9fd90 con 0x55b4bce8ea60 2023-10-06T21:19:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61fc056b80 10 client.0 _ll_register_callbacks cb 0x55b4bcbdb880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61e5d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61e5d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38670/0 (socket says 172.21.15.26:38670) 2023-10-06T21:19:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.630+0000 7f61e5d9a700 1 -- 172.21.15.26:0/461409700 learned_addr learned my addr 172.21.15.26:0/461409700 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.631+0000 7f61e5d9a700 1 -- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8e140 msgr2=0x55b4bcef7400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.631+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8e140 0x55b4bcef7400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.631+0000 7f61e5d9a700 1 -- 172.21.15.26:0/461409700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 msgr2=0x55b4bce9f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.631+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.631+0000 7f61e5d9a700 1 -- 172.21.15.26:0/461409700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d8011040 con 0x55b4bce8ea60 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.632+0000 7f61e5d9a700 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009db0 tx=0x7f61d0008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b279ec69fb62538 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.632+0000 7f61c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.632+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0025ce0 con 0x55b4bce8ea60 2023-10-06T21:19:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.632+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61d0022040 con 0x55b4bce8ea60 2023-10-06T21:19:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.632+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61d000ed80 con 0x55b4bce8ea60 2023-10-06T21:19:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.634+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61d0025590 con 0x55b4bce8ea60 2023-10-06T21:19:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 10 client.26668 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b4bcc332d0 con 0x55b4bce8ea60 2023-10-06T21:19:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d000ed80 con 0x55b4bce8ea60 2023-10-06T21:19:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f61d002e020 con 0x55b4bce8ea60 2023-10-06T21:19:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 10 client.26668 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 20 client.26668 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bceff240 0x55b4bcf01660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 4 client.26668 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:19:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.635+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b4bcddabb0 con 0x55b4bceff240 2023-10-06T21:19:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.636+0000 7f61e659b700 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bceff240 0x55b4bcf01660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.637+0000 7f61e659b700 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bceff240 0x55b4bcf01660 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f7761c32fac8f3 server_cookie=2884a10994bf161b in_seq=0 out_seq=0 2023-10-06T21:19:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.637+0000 7f61c9ffb700 10 client.26668 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.638+0000 7f61c9ffb700 1 -- 172.21.15.26:0/461409700 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b4bcddabb0 con 0x55b4bceff240 2023-10-06T21:19:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.638+0000 7f61c9ffb700 10 client.26668 handle_command_reply: tid=0 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.638+0000 7f61fc056b80 1 client.26668 shutdown 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.638+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 msgr2=0x55b4bcef7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.638+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009db0 tx=0x7f61d0008040 comp rx=0 tx=0).stop 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 shutdown_connections 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4bce99910 0x55b4bce9f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b4bceff240 0x55b4bcf01660 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4bce8e140 0x55b4bcef7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 --2- 172.21.15.26:0/461409700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bce8ea60 0x55b4bcef7940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 >> 172.21.15.26:0/461409700 conn(0x55b4bce032e0 msgr2=0x55b4bce98d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61c9ffb700 0 client.26668 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 shutdown_connections 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 1 -- 172.21.15.26:0/461409700 wait complete. 2023-10-06T21:19:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.639+0000 7f61fc056b80 20 client.26668 trim_cache size 0 max 16384 2023-10-06T21:19:38.645 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:19:38.645 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:19:38.645 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "9abf9fd8-06bd-4664-856f-e32ed7bdd587", 2023-10-06T21:19:38.645 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:19:38.645 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 msgr2=0x7f61cc0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 0x7f61cc0a3cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f61d8007450 tx=0x7f61d8009830 comp rx=0 tx=0).stop 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 msgr2=0x7f61e0113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0113a40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc003b40 comp rx=0 tx=0).stop 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 shutdown_connections 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e011a690 0x7f61e0118450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61cc0a1800 0x7f61cc0a3cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61e01083f0 0x7f61e0113a40 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc003b40 comp rx=0 tx=0).stop 2023-10-06T21:19:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 --2- 172.21.15.26:0/1088780333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61e010f640 0x7f61e0113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 >> 172.21.15.26:0/1088780333 conn(0x7f61e006d1a0 msgr2=0x7f61e011aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 shutdown_connections 2023-10-06T21:19:38.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:38.640+0000 7f61e87ff700 1 -- 172.21.15.26:0/1088780333 wait complete. 2023-10-06T21:19:38.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:19:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:38 smithi167 ceph-mon[134494]: pgmap v289: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:19:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:38 smithi078 ceph-mon[134896]: pgmap v289: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 1.83k op/s 2023-10-06T21:19:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 -- 172.21.15.26:0/2392269741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 msgr2=0x7f19a8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 --2- 172.21.15.26:0/2392269741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a8108870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19980098f0 tx=0x7f1998009c00 comp rx=0 tx=0).stop 2023-10-06T21:19:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 -- 172.21.15.26:0/2392269741 shutdown_connections 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 --2- 172.21.15.26:0/2392269741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a811ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 --2- 172.21.15.26:0/2392269741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a810f640 0x7f19a8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 --2- 172.21.15.26:0/2392269741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a8108870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.076+0000 7f19af5ff700 1 -- 172.21.15.26:0/2392269741 >> 172.21.15.26:0/2392269741 conn(0x7f19a806d1a0 msgr2=0x7f19a806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.077+0000 7f19af5ff700 1 -- 172.21.15.26:0/2392269741 shutdown_connections 2023-10-06T21:19:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.077+0000 7f19af5ff700 1 -- 172.21.15.26:0/2392269741 wait complete. 2023-10-06T21:19:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.079+0000 7f19af5ff700 1 Processor -- start 2023-10-06T21:19:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.079+0000 7f19af5ff700 1 -- start start 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a810f640 0x7f19a81a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a81aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a81aae80 con 0x7f19a81083f0 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a81aaff0 con 0x7f19a811a690 2023-10-06T21:19:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.080+0000 7f19af5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a81ab1b0 con 0x7f19a810f640 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19ad39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a81aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19ad39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56906/0 (socket says 172.21.15.26:56906) 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19ad39b700 1 -- 172.21.15.26:0/141276543 learned_addr learned my addr 172.21.15.26:0/141276543 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 -- 172.21.15.26:0/141276543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a810f640 msgr2=0x7f19a81a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a810f640 0x7f19a81a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 -- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 msgr2=0x7f19a81a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 -- 172.21.15.26:0/141276543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009520 con 0x7f19a811a690 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19ad39b700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.081+0000 7f19adb9c700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a81aa840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ef90 tx=0x7f19a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.082+0000 7f199e7fc700 1 -- 172.21.15.26:0/141276543 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0018070 con 0x7f19a811a690 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.082+0000 7f199e7fc700 1 -- 172.21.15.26:0/141276543 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a000f040 con 0x7f19a811a690 2023-10-06T21:19:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.082+0000 7f199e7fc700 1 -- 172.21.15.26:0/141276543 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0007800 con 0x7f19a811a690 2023-10-06T21:19:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.082+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a81ab430 con 0x7f19a811a690 2023-10-06T21:19:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.082+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81b2cd0 con 0x7f19a811a690 2023-10-06T21:19:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.086+0000 7f199e7fc700 1 -- 172.21.15.26:0/141276543 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f19a00079e0 con 0x7f19a811a690 2023-10-06T21:19:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.087+0000 7f19c2f7bb80 1 Processor -- start 2023-10-06T21:19:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.088+0000 7f19c2f7bb80 1 -- start start 2023-10-06T21:19:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.088+0000 7f199e7fc700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 0x7f19940a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.088+0000 7f199e7fc700 1 -- 172.21.15.26:0/141276543 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f19a00cceb0 con 0x7f19a811a690 2023-10-06T21:19:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.088+0000 7f19acb9a700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 0x7f19940a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd95584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95589e0 0x560bd9563350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd94c1460 con 0x560bd9563890 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd94b2c20 con 0x560bd95589e0 2023-10-06T21:19:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.089+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd94b0540 con 0x560bd95580c0 2023-10-06T21:19:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.100+0000 7f19adb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.100+0000 7f19adb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56910/0 (socket says 172.21.15.26:56910) 2023-10-06T21:19:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.100+0000 7f19adb9c700 1 -- 172.21.15.26:0/3865598231 learned_addr learned my addr 172.21.15.26:0/3865598231 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.102+0000 7f19ad39b700 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd95584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.110+0000 7f19adb9c700 1 -- 172.21.15.26:0/3865598231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 msgr2=0x560bd95584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.110+0000 7f19adb9c700 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd95584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.110+0000 7f19adb9c700 1 -- 172.21.15.26:0/3865598231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95589e0 msgr2=0x560bd9563350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.110+0000 7f19adb9c700 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95589e0 0x560bd9563350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.110+0000 7f19adb9c700 1 -- 172.21.15.26:0/3865598231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bd92d6de0 con 0x560bd9563890 2023-10-06T21:19:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.111+0000 7f19adb9c700 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009bf0 tx=0x7f19a0099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719dc40692ebba2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.146+0000 7f19937fe700 1 -- 172.21.15.26:0/3865598231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0099de0 con 0x560bd9563890 2023-10-06T21:19:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.146+0000 7f19937fe700 1 -- 172.21.15.26:0/3865598231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a009df30 con 0x560bd9563890 2023-10-06T21:19:39.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.146+0000 7f19937fe700 1 -- 172.21.15.26:0/3865598231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a009a040 con 0x560bd9563890 2023-10-06T21:19:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.149+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3865598231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 msgr2=0x560bd9565c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.149+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009bf0 tx=0x7f19a0099a50 comp rx=0 tx=0).stop 2023-10-06T21:19:39.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.149+0000 7f19acb9a700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 0x7f19940a3af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f199800c010 tx=0x7f19980058e0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.154+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3865598231 shutdown_connections 2023-10-06T21:19:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.154+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd9563890 0x560bd9565c80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.154+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd95584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.154+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3865598231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95589e0 0x560bd9563350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.154+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3865598231 >> 172.21.15.26:0/3865598231 conn(0x560bd94cd260 msgr2=0x560bd9554450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.155+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3865598231 shutdown_connections 2023-10-06T21:19:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.155+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3865598231 wait complete. 2023-10-06T21:19:39.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.156+0000 7f19c2f7bb80 1 -- start start 2023-10-06T21:19:39.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.156+0000 7f19c2f7bb80 1 Processor -- start 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95580c0 0x560bd9560710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd9563890 0x560bd9561190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd94abe30 con 0x560bd95589e0 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9448710 con 0x560bd95580c0 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd94483b0 con 0x560bd9563890 2023-10-06T21:19:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.160+0000 7f19c2f7bb80 10 client.0 _ll_register_callbacks cb 0x560bd92a5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:39.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.162+0000 7f19adb9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd9563890 0x560bd9561190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.162+0000 7f19acb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.162+0000 7f19acb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56918/0 (socket says 172.21.15.26:56918) 2023-10-06T21:19:39.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.162+0000 7f19acb9a700 1 -- 172.21.15.26:0/3268616630 learned_addr learned my addr 172.21.15.26:0/3268616630 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.163+0000 7f19acb9a700 1 -- 172.21.15.26:0/3268616630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd9563890 msgr2=0x560bd9561190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.163+0000 7f19acb9a700 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd9563890 0x560bd9561190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.163+0000 7f19acb9a700 1 -- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95580c0 msgr2=0x560bd9560710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.163+0000 7f19acb9a700 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95580c0 0x560bd9560710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.163+0000 7f19acb9a700 1 -- 172.21.15.26:0/3268616630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81b2cd0 con 0x560bd95589e0 2023-10-06T21:19:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.164+0000 7f19acb9a700 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19980052f0 tx=0x7f1998000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdeb7788e47180b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.165+0000 7f1990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:19:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.165+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199800fea0 con 0x560bd95589e0 2023-10-06T21:19:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.165+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f199800b970 con 0x560bd95589e0 2023-10-06T21:19:39.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.166+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bd954bf10 con 0x560bd95589e0 2023-10-06T21:19:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.166+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199800f910 con 0x560bd95589e0 2023-10-06T21:19:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.166+0000 7f19c2f7bb80 10 client.16485 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.166+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560bd92bfc60 con 0x560bd95589e0 2023-10-06T21:19:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1998033030 con 0x560bd95589e0 2023-10-06T21:19:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f199802c5f0 con 0x560bd95589e0 2023-10-06T21:19:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 10 client.16485 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:39.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 20 client.16485 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95844e0 0x560bd9586900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 4 client.16485 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.167+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bd94a4e10 con 0x560bd95844e0 2023-10-06T21:19:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.168+0000 7f19ad39b700 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95844e0 0x560bd9586900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.169+0000 7f19ad39b700 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95844e0 0x560bd9586900 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f9079886e08d08 server_cookie=d510fb384779f2e0 in_seq=0 out_seq=0 2023-10-06T21:19:39.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.170+0000 7f1990ff9700 10 client.16485 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.174+0000 7f1990ff9700 1 -- 172.21.15.26:0/3268616630 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560bd94a4e10 con 0x560bd95844e0 2023-10-06T21:19:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.174+0000 7f1990ff9700 10 client.16485 handle_command_reply: tid=0 2023-10-06T21:19:39.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.174+0000 7f19c2f7bb80 1 client.16485 shutdown 2023-10-06T21:19:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 msgr2=0x560bd9560c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19980052f0 tx=0x7f1998000bc0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 shutdown_connections 2023-10-06T21:19:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd9563890 0x560bd9561190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95844e0 0x560bd9586900 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd95580c0 0x560bd9560710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/3268616630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd9560c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f1990ff9700 0 client.16485 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:39.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.177+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 >> 172.21.15.26:0/3268616630 conn(0x560bd94cd260 msgr2=0x560bd9553f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.178+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 shutdown_connections 2023-10-06T21:19:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.178+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/3268616630 wait complete. 2023-10-06T21:19:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.178+0000 7f19c2f7bb80 20 client.16485 trim_cache size 0 max 16384 2023-10-06T21:19:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 Processor -- start 2023-10-06T21:19:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 -- start start 2023-10-06T21:19:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd955b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd955d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd932c920 con 0x560bd95589e0 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9581b20 con 0x560bd9563890 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.207+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9550a70 con 0x560bd95580c0 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56934/0 (socket says 172.21.15.26:56934) 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19adb9c700 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd955d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 -- 172.21.15.26:0/1852068465 learned_addr learned my addr 172.21.15.26:0/1852068465 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 -- 172.21.15.26:0/1852068465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 msgr2=0x560bd955b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd955b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 -- 172.21.15.26:0/1852068465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 msgr2=0x560bd955d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd955d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 -- 172.21.15.26:0/1852068465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bd92fd250 con 0x560bd95589e0 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.208+0000 7f19acb9a700 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f199802ca00 tx=0x7f19980057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c91a76e2e8f07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.209+0000 7f1993fff700 1 -- 172.21.15.26:0/1852068465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998005950 con 0x560bd95589e0 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.209+0000 7f1993fff700 1 -- 172.21.15.26:0/1852068465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1998033040 con 0x560bd95589e0 2023-10-06T21:19:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.209+0000 7f1993fff700 1 -- 172.21.15.26:0/1852068465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998003900 con 0x560bd95589e0 2023-10-06T21:19:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1852068465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 msgr2=0x560bd956a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f199802ca00 tx=0x7f19980057e0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1852068465 shutdown_connections 2023-10-06T21:19:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd955d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd955b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1852068465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a370 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1852068465 >> 172.21.15.26:0/1852068465 conn(0x560bd94cd260 msgr2=0x560bd955ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1852068465 shutdown_connections 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.210+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1852068465 wait complete. 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.211+0000 7f19c2f7bb80 1 -- start start 2023-10-06T21:19:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.211+0000 7f19c2f7bb80 1 Processor -- start 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd9568360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9568940 con 0x560bd95589e0 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9568ab0 con 0x560bd9563890 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bd9568c70 con 0x560bd95580c0 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.212+0000 7f19c2f7bb80 10 client.0 _ll_register_callbacks cb 0x7f199800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.213+0000 7f19ad39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.213+0000 7f19ad39b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38730/0 (socket says 172.21.15.26:38730) 2023-10-06T21:19:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.213+0000 7f19ad39b700 1 -- 172.21.15.26:0/1385365559 learned_addr learned my addr 172.21.15.26:0/1385365559 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 -- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 msgr2=0x560bd9568360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd9568360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 -- 172.21.15.26:0/1385365559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 msgr2=0x560bd956a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 -- 172.21.15.26:0/1385365559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bd92bfc60 con 0x560bd95580c0 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f19ad39b700 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x560bd9587250 tx=0x7f19a4004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb4ba0b5d7404ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.214+0000 7f1990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4017040 con 0x560bd95580c0 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f19a401b540 con 0x560bd95580c0 2023-10-06T21:19:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a009df30 con 0x560bd95580c0 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a401b730 con 0x560bd95580c0 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f19c2f7bb80 10 client.26677 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.215+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560bd92d52b0 con 0x560bd95580c0 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a009df30 con 0x560bd95580c0 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f19a40049f0 con 0x560bd95580c0 2023-10-06T21:19:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 10 client.26677 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 20 client.26677 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95d95d0 0x560bd95db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 4 client.26677 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560bd94a4b30 con 0x560bd95d95d0 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.216+0000 7f19acb9a700 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95d95d0 0x560bd95db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.217+0000 7f19acb9a700 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95d95d0 0x560bd95db9f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3064182535729fd server_cookie=ce274389aeec4632 in_seq=0 out_seq=0 2023-10-06T21:19:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.217+0000 7f1990ff9700 10 client.26677 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f1990ff9700 1 -- 172.21.15.26:0/1385365559 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560bd94a4b30 con 0x560bd95d95d0 2023-10-06T21:19:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f1990ff9700 10 client.26677 handle_command_reply: tid=0 2023-10-06T21:19:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 client.26677 shutdown 2023-10-06T21:19:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 msgr2=0x560bd9569b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x560bd9587250 tx=0x7f19a4004320 comp rx=0 tx=0).stop 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 shutdown_connections 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560bd95d95d0 0x560bd95db9f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560bd9563890 0x560bd9568360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd95580c0 0x560bd9569b10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 --2- 172.21.15.26:0/1385365559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560bd95589e0 0x560bd956a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 >> 172.21.15.26:0/1385365559 conn(0x560bd94cd260 msgr2=0x560bd9553660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.221+0000 7f1990ff9700 0 client.26677 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.222+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 shutdown_connections 2023-10-06T21:19:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.222+0000 7f19c2f7bb80 1 -- 172.21.15.26:0/1385365559 wait complete. 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.222+0000 7f19c2f7bb80 20 client.26677 trim_cache size 0 max 16384 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "9abf9fd8-06bd-4664-856f-e32ed7bdd587": { 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "9abf9fd8-06bd-4664-856f-e32ed7bdd587", 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:19:39.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.223+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 msgr2=0x7f19940a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.223+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 0x7f19940a3af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f199800c010 tx=0x7f19980058e0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.223+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 msgr2=0x7f19a81aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.223+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a81aa840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ef90 tx=0x7f19a000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 shutdown_connections 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19a811a690 0x7f19a81aa840 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f19940a1630 0x7f19940a3af0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19a81083f0 0x7f19a81a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 --2- 172.21.15.26:0/141276543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a810f640 0x7f19a81a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.224+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 >> 172.21.15.26:0/141276543 conn(0x7f19a806d1a0 msgr2=0x7f19a8118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.228+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 shutdown_connections 2023-10-06T21:19:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:19:39.228+0000 7f19af5ff700 1 -- 172.21.15.26:0/141276543 wait complete. 2023-10-06T21:19:39.258 INFO:tasks.cephfs.filesystem:scrub status for tag:9abf9fd8-06bd-4664-856f-e32ed7bdd587 - {'path': '/', 'tag': '9abf9fd8-06bd-4664-856f-e32ed7bdd587', 'options': 'recursive,force'} 2023-10-06T21:19:39.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:39 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:19:39.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:39 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:19:39.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:39 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:39.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:39 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:19:39.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:39 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:39.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:39 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:19:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:39 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:19:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:39 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:39 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:19:39.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:39 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:39.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:39 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:19:39.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:39 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:19:39.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:39 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:39.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:39 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:19:39.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:39 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: pgmap v290: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-10-06T21:19:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:40 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: pgmap v290: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-10-06T21:19:40.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:40 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: pgmap v290: 97 pgs: 97 active+clean; 6.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2023-10-06T21:19:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:40 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:19:41.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:41 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:41.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:41 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:41.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:41 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:19:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 540.0 s, 1026.1 tps, lat 99.173 ms stddev 115.957 2023-10-06T21:19:42.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:42 smithi167 ceph-mon[134494]: pgmap v291: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-10-06T21:19:42.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:42 smithi026 ceph-mon[98490]: pgmap v291: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-10-06T21:19:42.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:42 smithi078 ceph-mon[134896]: pgmap v291: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-10-06T21:19:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 -- 172.21.15.26:0/388621006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 msgr2=0x7f7d28100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/388621006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d28100230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009990 tx=0x7f7d20009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 -- 172.21.15.26:0/388621006 shutdown_connections 2023-10-06T21:19:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/388621006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d28105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/388621006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d28100230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/388621006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d28104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.645+0000 7f7d2dbff700 1 -- 172.21.15.26:0/388621006 >> 172.21.15.26:0/388621006 conn(0x7f7d280fb860 msgr2=0x7f7d280fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.646+0000 7f7d2dbff700 1 -- 172.21.15.26:0/388621006 shutdown_connections 2023-10-06T21:19:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.646+0000 7f7d2dbff700 1 -- 172.21.15.26:0/388621006 wait complete. 2023-10-06T21:19:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.647+0000 7f7d2dbff700 1 Processor -- start 2023-10-06T21:19:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.647+0000 7f7d2dbff700 1 -- start start 2023-10-06T21:19:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d281a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d2819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819da90 con 0x7f7d28100770 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819dc00 con 0x7f7d28105250 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d2dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819ddc0 con 0x7f7d280ffe50 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.650+0000 7f7d277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.651+0000 7f7d277fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60306/0 (socket says 172.21.15.26:60306) 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.651+0000 7f7d277fe700 1 -- 172.21.15.26:0/2016867896 learned_addr learned my addr 172.21.15.26:0/2016867896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.651+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d2819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.651+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d281a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 -- 172.21.15.26:0/2016867896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 msgr2=0x7f7d281a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 -- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 msgr2=0x7f7d281a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d281a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 -- 172.21.15.26:0/2016867896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200095f0 con 0x7f7d28105250 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d277fe700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d281a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d2819d420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d720 tx=0x7f7d1000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d24ff9700 1 -- 172.21.15.26:0/2016867896 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10004220 con 0x7f7d28105250 2023-10-06T21:19:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d24ff9700 1 -- 172.21.15.26:0/2016867896 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d10004380 con 0x7f7d28105250 2023-10-06T21:19:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.652+0000 7f7d24ff9700 1 -- 172.21.15.26:0/2016867896 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10003b80 con 0x7f7d28105250 2023-10-06T21:19:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.654+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2819e0a0 con 0x7f7d28105250 2023-10-06T21:19:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.654+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281aa5b0 con 0x7f7d28105250 2023-10-06T21:19:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.660+0000 7f7d24ff9700 1 -- 172.21.15.26:0/2016867896 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7d10003ce0 con 0x7f7d28105250 2023-10-06T21:19:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.661+0000 7f7d41585b80 1 Processor -- start 2023-10-06T21:19:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.663+0000 7f7d24ff9700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 0x7f7d140a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.664+0000 7f7d24ff9700 1 -- 172.21.15.26:0/2016867896 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d100cc3b0 con 0x7f7d28105250 2023-10-06T21:19:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 -- start start 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 0x7f7d140a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e8c00 0x560c7f8e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e9520 0x560c7f8f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f852400 con 0x560c7f8e9520 2023-10-06T21:19:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f843bc0 con 0x560c7f8f43d0 2023-10-06T21:19:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.665+0000 7f7d41585b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f8414e0 con 0x560c7f8e8c00 2023-10-06T21:19:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49524/0 (socket says 172.21.15.26:49524) 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 -- 172.21.15.26:0/2992513633 learned_addr learned my addr 172.21.15.26:0/2992513633 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 -- 172.21.15.26:0/2992513633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e8c00 msgr2=0x560c7f8e8fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e8c00 0x560c7f8e8fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 -- 172.21.15.26:0/2992513633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e9520 msgr2=0x560c7f8f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e9520 0x560c7f8f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.666+0000 7f7d27fff700 1 -- 172.21.15.26:0/2992513633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7f667d80 con 0x560c7f8f43d0 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.667+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d10095d90 tx=0x7f7d10098e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4600b6dc9b4239 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.667+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 0x7f7d140a3cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d280fef80 tx=0x7f7d18007a80 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.668+0000 7f7d1dffb700 1 -- 172.21.15.26:0/2992513633 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10099a20 con 0x560c7f8f43d0 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.668+0000 7f7d1dffb700 1 -- 172.21.15.26:0/2992513633 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d10099320 con 0x560c7f8f43d0 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.668+0000 7f7d1dffb700 1 -- 172.21.15.26:0/2992513633 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10099580 con 0x560c7f8f43d0 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 -- 172.21.15.26:0/2992513633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 msgr2=0x560c7f8f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d10095d90 tx=0x7f7d10098e60 comp rx=0 tx=0).stop 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 -- 172.21.15.26:0/2992513633 shutdown_connections 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8f67c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e8c00 0x560c7f8e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2992513633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e9520 0x560c7f8f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 -- 172.21.15.26:0/2992513633 >> 172.21.15.26:0/2992513633 conn(0x560c7f85e200 msgr2=0x560c7f8e5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 -- 172.21.15.26:0/2992513633 shutdown_connections 2023-10-06T21:19:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.669+0000 7f7d41585b80 1 -- 172.21.15.26:0/2992513633 wait complete. 2023-10-06T21:19:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.670+0000 7f7d41585b80 1 -- start start 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.671+0000 7f7d41585b80 1 Processor -- start 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8e8c00 0x560c7f8f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 0x560c7f8f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f83cdd0 con 0x560c7f8f43d0 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f7d96b0 con 0x560c7f8e8c00 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f7d9350 con 0x560c7f8e9520 2023-10-06T21:19:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d41585b80 10 client.0 _ll_register_callbacks cb 0x560c7f6367a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60334/0 (socket says 172.21.15.26:60334) 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d26ffd700 1 -- 172.21.15.26:0/2700503438 learned_addr learned my addr 172.21.15.26:0/2700503438 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.672+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 0x560c7f8f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 -- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8e8c00 msgr2=0x560c7f8f1160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8e8c00 0x560c7f8f1160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 -- 172.21.15.26:0/2700503438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 msgr2=0x560c7f8f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 0x560c7f8f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 -- 172.21.15.26:0/2700503438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281aa5b0 con 0x560c7f8e9520 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d27fff700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 0x560c7f8f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:19:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d18017fd0 tx=0x7f7d18004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce03a7bf6bfb14c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.673+0000 7f7d06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1801f070 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d1801a0d0 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c7f8dcd50 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1801a290 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d41585b80 10 client.26686 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560c7f650c00 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.674+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d18030020 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7d18018530 con 0x560c7f8e9520 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 10 client.26686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 20 client.26686 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f914f30 0x560c7f917350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 4 client.26686 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c7f835db0 con 0x560c7f914f30 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.675+0000 7f7d277fe700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f914f30 0x560c7f917350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.676+0000 7f7d277fe700 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f914f30 0x560c7f917350 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2cc06a9f666a5ea server_cookie=77569a535003194b in_seq=0 out_seq=0 2023-10-06T21:19:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.676+0000 7f7d06ffd700 10 client.26686 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.676+0000 7f7d06ffd700 1 -- 172.21.15.26:0/2700503438 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c7f835db0 con 0x560c7f914f30 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.676+0000 7f7d06ffd700 10 client.26686 handle_command_reply: tid=0 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.676+0000 7f7d41585b80 1 client.26686 shutdown 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 msgr2=0x560c7f8f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7d18017fd0 tx=0x7f7d18004b40 comp rx=0 tx=0).stop 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 shutdown_connections 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8f43d0 0x560c7f8f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f914f30 0x560c7f917350 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8e8c00 0x560c7f8f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 --2- 172.21.15.26:0/2700503438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f16a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 >> 172.21.15.26:0/2700503438 conn(0x560c7f85e200 msgr2=0x560c7f8e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d06ffd700 0 client.26686 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 shutdown_connections 2023-10-06T21:19:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.677+0000 7f7d41585b80 1 -- 172.21.15.26:0/2700503438 wait complete. 2023-10-06T21:19:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.678+0000 7f7d41585b80 20 client.26686 trim_cache size 0 max 16384 2023-10-06T21:19:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.705+0000 7f7d41585b80 1 Processor -- start 2023-10-06T21:19:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 -- start start 2023-10-06T21:19:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f6bd8c0 con 0x560c7f8e8c00 2023-10-06T21:19:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f8ef7a0 con 0x560c7f8f43d0 2023-10-06T21:19:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.706+0000 7f7d41585b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f8e4920 con 0x560c7f8e9520 2023-10-06T21:19:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49544/0 (socket says 172.21.15.26:49544) 2023-10-06T21:19:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60340/0 (socket says 172.21.15.26:60340) 2023-10-06T21:19:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d27fff700 1 -- 172.21.15.26:0/197501869 learned_addr learned my addr 172.21.15.26:0/197501869 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d277fe700 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 -- 172.21.15.26:0/197501869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 msgr2=0x560c7f8ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 -- 172.21.15.26:0/197501869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 msgr2=0x560c7f8ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.707+0000 7f7d26ffd700 1 -- 172.21.15.26:0/197501869 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7f68e1f0 con 0x560c7f8e9520 2023-10-06T21:19:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.708+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x560c7f8dc980 tx=0x7f7d18004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836ead75a0c396d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.709+0000 7f7d1e7fc700 1 -- 172.21.15.26:0/197501869 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1801f040 con 0x560c7f8e9520 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.709+0000 7f7d1e7fc700 1 -- 172.21.15.26:0/197501869 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d18030030 con 0x560c7f8e9520 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.709+0000 7f7d1e7fc700 1 -- 172.21.15.26:0/197501869 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1802c680 con 0x560c7f8e9520 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- 172.21.15.26:0/197501869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 msgr2=0x560c7f8f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x560c7f8dc980 tx=0x7f7d18004ad0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- 172.21.15.26:0/197501869 shutdown_connections 2023-10-06T21:19:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f8ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 --2- 172.21.15.26:0/197501869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f0960 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- 172.21.15.26:0/197501869 >> 172.21.15.26:0/197501869 conn(0x560c7f85e200 msgr2=0x560c7f916f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- 172.21.15.26:0/197501869 shutdown_connections 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- 172.21.15.26:0/197501869 wait complete. 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.710+0000 7f7d41585b80 1 -- start start 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 Processor -- start 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f8f9670 con 0x560c7f8e8c00 2023-10-06T21:19:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f9682e0 con 0x560c7f8f43d0 2023-10-06T21:19:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.711+0000 7f7d41585b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7f979430 con 0x560c7f8e9520 2023-10-06T21:19:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.712+0000 7f7d41585b80 10 client.0 _ll_register_callbacks cb 0x7f7d180179f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.712+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.713+0000 7f7d27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:49558/0 (socket says 172.21.15.26:49558) 2023-10-06T21:19:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.713+0000 7f7d27fff700 1 -- 172.21.15.26:0/3873658976 learned_addr learned my addr 172.21.15.26:0/3873658976 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.713+0000 7f7d277fe700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.713+0000 7f7d26ffd700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 -- 172.21.15.26:0/3873658976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 msgr2=0x560c7f8f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 -- 172.21.15.26:0/3873658976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 msgr2=0x560c7f8f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 -- 172.21.15.26:0/3873658976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c7f650c00 con 0x560c7f8f43d0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d27fff700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d10099020 tx=0x7f7d1000fe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc731f99e27c284 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.714+0000 7f7d06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.715+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1009d470 con 0x560c7f8f43d0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.715+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d100a16b0 con 0x560c7f8f43d0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.715+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d10099320 con 0x560c7f8f43d0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.716+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d10098810 con 0x560c7f8f43d0 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.716+0000 7f7d41585b80 10 client.26804 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.716+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560c7f666250 con 0x560c7f8f43d0 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d10099320 con 0x560c7f8f43d0 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7d100ab070 con 0x560c7f8f43d0 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 10 client.26804 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 10 client.26804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 20 client.26804 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f97a6b0 0x560c7f97cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 4 client.26804 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:19:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.717+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c7f835ad0 con 0x560c7f97a6b0 2023-10-06T21:19:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.718+0000 7f7d277fe700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f97a6b0 0x560c7f97cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.719+0000 7f7d06ffd700 10 client.26804 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.719+0000 7f7d277fe700 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f97a6b0 0x560c7f97cad0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6428bb5fcbdf0e0 server_cookie=c5c0cd9852f39145 in_seq=0 out_seq=0 2023-10-06T21:19:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:43 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:19:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:43 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:19:43.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:43 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:19:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:43 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:19:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:43 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:19:43.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:43 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d06ffd700 1 -- 172.21.15.26:0/3873658976 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c7f835ad0 con 0x560c7f97a6b0 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d06ffd700 10 client.26804 handle_command_reply: tid=0 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 client.26804 shutdown 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 msgr2=0x560c7f967da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d10099020 tx=0x7f7d1000fe50 comp rx=0 tx=0).stop 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 shutdown_connections 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560c7f97a6b0 0x560c7f97cad0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c7f8f43d0 0x560c7f967da0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c7f8e8c00 0x560c7f8f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 --2- 172.21.15.26:0/3873658976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c7f8e9520 0x560c7f8f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 >> 172.21.15.26:0/3873658976 conn(0x560c7f85e200 msgr2=0x560c7f8f1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d06ffd700 0 client.26804 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 shutdown_connections 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 1 -- 172.21.15.26:0/3873658976 wait complete. 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.823+0000 7f7d41585b80 20 client.26804 trim_cache size 0 max 16384 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:19:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:19:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:19:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 msgr2=0x7f7d140a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 0x7f7d140a3cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d280fef80 tx=0x7f7d18007a80 comp rx=0 tx=0).stop 2023-10-06T21:19:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 msgr2=0x7f7d2819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d2819d420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d720 tx=0x7f7d1000da30 comp rx=0 tx=0).stop 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 shutdown_connections 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d28105250 0x7f7d2819d420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d280ffe50 0x7f7d281a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d140a1800 0x7f7d140a3cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 --2- 172.21.15.26:0/2016867896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d28100770 0x7f7d281a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 >> 172.21.15.26:0/2016867896 conn(0x7f7d280fb860 msgr2=0x7f7d28102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 shutdown_connections 2023-10-06T21:19:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:43.824+0000 7f7d2dbff700 1 -- 172.21.15.26:0/2016867896 wait complete. 2023-10-06T21:19:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:43 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:19:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:43 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:19:43.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:43 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:19:44.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:44 smithi167 ceph-mon[134494]: pgmap v292: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 31 MiB/s wr, 1.89k op/s 2023-10-06T21:19:44.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:44 smithi026 ceph-mon[98490]: pgmap v292: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 31 MiB/s wr, 1.89k op/s 2023-10-06T21:19:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:44 smithi078 ceph-mon[134896]: pgmap v292: 97 pgs: 97 active+clean; 6.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 31 MiB/s wr, 1.89k op/s 2023-10-06T21:19:45.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:45 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:45.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:45 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:45.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:45 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:46.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:46 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:46 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:19:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:46 smithi167 ceph-mon[134494]: pgmap v293: 97 pgs: 97 active+clean; 6.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-10-06T21:19:46.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:46 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:46.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:46 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:19:46.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:46 smithi026 ceph-mon[98490]: pgmap v293: 97 pgs: 97 active+clean; 6.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-10-06T21:19:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:46 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:46 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:19:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:46 smithi078 ceph-mon[134896]: pgmap v293: 97 pgs: 97 active+clean; 6.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 32 MiB/s wr, 1.90k op/s 2023-10-06T21:19:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 545.0 s, 1229.2 tps, lat 81.481 ms stddev 90.309 2023-10-06T21:19:48.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:48 smithi167 ceph-mon[134494]: pgmap v294: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 35 MiB/s wr, 2.05k op/s 2023-10-06T21:19:48.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:48 smithi026 ceph-mon[98490]: pgmap v294: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 35 MiB/s wr, 2.05k op/s 2023-10-06T21:19:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:48 smithi078 ceph-mon[134896]: pgmap v294: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 35 MiB/s wr, 2.05k op/s 2023-10-06T21:19:50.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:50 smithi167 ceph-mon[134494]: pgmap v295: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 15 MiB/s wr, 889 op/s 2023-10-06T21:19:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:50 smithi026 ceph-mon[98490]: pgmap v295: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 15 MiB/s wr, 889 op/s 2023-10-06T21:19:50.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:50 smithi078 ceph-mon[134896]: pgmap v295: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 15 MiB/s wr, 889 op/s 2023-10-06T21:19:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 550.0 s, 1065.9 tps, lat 92.518 ms stddev 111.652 2023-10-06T21:19:52.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:52 smithi026 ceph-mon[98490]: pgmap v296: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-10-06T21:19:52.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:52 smithi078 ceph-mon[134896]: pgmap v296: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-10-06T21:19:53.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:52 smithi167 ceph-mon[134494]: pgmap v296: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2023-10-06T21:19:54.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:54 smithi167 ceph-mon[134494]: pgmap v297: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 775 op/s 2023-10-06T21:19:54.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:54 smithi026 ceph-mon[98490]: pgmap v297: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 775 op/s 2023-10-06T21:19:54.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:54 smithi078 ceph-mon[134896]: pgmap v297: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 14 MiB/s wr, 775 op/s 2023-10-06T21:19:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:56 smithi167 ceph-mon[134494]: pgmap v298: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 740 op/s 2023-10-06T21:19:56.830 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:56 smithi026 ceph-mon[98490]: pgmap v298: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 740 op/s 2023-10-06T21:19:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:56 smithi078 ceph-mon[134896]: pgmap v298: 97 pgs: 97 active+clean; 6.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 740 op/s 2023-10-06T21:19:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.130+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/2235893285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 msgr2=0x7fbcb8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.130+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/2235893285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8107eb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009990 tx=0x7fbcb4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 555.0 s, 1060.4 tps, lat 93.129 ms stddev 108.747 2023-10-06T21:19:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.137+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/2235893285 shutdown_connections 2023-10-06T21:19:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.137+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/2235893285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb811a690 0x7fbcb811ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.137+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/2235893285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8107eb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.137+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/2235893285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb810fa90 0x7fbcb81083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.137+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/2235893285 >> 172.21.15.26:0/2235893285 conn(0x7fbcb806cfa0 msgr2=0x7fbcb806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.141+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/2235893285 shutdown_connections 2023-10-06T21:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.150+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/2235893285 wait complete. 2023-10-06T21:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.151+0000 7fbcbf5ff700 1 Processor -- start 2023-10-06T21:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 -- start start 2023-10-06T21:19:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8118990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb811a690 0x7fbcb8111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8111f60 con 0x7fbcb810f620 2023-10-06T21:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb81120d0 con 0x7fbcb810fa90 2023-10-06T21:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.152+0000 7fbcbf5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8112290 con 0x7fbcb811a690 2023-10-06T21:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.153+0000 7fbcbcb9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.153+0000 7fbcbcb9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33512/0 (socket says 172.21.15.26:33512) 2023-10-06T21:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.153+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/3638863182 learned_addr learned my addr 172.21.15.26:0/3638863182 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.156+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb811a690 0x7fbcb8111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/3638863182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb811a690 msgr2=0x7fbcb8111790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb811a690 0x7fbcb8111790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 msgr2=0x7fbcb8118990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8118990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.166+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/3638863182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40095f0 con 0x7fbcb810fa90 2023-10-06T21:19:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.171+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbca800b530 tx=0x7fbca800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.183+0000 7fbcae7fc700 1 -- 172.21.15.26:0/3638863182 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8011830 con 0x7fbcb810fa90 2023-10-06T21:19:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.183+0000 7fbcae7fc700 1 -- 172.21.15.26:0/3638863182 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca8011e50 con 0x7fbcb810fa90 2023-10-06T21:19:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.183+0000 7fbcae7fc700 1 -- 172.21.15.26:0/3638863182 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800f2e0 con 0x7fbcb810fa90 2023-10-06T21:19:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.184+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb8112570 con 0x7fbcb810fa90 2023-10-06T21:19:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.184+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb81167c0 con 0x7fbcb810fa90 2023-10-06T21:19:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.191+0000 7fbcd2f79b80 1 Processor -- start 2023-10-06T21:19:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 -- start start 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e79e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5dd8400 con 0x55abf5e6ec00 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5dc9bc0 con 0x55abf5e7a3d0 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.192+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5dc74e0 con 0x55abf5e6f520 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.193+0000 7fbcbdb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.193+0000 7fbcbdb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54704/0 (socket says 172.21.15.26:54704) 2023-10-06T21:19:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.193+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/193203811 learned_addr learned my addr 172.21.15.26:0/193203811 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.194+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/193203811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 msgr2=0x55abf5e79e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.194+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e79e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.194+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/193203811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e7a3d0 msgr2=0x55abf5e7c7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:19:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.194+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7c7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.194+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/193203811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf5bedd80 con 0x55abf5e6ec00 2023-10-06T21:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.210+0000 7fbcae7fc700 1 -- 172.21.15.26:0/3638863182 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbca8010680 con 0x7fbcb810fa90 2023-10-06T21:19:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.212+0000 7fbcae7fc700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 0x7fbca40a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.213+0000 7fbcae7fc700 1 -- 172.21.15.26:0/3638863182 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbca80cc9e0 con 0x7fbcb810fa90 2023-10-06T21:19:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.219+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0003f90 tx=0x7fbcb0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e48853b9d53e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.233+0000 7fbca37fe700 1 -- 172.21.15.26:0/193203811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004bb0 con 0x55abf5e6ec00 2023-10-06T21:19:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.233+0000 7fbca37fe700 1 -- 172.21.15.26:0/193203811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb0004d10 con 0x55abf5e6ec00 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.233+0000 7fbca37fe700 1 -- 172.21.15.26:0/193203811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0007cc0 con 0x55abf5e6ec00 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.233+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 0x7fbca40a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.234+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/193203811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 msgr2=0x55abf5e6efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.234+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0003f90 tx=0x7fbcb0009ba0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.234+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 0x7fbca40a3cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400b5c0 tx=0x7fbcb4005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.235+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/193203811 shutdown_connections 2023-10-06T21:19:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.235+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.235+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e6efe0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.235+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/193203811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.235+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/193203811 >> 172.21.15.26:0/193203811 conn(0x55abf5de4200 msgr2=0x55abf5e6b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.238+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/193203811 shutdown_connections 2023-10-06T21:19:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.239+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/193203811 wait complete. 2023-10-06T21:19:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.239+0000 7fbcd2f79b80 1 -- start start 2023-10-06T21:19:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.240+0000 7fbcd2f79b80 1 Processor -- start 2023-10-06T21:19:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e77740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 0x55abf5e77c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5dc2dd0 con 0x55abf5e7a3d0 2023-10-06T21:19:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5d5f6b0 con 0x55abf5e6ec00 2023-10-06T21:19:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5d5f350 con 0x55abf5e6f520 2023-10-06T21:19:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.241+0000 7fbcd2f79b80 10 client.0 _ll_register_callbacks cb 0x55abf5bbc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.244+0000 7fbcbcb9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e77740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.244+0000 7fbcbd39b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.244+0000 7fbcbd39b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e77200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33534/0 (socket says 172.21.15.26:33534) 2023-10-06T21:19:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.244+0000 7fbcbd39b700 1 -- 172.21.15.26:0/652641943 learned_addr learned my addr 172.21.15.26:0/652641943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.250+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 0x55abf5e77c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.251+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/652641943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 msgr2=0x55abf5e77740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.251+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e77740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.251+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 msgr2=0x55abf5e77200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.251+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e77200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.251+0000 7fbcbdb9c700 1 -- 172.21.15.26:0/652641943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb0003bf0 con 0x55abf5e7a3d0 2023-10-06T21:19:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.252+0000 7fbcbdb9c700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 0x55abf5e77c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0000f80 tx=0x7fbcb0007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf59917e27bebb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.252+0000 7fbca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.253+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004550 con 0x55abf5e7a3d0 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.253+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbcb00046b0 con 0x55abf5e7a3d0 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.253+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abf5e62d50 con 0x55abf5e7a3d0 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.254+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb0004890 con 0x55abf5e7a3d0 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.254+0000 7fbcd2f79b80 10 client.16515 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.254+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55abf5bd6c00 con 0x55abf5e7a3d0 2023-10-06T21:19:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abf5e62d50 con 0x55abf5e7a3d0 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbcb0035070 con 0x55abf5e7a3d0 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 10 client.16515 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 20 client.16515 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5e89e80 0x55abf5e8c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 4 client.16515 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:19:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.255+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abf5dbbdb0 con 0x55abf5e89e80 2023-10-06T21:19:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.268+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5e89e80 0x55abf5e8c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.290+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5e89e80 0x55abf5e8c2a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c8284dec7d433f server_cookie=9b3b24f72305c0d1 in_seq=0 out_seq=0 2023-10-06T21:19:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.290+0000 7fbca0ff9700 10 client.16515 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.321+0000 7fbca0ff9700 1 -- 172.21.15.26:0/652641943 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abf5dbbdb0 con 0x55abf5e89e80 2023-10-06T21:19:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.321+0000 7fbca0ff9700 10 client.16515 handle_command_reply: tid=0 2023-10-06T21:19:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.321+0000 7fbcd2f79b80 1 client.16515 shutdown 2023-10-06T21:19:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.322+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 msgr2=0x55abf5e77c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.322+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 0x55abf5e77c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0000f80 tx=0x7fbcb0007b40 comp rx=0 tx=0).stop 2023-10-06T21:19:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 shutdown_connections 2023-10-06T21:19:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e7a3d0 0x55abf5e77c80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5e89e80 0x55abf5e8c2a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/652641943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e6f520 0x55abf5e77740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.342+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 >> 172.21.15.26:0/652641943 conn(0x55abf5de4200 msgr2=0x55abf5e6ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.350+0000 7fbca0ff9700 0 client.16515 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.358+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 shutdown_connections 2023-10-06T21:19:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.365+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/652641943 wait complete. 2023-10-06T21:19:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.374+0000 7fbcd2f79b80 20 client.16515 trim_cache size 0 max 16384 2023-10-06T21:19:57.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.405+0000 7fbcd2f79b80 1 Processor -- start 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.408+0000 7fbcd2f79b80 1 -- start start 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e80f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e74490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5c438c0 con 0x55abf5e6f520 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5e67860 con 0x55abf5e6ec00 2023-10-06T21:19:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.409+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5e81450 con 0x55abf5e7a3d0 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.410+0000 7fbcbcb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.410+0000 7fbcbcb9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54724/0 (socket says 172.21.15.26:54724) 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.410+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/2281603220 learned_addr learned my addr 172.21.15.26:0/2281603220 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.411+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/2281603220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 msgr2=0x55abf5e74490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.411+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e74490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.411+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/2281603220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 msgr2=0x55abf5e80f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.411+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e80f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.411+0000 7fbcbcb9a700 1 -- 172.21.15.26:0/2281603220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf5c141f0 con 0x55abf5e6f520 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.412+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbca8095f90 tx=0x7fbca8010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75f3aabdc901511 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.415+0000 7fbca3fff700 1 -- 172.21.15.26:0/2281603220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a1e70 con 0x55abf5e6f520 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.415+0000 7fbca3fff700 1 -- 172.21.15.26:0/2281603220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca80b2220 con 0x55abf5e6f520 2023-10-06T21:19:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.415+0000 7fbca3fff700 1 -- 172.21.15.26:0/2281603220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8099d40 con 0x55abf5e6f520 2023-10-06T21:19:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.422+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/2281603220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 msgr2=0x55abf5e77ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.422+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbca8095f90 tx=0x7fbca8010a80 comp rx=0 tx=0).stop 2023-10-06T21:19:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/2281603220 shutdown_connections 2023-10-06T21:19:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e74490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6ec00 0x55abf5e80f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/2281603220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6f520 0x55abf5e77ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/2281603220 >> 172.21.15.26:0/2281603220 conn(0x55abf5de4200 msgr2=0x55abf5e72530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/2281603220 shutdown_connections 2023-10-06T21:19:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.423+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/2281603220 wait complete. 2023-10-06T21:19:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.424+0000 7fbcd2f79b80 1 -- start start 2023-10-06T21:19:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.424+0000 7fbcd2f79b80 1 Processor -- start 2023-10-06T21:19:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6f520 0x55abf5e80cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5e7f5f0 con 0x55abf5e6ec00 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5e7f760 con 0x55abf5e6f520 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf5e7f920 con 0x55abf5e7a3d0 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.425+0000 7fbcd2f79b80 10 client.0 _ll_register_callbacks cb 0x7fbcb0003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.426+0000 7fbcbdb9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.427+0000 7fbcbd39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.427+0000 7fbcbd39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54740/0 (socket says 172.21.15.26:54740) 2023-10-06T21:19:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.427+0000 7fbcbd39b700 1 -- 172.21.15.26:0/1316991720 learned_addr learned my addr 172.21.15.26:0/1316991720 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:19:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.427+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6f520 0x55abf5e80cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.428+0000 7fbcbd39b700 1 -- 172.21.15.26:0/1316991720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 msgr2=0x55abf5e7ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.428+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.428+0000 7fbcbd39b700 1 -- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6f520 msgr2=0x55abf5e80cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:19:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.428+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6f520 0x55abf5e80cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.428+0000 7fbcbd39b700 1 -- 172.21.15.26:0/1316991720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb81167c0 con 0x55abf5e6ec00 2023-10-06T21:19:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.429+0000 7fbcbd39b700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400bb60 tx=0x7fbcb401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5063779c248f01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:19:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.429+0000 7fbca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:19:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.430+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb402b040 con 0x55abf5e6ec00 2023-10-06T21:19:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.430+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbcb4033050 con 0x55abf5e6ec00 2023-10-06T21:19:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.430+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcb0004d10 con 0x55abf5e6ec00 2023-10-06T21:19:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.431+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb4033210 con 0x55abf5e6ec00 2023-10-06T21:19:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.431+0000 7fbcd2f79b80 10 client.16521 fetch_fsmap learned FSMap version 49 2023-10-06T21:19:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.431+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55abf5bec250 con 0x55abf5e6ec00 2023-10-06T21:19:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.434+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb0004d10 con 0x55abf5e6ec00 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.434+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbcb403c450 con 0x55abf5e6ec00 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 20 client.16521 populate_metadata read hostname 'smithi026' 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5edeec0 0x55abf5ee12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 4 client.16521 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:19:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.435+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abf5dbbad0 con 0x55abf5edeec0 2023-10-06T21:19:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.441+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5edeec0 0x55abf5ee12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:19:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.441+0000 7fbcbcb9a700 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5edeec0 0x55abf5ee12e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1950809803134f8e server_cookie=2e36eb4af6481762 in_seq=0 out_seq=0 2023-10-06T21:19:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.442+0000 7fbca0ff9700 10 client.16521 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbca0ff9700 1 -- 172.21.15.26:0/1316991720 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abf5dbbad0 con 0x55abf5edeec0 2023-10-06T21:19:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbca0ff9700 10 client.16521 handle_command_reply: tid=0 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 client.16521 shutdown 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 msgr2=0x55abf5e80770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400bb60 tx=0x7fbcb401a050 comp rx=0 tx=0).stop 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 shutdown_connections 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf5e7a3d0 0x55abf5e7ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf5e6ec00 0x55abf5e80770 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf5edeec0 0x55abf5ee12e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 --2- 172.21.15.26:0/1316991720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf5e6f520 0x55abf5e80cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 >> 172.21.15.26:0/1316991720 conn(0x55abf5de4200 msgr2=0x55abf5e759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbca0ff9700 0 client.16521 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:19:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.589+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 shutdown_connections 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcd2f79b80 1 -- 172.21.15.26:0/1316991720 wait complete. 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcd2f79b80 20 client.16521 trim_cache size 0 max 16384 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:19:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 msgr2=0x7fbca40a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 0x7fbca40a3cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400b5c0 tx=0x7fbcb4005fb0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 msgr2=0x7fbcb8118f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.590+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbca800b530 tx=0x7fbca800b840 comp rx=0 tx=0).stop 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 shutdown_connections 2023-10-06T21:19:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb811a690 0x7fbcb8111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbca40a1800 0x7fbca40a3cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb810f620 0x7fbcb8118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 --2- 172.21.15.26:0/3638863182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcb810fa90 0x7fbcb8118f10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 >> 172.21.15.26:0/3638863182 conn(0x7fbcb806cfa0 msgr2=0x7fbcb806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 shutdown_connections 2023-10-06T21:19:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:19:57.591+0000 7fbcbf5ff700 1 -- 172.21.15.26:0/3638863182 wait complete. 2023-10-06T21:19:58.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:19:58 smithi167 ceph-mon[134494]: pgmap v299: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 985 op/s 2023-10-06T21:19:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:19:58 smithi026 ceph-mon[98490]: pgmap v299: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 985 op/s 2023-10-06T21:19:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:19:58 smithi078 ceph-mon[134896]: pgmap v299: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 985 op/s 2023-10-06T21:20:00.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:00 smithi167 ceph-mon[134494]: pgmap v300: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-10-06T21:20:00.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:00 smithi167 ceph-mon[134494]: overall HEALTH_OK 2023-10-06T21:20:00.849 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:00 smithi026 ceph-mon[98490]: pgmap v300: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-10-06T21:20:00.849 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:00 smithi026 ceph-mon[98490]: overall HEALTH_OK 2023-10-06T21:20:00.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:00 smithi078 ceph-mon[134896]: pgmap v300: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-10-06T21:20:00.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:00 smithi078 ceph-mon[134896]: overall HEALTH_OK 2023-10-06T21:20:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 560.0 s, 1177.5 tps, lat 86.638 ms stddev 113.826 2023-10-06T21:20:02.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:02 smithi167 ceph-mon[134494]: pgmap v301: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2023-10-06T21:20:02.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:02 smithi026 ceph-mon[98490]: pgmap v301: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2023-10-06T21:20:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:02 smithi078 ceph-mon[134896]: pgmap v301: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2023-10-06T21:20:04.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:04 smithi167 ceph-mon[134494]: pgmap v302: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-10-06T21:20:04.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:04 smithi026 ceph-mon[98490]: pgmap v302: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-10-06T21:20:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:04 smithi078 ceph-mon[134896]: pgmap v302: 97 pgs: 97 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-10-06T21:20:06.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:06 smithi167 ceph-mon[134494]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:20:06.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:06 smithi167 ceph-mon[134494]: pgmap v304: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:20:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:06 smithi026 ceph-mon[98490]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:20:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:06 smithi026 ceph-mon[98490]: pgmap v304: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:20:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:06 smithi078 ceph-mon[134896]: osdmap e96: 12 total, 12 up, 12 in 2023-10-06T21:20:06.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:06 smithi078 ceph-mon[134896]: pgmap v304: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 22 MiB/s wr, 1.54k op/s 2023-10-06T21:20:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 565.0 s, 1238.0 tps, lat 81.192 ms stddev 101.009 2023-10-06T21:20:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:08 smithi167 ceph-mon[134494]: pgmap v305: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:08.830 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:08 smithi026 ceph-mon[98490]: pgmap v305: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:08 smithi078 ceph-mon[134896]: pgmap v305: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:09.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.592+0000 7fd0975ff700 1 -- 172.21.15.26:0/1367746048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 msgr2=0x7fd0901014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.592+0000 7fd0975ff700 1 --2- 172.21.15.26:0/1367746048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0901014a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009f90 tx=0x7fd08c00b330 comp rx=0 tx=0).stop 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 -- 172.21.15.26:0/1367746048 shutdown_connections 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 --2- 172.21.15.26:0/1367746048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0901061e0 0x7fd09010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 --2- 172.21.15.26:0/1367746048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0901014a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 --2- 172.21.15.26:0/1367746048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd090101a70 0x7fd090105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 -- 172.21.15.26:0/1367746048 >> 172.21.15.26:0/1367746048 conn(0x7fd0900fc960 msgr2=0x7fd0900fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 -- 172.21.15.26:0/1367746048 shutdown_connections 2023-10-06T21:20:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.593+0000 7fd0975ff700 1 -- 172.21.15.26:0/1367746048 wait complete. 2023-10-06T21:20:09.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.594+0000 7fd0975ff700 1 Processor -- start 2023-10-06T21:20:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 -- start start 2023-10-06T21:20:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0901061e0 0x7fd0900736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd090073c30 0x7fd0901a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090075100 con 0x7fd0901061e0 2023-10-06T21:20:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901a9d40 con 0x7fd0901010c0 2023-10-06T21:20:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.595+0000 7fd0975ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0901a9e80 con 0x7fd090073c30 2023-10-06T21:20:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38890/0 (socket says 172.21.15.26:38890) 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 -- 172.21.15.26:0/981731161 learned_addr learned my addr 172.21.15.26:0/981731161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd095b9c700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd090073c30 0x7fd0901a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 -- 172.21.15.26:0/981731161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd090073c30 msgr2=0x7fd0901a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd090073c30 0x7fd0901a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 -- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0901061e0 msgr2=0x7fd0900736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0901061e0 0x7fd0900736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.596+0000 7fd09539b700 1 -- 172.21.15.26:0/981731161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c009c40 con 0x7fd0901010c0 2023-10-06T21:20:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd09539b700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b6c0 tx=0x7fd08c004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd0867fc700 1 -- 172.21.15.26:0/981731161 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00e040 con 0x7fd0901010c0 2023-10-06T21:20:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0901aa100 con 0x7fd0901010c0 2023-10-06T21:20:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd0867fc700 1 -- 172.21.15.26:0/981731161 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08c00be00 con 0x7fd0901010c0 2023-10-06T21:20:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd0867fc700 1 -- 172.21.15.26:0/981731161 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c005400 con 0x7fd0901010c0 2023-10-06T21:20:09.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.597+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901aa5d0 con 0x7fd0901010c0 2023-10-06T21:20:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.601+0000 7fd0867fc700 1 -- 172.21.15.26:0/981731161 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fd08c01da30 con 0x7fd0901010c0 2023-10-06T21:20:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.604+0000 7fd0867fc700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 0x7fd07c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.604+0000 7fd0867fc700 1 -- 172.21.15.26:0/981731161 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fd08c0ce870 con 0x7fd0901010c0 2023-10-06T21:20:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.605+0000 7fd0aaef1b80 1 Processor -- start 2023-10-06T21:20:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.605+0000 7fd094b9a700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 0x7fd07c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.606+0000 7fd0aaef1b80 1 -- start start 2023-10-06T21:20:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.606+0000 7fd094b9a700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 0x7fd07c0a3b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd080005fd0 tx=0x7fd080005d00 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.606+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f09e0 0x5638064fb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064fb890 0x5638064fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563806459460 con 0x5638064f00c0 2023-10-06T21:20:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380644ac20 con 0x5638064f09e0 2023-10-06T21:20:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563806448540 con 0x5638064fb890 2023-10-06T21:20:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd09539b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd09539b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54084/0 (socket says 172.21.15.26:54084) 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd09539b700 1 -- 172.21.15.26:0/980555827 learned_addr learned my addr 172.21.15.26:0/980555827 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.607+0000 7fd094b9a700 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f09e0 0x5638064fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd095b9c700 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064fb890 0x5638064fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd09539b700 1 -- 172.21.15.26:0/980555827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064fb890 msgr2=0x5638064fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd09539b700 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064fb890 0x5638064fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd09539b700 1 -- 172.21.15.26:0/980555827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f09e0 msgr2=0x5638064fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd09539b700 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f09e0 0x5638064fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.608+0000 7fd09539b700 1 -- 172.21.15.26:0/980555827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56380626ede0 con 0x5638064f00c0 2023-10-06T21:20:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.609+0000 7fd09539b700 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0982b0 tx=0x7fd08c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecaa52b5919c4c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.610+0000 7fd07b7fe700 1 -- 172.21.15.26:0/980555827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c09bfa0 con 0x5638064f00c0 2023-10-06T21:20:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.610+0000 7fd07b7fe700 1 -- 172.21.15.26:0/980555827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08c09b840 con 0x5638064f00c0 2023-10-06T21:20:09.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.610+0000 7fd07b7fe700 1 -- 172.21.15.26:0/980555827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c098130 con 0x5638064f00c0 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/980555827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 msgr2=0x5638064f04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0982b0 tx=0x7fd08c00b330 comp rx=0 tx=0).stop 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/980555827 shutdown_connections 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064fb890 0x5638064fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f00c0 0x5638064f04a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/980555827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f09e0 0x5638064fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.611+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/980555827 >> 172.21.15.26:0/980555827 conn(0x563806465260 msgr2=0x5638064ec450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.612+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/980555827 shutdown_connections 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.612+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/980555827 wait complete. 2023-10-06T21:20:09.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.612+0000 7fd0aaef1b80 1 -- start start 2023-10-06T21:20:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.612+0000 7fd0aaef1b80 1 Processor -- start 2023-10-06T21:20:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 0x5638064fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064fb890 0x563806506cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563806443e30 con 0x5638064f09e0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638063e0710 con 0x5638064fb890 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638063e03b0 con 0x5638064f00c0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd0aaef1b80 10 client.0 _ll_register_callbacks cb 0x56380623d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd09539b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd09539b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36186/0 (socket says 172.21.15.26:36186) 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd09539b700 1 -- 172.21.15.26:0/3320928195 learned_addr learned my addr 172.21.15.26:0/3320928195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd095b9c700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064fb890 0x563806506cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.613+0000 7fd094b9a700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 0x5638064fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 -- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064fb890 msgr2=0x563806506cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064fb890 0x563806506cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 -- 172.21.15.26:0/3320928195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 msgr2=0x5638064fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 0x5638064fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 -- 172.21.15.26:0/3320928195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901aa5d0 con 0x5638064f00c0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd094b9a700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 0x5638064fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd09539b700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0a3610 tx=0x7fd08c09bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c3a0b5adcb09b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd078ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0862f0 con 0x5638064f00c0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fd08c0a3b40 con 0x5638064f00c0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.614+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638064e3f10 con 0x5638064f00c0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd08c09baa0 con 0x5638064f00c0 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 10 client.26713 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563806257c60 con 0x5638064f00c0 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638064e3f10 con 0x5638064f00c0 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd08c0a36b0 con 0x5638064f00c0 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 10 client.26713 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 20 client.26713 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380650ab90 0x56380650cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 4 client.26713 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.615+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56380643ce10 con 0x56380650ab90 2023-10-06T21:20:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.616+0000 7fd094b9a700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380650ab90 0x56380650cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.619+0000 7fd094b9a700 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380650ab90 0x56380650cf80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70134ed246ec3b8e server_cookie=e8e404ca8060c30f in_seq=0 out_seq=0 2023-10-06T21:20:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.619+0000 7fd078ff9700 10 client.26713 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd078ff9700 1 -- 172.21.15.26:0/3320928195 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56380643ce10 con 0x56380650ab90 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd078ff9700 10 client.26713 handle_command_reply: tid=0 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 client.26713 shutdown 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 msgr2=0x5638064fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0a3610 tx=0x7fd08c09bfa0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 shutdown_connections 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380650ab90 0x56380650cf80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064fb890 0x563806506cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f00c0 0x5638064fbc70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/3320928195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064f09e0 0x5638064fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 >> 172.21.15.26:0/3320928195 conn(0x563806465260 msgr2=0x5638064ebfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd078ff9700 0 client.26713 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 shutdown_connections 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.620+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/3320928195 wait complete. 2023-10-06T21:20:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.621+0000 7fd0aaef1b80 20 client.26713 trim_cache size 0 max 16384 2023-10-06T21:20:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.655+0000 7fd0aaef1b80 1 Processor -- start 2023-10-06T21:20:09.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.656+0000 7fd0aaef1b80 1 -- start start 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064f5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x56380650cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638062c4920 con 0x5638064fb890 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638064fc520 con 0x5638064f00c0 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.658+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638064fb0e0 con 0x5638064f09e0 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.663+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.663+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38920/0 (socket says 172.21.15.26:38920) 2023-10-06T21:20:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.663+0000 7fd09539b700 1 -- 172.21.15.26:0/2557804699 learned_addr learned my addr 172.21.15.26:0/2557804699 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.663+0000 7fd094b9a700 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.664+0000 7fd09539b700 1 -- 172.21.15.26:0/2557804699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 msgr2=0x5638064f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.664+0000 7fd09539b700 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.664+0000 7fd09539b700 1 -- 172.21.15.26:0/2557804699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 msgr2=0x56380650cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.664+0000 7fd09539b700 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x56380650cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.664+0000 7fd09539b700 1 -- 172.21.15.26:0/2557804699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638064e8140 con 0x5638064f00c0 2023-10-06T21:20:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.665+0000 7fd09539b700 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd08c097d40 tx=0x7fd08c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32ab4d324e4098e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.665+0000 7fd07bfff700 1 -- 172.21.15.26:0/2557804699 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0a3b40 con 0x5638064f00c0 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.665+0000 7fd07bfff700 1 -- 172.21.15.26:0/2557804699 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08c0860b0 con 0x5638064f00c0 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.665+0000 7fd07bfff700 1 -- 172.21.15.26:0/2557804699 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c09baa0 con 0x5638064f00c0 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.666+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2557804699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 msgr2=0x5638064f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.666+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd08c097d40 tx=0x7fd08c00b330 comp rx=0 tx=0).stop 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2557804699 shutdown_connections 2023-10-06T21:20:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x56380650cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064f5000 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2557804699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2557804699 >> 172.21.15.26:0/2557804699 conn(0x563806465260 msgr2=0x563806507410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2557804699 shutdown_connections 2023-10-06T21:20:09.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.667+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2557804699 wait complete. 2023-10-06T21:20:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.668+0000 7fd0aaef1b80 1 -- start start 2023-10-06T21:20:09.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.668+0000 7fd0aaef1b80 1 Processor -- start 2023-10-06T21:20:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x5638064ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380655d960 con 0x5638064fb890 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380655daa0 con 0x5638064f00c0 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56380655dc60 con 0x5638064f09e0 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd0aaef1b80 10 client.0 _ll_register_callbacks cb 0x7fd08c09b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd09539b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38922/0 (socket says 172.21.15.26:38922) 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd09539b700 1 -- 172.21.15.26:0/2933001286 learned_addr learned my addr 172.21.15.26:0/2933001286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd095b9c700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x5638064ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd09539b700 1 -- 172.21.15.26:0/2933001286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 msgr2=0x5638064ff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.669+0000 7fd09539b700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd09539b700 1 -- 172.21.15.26:0/2933001286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 msgr2=0x5638064ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd09539b700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x5638064ff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd09539b700 1 -- 172.21.15.26:0/2933001286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563806257c60 con 0x5638064f00c0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd09539b700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0982b0 tx=0x7fd08c0b41a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7af034f274b6d16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd078ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c09cc10 con 0x5638064f00c0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fd08c09baa0 con 0x5638064f00c0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd08c09b840 con 0x5638064f00c0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd08c09bca0 con 0x5638064f00c0 2023-10-06T21:20:09.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd0aaef1b80 10 client.26831 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.670+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563806295250 con 0x5638064f00c0 2023-10-06T21:20:09.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08c09b840 con 0x5638064f00c0 2023-10-06T21:20:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd08c01d670 con 0x5638064f00c0 2023-10-06T21:20:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 10 client.26831 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 10 client.26831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 20 client.26831 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:09.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380655ee40 0x563806561230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 4 client.26831 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.671+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56380643cb30 con 0x56380655ee40 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd094b9a700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380655ee40 0x563806561230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd078ff9700 10 client.26831 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd094b9a700 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380655ee40 0x563806561230 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8473c99d89d6ecb server_cookie=3d2d0867ba52681d in_seq=0 out_seq=0 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd078ff9700 1 -- 172.21.15.26:0/2933001286 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56380643cb30 con 0x56380655ee40 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd078ff9700 10 client.26831 handle_command_reply: tid=0 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.672+0000 7fd0aaef1b80 1 client.26831 shutdown 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 msgr2=0x5638064fec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0982b0 tx=0x7fd08c0b41a0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 shutdown_connections 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638064fb890 0x5638064ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56380655ee40 0x563806561230 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638064f00c0 0x5638064fec70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 --2- 172.21.15.26:0/2933001286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638064f09e0 0x5638064ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 >> 172.21.15.26:0/2933001286 conn(0x563806465260 msgr2=0x56380650cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd078ff9700 0 client.26831 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 shutdown_connections 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 1 -- 172.21.15.26:0/2933001286 wait complete. 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.673+0000 7fd0aaef1b80 20 client.26831 trim_cache size 0 max 16384 2023-10-06T21:20:09.679 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:09.680 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:20:09.680 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:20:09.680 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 msgr2=0x7fd07c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 0x7fd07c0a3b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd080005fd0 tx=0x7fd080005d00 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 msgr2=0x7fd0900731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b6c0 tx=0x7fd08c004b80 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 shutdown_connections 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd090073c30 0x7fd0901a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0901010c0 0x7fd0900731b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fd07c0a1670 0x7fd07c0a3b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 --2- 172.21.15.26:0/981731161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0901061e0 0x7fd0900736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 >> 172.21.15.26:0/981731161 conn(0x7fd0900fc960 msgr2=0x7fd090104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 shutdown_connections 2023-10-06T21:20:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:09.674+0000 7fd0975ff700 1 -- 172.21.15.26:0/981731161 wait complete. 2023-10-06T21:20:09.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:20:09.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.015+0000 7f748e3ff700 1 -- 172.21.15.26:0/3046992166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 msgr2=0x7f7488116520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.015+0000 7f748e3ff700 1 --2- 172.21.15.26:0/3046992166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f7488116520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7478009990 tx=0x7f7478009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 -- 172.21.15.26:0/3046992166 shutdown_connections 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 --2- 172.21.15.26:0/3046992166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f7488116520 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 --2- 172.21.15.26:0/3046992166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748810f660 0x7f748810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 --2- 172.21.15.26:0/3046992166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74881096a0 0x7f7488109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 -- 172.21.15.26:0/3046992166 >> 172.21.15.26:0/3046992166 conn(0x7f748806dca0 msgr2=0x7f74880700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 -- 172.21.15.26:0/3046992166 shutdown_connections 2023-10-06T21:20:10.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.017+0000 7f748e3ff700 1 -- 172.21.15.26:0/3046992166 wait complete. 2023-10-06T21:20:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 Processor -- start 2023-10-06T21:20:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 -- start start 2023-10-06T21:20:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74881096a0 0x7f74881141b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f748810f660 0x7f74881aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881aacb0 con 0x7f74881096a0 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881aae20 con 0x7f748810f660 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.019+0000 7f748e3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881aafe0 con 0x7f748810a060 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36222/0 (socket says 172.21.15.26:36222) 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 -- 172.21.15.26:0/2157176582 learned_addr learned my addr 172.21.15.26:0/2157176582 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f7487fff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74881096a0 0x7f74881141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f748c99c700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f748810f660 0x7f74881aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 -- 172.21.15.26:0/2157176582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f748810f660 msgr2=0x7f74881aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f748810f660 0x7f74881aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 -- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74881096a0 msgr2=0x7f74881141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74881096a0 0x7f74881141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.020+0000 7f74877fe700 1 -- 172.21.15.26:0/2157176582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780095f0 con 0x7f748810a060 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f74877fe700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f748000ef10 tx=0x7f748000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000c9c0 con 0x7f748810a060 2023-10-06T21:20:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748000f040 con 0x7f748810a060 2023-10-06T21:20:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480014480 con 0x7f748810a060 2023-10-06T21:20:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74881ab260 con 0x7f748810a060 2023-10-06T21:20:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.021+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74881b2b40 con 0x7f748810a060 2023-10-06T21:20:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.023+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748810c8f0 con 0x7f748810a060 2023-10-06T21:20:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.026+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7480007690 con 0x7f748810a060 2023-10-06T21:20:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.029+0000 7f74857fa700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 0x7f74700a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.029+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f74800cd3c0 con 0x7f748810a060 2023-10-06T21:20:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.029+0000 7f7487fff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 0x7f74700a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.030+0000 7f7487fff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 0x7f74700a3bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f747c009be0 tx=0x7f747c009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.032+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7480096640 con 0x7f748810a060 2023-10-06T21:20:10.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.464+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74881195c0 con 0x7f748810a060 2023-10-06T21:20:10.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.470+0000 7f74857fa700 1 -- 172.21.15.26:0/2157176582 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11314 (secure 0 0 0) 0x7f7480099c60 con 0x7f748810a060 2023-10-06T21:20:10.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:20:10.477 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:19:09.459935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 msgr2=0x7f74700a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 0x7f74700a3bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f747c009be0 tx=0x7f747c009260 comp rx=0 tx=0).stop 2023-10-06T21:20:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 msgr2=0x7f74881146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f748000ef10 tx=0x7f748000c3a0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 shutdown_connections 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f748810f660 0x7f74881aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f74700a1730 0x7f74700a3bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74881096a0 0x7f74881141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 --2- 172.21.15.26:0/2157176582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f748810a060 0x7f74881146f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.482+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 >> 172.21.15.26:0/2157176582 conn(0x7f748806dca0 msgr2=0x7f748806a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.491+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 shutdown_connections 2023-10-06T21:20:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.492+0000 7f748e3ff700 1 -- 172.21.15.26:0/2157176582 wait complete. 2023-10-06T21:20:10.498 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-10-06T21:20:10.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:20:10.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:10 smithi167 ceph-mon[134494]: pgmap v306: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:10.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2157176582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:10.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:10 smithi026 ceph-mon[98490]: pgmap v306: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:10.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2157176582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 -- 172.21.15.26:0/74691891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073250 msgr2=0x7f9090073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 --2- 172.21.15.26:0/74691891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073250 0x7f9090073630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9080009990 tx=0x7f9080009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 -- 172.21.15.26:0/74691891 shutdown_connections 2023-10-06T21:20:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 --2- 172.21.15.26:0/74691891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090111e50 0x7f9090114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 --2- 172.21.15.26:0/74691891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073250 0x7f9090073630 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 --2- 172.21.15.26:0/74691891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090073b70 0x7f9090111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 -- 172.21.15.26:0/74691891 >> 172.21.15.26:0/74691891 conn(0x7f90900fc950 msgr2=0x7f90900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 -- 172.21.15.26:0/74691891 shutdown_connections 2023-10-06T21:20:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.842+0000 7f90951ff700 1 -- 172.21.15.26:0/74691891 wait complete. 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 -- 172.21.15.26:0/529618315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 msgr2=0x7fe0cc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/529618315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc102160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80099c0 tx=0x7fe0c8009cd0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 -- 172.21.15.26:0/529618315 shutdown_connections 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/529618315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc1000a0 0x7fe0cc10fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/529618315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc102160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/529618315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc0ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.842+0000 7fe0d41ff700 1 -- 172.21.15.26:0/529618315 >> 172.21.15.26:0/529618315 conn(0x7fe0cc0762e0 msgr2=0x7fe0cc0766f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.843+0000 7fe0d41ff700 1 -- 172.21.15.26:0/529618315 shutdown_connections 2023-10-06T21:20:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.843+0000 7fe0d41ff700 1 -- 172.21.15.26:0/529618315 wait complete. 2023-10-06T21:20:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.843+0000 7f90951ff700 1 Processor -- start 2023-10-06T21:20:10.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.843+0000 7f90951ff700 1 -- start start 2023-10-06T21:20:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.844+0000 7fe0d41ff700 1 Processor -- start 2023-10-06T21:20:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090111e50 0x7f909019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019dc00 con 0x7f9090073250 2023-10-06T21:20:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019dd70 con 0x7f9090073b70 2023-10-06T21:20:10.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f90951ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019df30 con 0x7f9090111e50 2023-10-06T21:20:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.844+0000 7fe0d41ff700 1 -- start start 2023-10-06T21:20:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090111e50 0x7f909019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35646/0 (socket says 172.21.15.26:35646) 2023-10-06T21:20:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52390/0 (socket says 172.21.15.26:52390) 2023-10-06T21:20:10.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.844+0000 7f908ed9d700 1 -- 172.21.15.26:0/1832234472 learned_addr learned my addr 172.21.15.26:0/1832234472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.844+0000 7fe0d41ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d41ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc1000a0 0x7fe0cc10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d41ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc1a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d41ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc1a9ab0 con 0x7fe0cc0ff6e0 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d41ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc1a9bf0 con 0x7fe0cc101d80 2023-10-06T21:20:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d41ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc1a9d30 con 0x7fe0cc1000a0 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 -- 172.21.15.26:0/1832234472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090111e50 msgr2=0x7f909019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090111e50 0x7f909019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 -- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 msgr2=0x7f90901a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 -- 172.21.15.26:0/1832234472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800095f0 con 0x7f9090073b70 2023-10-06T21:20:10.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908ed9d700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc1a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35648/0 (socket says 172.21.15.26:35648) 2023-10-06T21:20:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/1649136429 learned_addr learned my addr 172.21.15.26:0/1649136429 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d179a700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc1000a0 0x7fe0cc10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.845+0000 7f908e59c700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f908400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 -- 172.21.15.26:0/1649136429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc1000a0 msgr2=0x7fe0cc10e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc1000a0 0x7fe0cc10e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 -- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 msgr2=0x7fe0cc10e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d279c700 1 -- 172.21.15.26:0/1649136429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80095f0 con 0x7fe0cc101d80 2023-10-06T21:20:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.845+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:10.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.846+0000 7f9077fff700 1 -- 172.21.15.26:0/1832234472 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084011830 con 0x7f9090073b70 2023-10-06T21:20:10.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.846+0000 7f9077fff700 1 -- 172.21.15.26:0/1832234472 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9084011e50 con 0x7f9090073b70 2023-10-06T21:20:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.846+0000 7fe0d279c700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc1a9570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4007f10 tx=0x7fe0c400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.846+0000 7f9077fff700 1 -- 172.21.15.26:0/1832234472 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084010d20 con 0x7f9090073b70 2023-10-06T21:20:10.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.846+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909019e210 con 0x7f9090073b70 2023-10-06T21:20:10.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.846+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901aa510 con 0x7f9090073b70 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.846+0000 7fe0c2ffd700 1 -- 172.21.15.26:0/1649136429 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4004db0 con 0x7fe0cc101d80 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.846+0000 7fe0c2ffd700 1 -- 172.21.15.26:0/1649136429 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c400f040 con 0x7fe0cc101d80 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.846+0000 7fe0c2ffd700 1 -- 172.21.15.26:0/1649136429 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40059b0 con 0x7fe0cc101d80 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.847+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0cc1aa010 con 0x7fe0cc101d80 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.847+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc1aa4a0 con 0x7fe0cc101d80 2023-10-06T21:20:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.853+0000 7fe0e7aefb80 1 Processor -- start 2023-10-06T21:20:10.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.852+0000 7f90a8bb7b80 1 Processor -- start 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.852+0000 7f90a8bb7b80 1 -- start start 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976a9b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b9e0 0x562976ab7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976a04460 con 0x562976a9b9e0 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629769f5c20 con 0x562976a9b0c0 2023-10-06T21:20:10.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.853+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629769f3540 con 0x562976ab78a0 2023-10-06T21:20:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 -- start start 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476325fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x5574763337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747628f400 con 0x557476325c00 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.854+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557476280bc0 con 0x5574763313d0 2023-10-06T21:20:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.855+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747627e4e0 con 0x557476326520 2023-10-06T21:20:10.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.859+0000 7f908e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.859+0000 7f908e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38906/0 (socket says 172.21.15.26:38906) 2023-10-06T21:20:10.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.859+0000 7f908e59c700 1 -- 172.21.15.26:0/931449243 learned_addr learned my addr 172.21.15.26:0/931449243 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.860+0000 7f9077fff700 1 -- 172.21.15.26:0/1832234472 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9084010380 con 0x7f9090073b70 2023-10-06T21:20:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.861+0000 7f908f59e700 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976a9b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.862+0000 7f9077fff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 0x7f90780a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.863+0000 7f9077fff700 1 -- 172.21.15.26:0/1832234472 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f90840cc910 con 0x7f9090073b70 2023-10-06T21:20:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.866+0000 7f908e59c700 1 -- 172.21.15.26:0/931449243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 msgr2=0x562976a9b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.866+0000 7f908e59c700 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976a9b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.866+0000 7f908e59c700 1 -- 172.21.15.26:0/931449243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b9e0 msgr2=0x562976ab7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.866+0000 7f908e59c700 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b9e0 0x562976ab7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.866+0000 7f908e59c700 1 -- 172.21.15.26:0/931449243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562976819de0 con 0x562976ab78a0 2023-10-06T21:20:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.856+0000 7fe0d179a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x5574763337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.856+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.856+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38902/0 (socket says 172.21.15.26:38902) 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.856+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/933934033 learned_addr learned my addr 172.21.15.26:0/933934033 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.861+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/933934033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 msgr2=0x5574763337c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.861+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x5574763337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.861+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/933934033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 msgr2=0x557476325fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.861+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476325fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.861+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/933934033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574760a4d80 con 0x557476326520 2023-10-06T21:20:10.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.873+0000 7f908f59e700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 0x7f90780a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.875+0000 7f908e59c700 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90840ca010 tx=0x7f9084010780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69016a3613fb2c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.879 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:10 smithi078 ceph-mon[134896]: pgmap v306: 97 pgs: 18 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 46 active+clean; 6.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2023-10-06T21:20:10.880 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2157176582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.869+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b580 tx=0x7fe0c80054b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7058c881e88fd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.884+0000 7fe0c2ffd700 1 -- 172.21.15.26:0/1649136429 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe0c4004f10 con 0x7fe0cc101d80 2023-10-06T21:20:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.885+0000 7fe0b7fff700 1 -- 172.21.15.26:0/933934033 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c801d070 con 0x557476326520 2023-10-06T21:20:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.885+0000 7fe0b7fff700 1 -- 172.21.15.26:0/933934033 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c801e040 con 0x557476326520 2023-10-06T21:20:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.885+0000 7fe0b7fff700 1 -- 172.21.15.26:0/933934033 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800f460 con 0x557476326520 2023-10-06T21:20:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.886+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/933934033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 msgr2=0x557476330e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.886+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800b580 tx=0x7fe0c80054b0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.886+0000 7f90757fa700 1 -- 172.21.15.26:0/931449243 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084095ca0 con 0x562976ab78a0 2023-10-06T21:20:10.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.886+0000 7f90757fa700 1 -- 172.21.15.26:0/931449243 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9084099730 con 0x562976ab78a0 2023-10-06T21:20:10.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.886+0000 7f90757fa700 1 -- 172.21.15.26:0/931449243 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084099e20 con 0x562976ab78a0 2023-10-06T21:20:10.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.890+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/931449243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 msgr2=0x562976ab9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.890+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90840ca010 tx=0x7f9084010780 comp rx=0 tx=0).stop 2023-10-06T21:20:10.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/931449243 shutdown_connections 2023-10-06T21:20:10.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab9c90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976a9b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/931449243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b9e0 0x562976ab7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/931449243 >> 172.21.15.26:0/931449243 conn(0x562976a10260 msgr2=0x562976a97450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.891+0000 7f908f59e700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 0x7f90780a3cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9088009fd0 tx=0x7f9088004be0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.902+0000 7fe0c2ffd700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 0x7fe0b80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0c2ffd700 1 -- 172.21.15.26:0/1649136429 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fe0c40cc920 con 0x7fe0cc101d80 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/933934033 shutdown_connections 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x5574763337c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476325fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/933934033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476330e90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.903+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/933934033 >> 172.21.15.26:0/933934033 conn(0x55747629b200 msgr2=0x557476322080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.902+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/931449243 shutdown_connections 2023-10-06T21:20:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.908+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/931449243 wait complete. 2023-10-06T21:20:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.908+0000 7f90a8bb7b80 1 -- start start 2023-10-06T21:20:10.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.909+0000 7f90a8bb7b80 1 Processor -- start 2023-10-06T21:20:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.910+0000 7fe0d179a700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 0x7fe0b80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976ab4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b9e0 0x562976ab4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629769eee30 con 0x562976ab78a0 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297698b710 con 0x562976a9b0c0 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297698b3b0 con 0x562976a9b9e0 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.910+0000 7f90a8bb7b80 10 client.0 _ll_register_callbacks cb 0x5629767e8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.911+0000 7f908f59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.911+0000 7f908f59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35678/0 (socket says 172.21.15.26:35678) 2023-10-06T21:20:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.911+0000 7f908f59e700 1 -- 172.21.15.26:0/1737918961 learned_addr learned my addr 172.21.15.26:0/1737918961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.913+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/933934033 shutdown_connections 2023-10-06T21:20:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.916+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/933934033 wait complete. 2023-10-06T21:20:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.916+0000 7fe0e7aefb80 1 -- start start 2023-10-06T21:20:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.917+0000 7fe0e7aefb80 1 Processor -- start 2023-10-06T21:20:10.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.912+0000 7f908ed9d700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976ab4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.912+0000 7f908e59c700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b9e0 0x562976ab4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.921+0000 7fe0d179a700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 0x7fe0b80a3bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc004790 tx=0x7fe0bc010430 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476330dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x55747632e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x55747632e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557476279dd0 con 0x557476325c00 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574762166b0 con 0x5574763313d0 2023-10-06T21:20:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.923+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557476216350 con 0x557476326520 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.924+0000 7fe0e7aefb80 10 client.0 _ll_register_callbacks cb 0x5574760737a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.924+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476330dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.924+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476330dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35686/0 (socket says 172.21.15.26:35686) 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.924+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/3199776964 learned_addr learned my addr 172.21.15.26:0/3199776964 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.925+0000 7fe0d179a700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x55747632e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d279c700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x55747632e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d179a700 1 -- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 msgr2=0x55747632e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d179a700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x55747632e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d179a700 1 -- 172.21.15.26:0/3199776964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 msgr2=0x557476330dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d179a700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476330dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.926+0000 7fe0d179a700 1 -- 172.21.15.26:0/3199776964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc1107c0 con 0x557476326520 2023-10-06T21:20:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.928+0000 7fe0d179a700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x55747632e050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc10fc40 tx=0x7fe0bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d98a0dbe86fd613 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.928+0000 7fe0b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.929+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc021070 con 0x557476326520 2023-10-06T21:20:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.929+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fe0bc01cbc0 con 0x557476326520 2023-10-06T21:20:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.929+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557476319d50 con 0x557476326520 2023-10-06T21:20:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.930+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0bc015070 con 0x557476326520 2023-10-06T21:20:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.930+0000 7fe0e7aefb80 10 client.26743 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.930+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55747608dc00 con 0x557476326520 2023-10-06T21:20:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.924+0000 7f908f59e700 1 -- 172.21.15.26:0/1737918961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b9e0 msgr2=0x562976ab4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.925+0000 7f908f59e700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b9e0 0x562976ab4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.925+0000 7f908f59e700 1 -- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 msgr2=0x562976ab4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.925+0000 7f908f59e700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976ab4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.925+0000 7f908f59e700 1 -- 172.21.15.26:0/1737918961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901aa510 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.926+0000 7f908f59e700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9088016970 tx=0x7f908800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbb24eb9bfe32d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.926+0000 7f9072ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.927+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088010030 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.927+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f908801e040 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.927+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562976a8ef10 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.927+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90880159e0 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.928+0000 7f90a8bb7b80 10 client.16545 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.928+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562976802c60 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562976a8ef10 con 0x562976ab78a0 2023-10-06T21:20:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f908802a070 con 0x562976ab78a0 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 10 client.16545 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 20 client.16545 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976ad83e0 0x562976ada8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 4 client.16545 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.929+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629769e7e10 con 0x562976ad83e0 2023-10-06T21:20:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557476319d50 con 0x557476326520 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe0bc00f3c0 con 0x557476326520 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 10 client.26743 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 10 client.26743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 20 client.26743 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557476362f40 0x557476365360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 4 client.26743 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557476272db0 con 0x557476362f40 2023-10-06T21:20:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.933+0000 7f908ed9d700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976ad83e0 0x562976ada8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.934+0000 7f908ed9d700 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976ad83e0 0x562976ada8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e81e58bca046602 server_cookie=7e9e963c40ba2d1d in_seq=0 out_seq=0 2023-10-06T21:20:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.934+0000 7f9072ff5700 10 client.16545 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.934+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557476362f40 0x557476365360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f9072ff5700 1 -- 172.21.15.26:0/1737918961 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629769e7e10 con 0x562976ad83e0 2023-10-06T21:20:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f9072ff5700 10 client.16545 handle_command_reply: tid=0 2023-10-06T21:20:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.935+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557476362f40 0x557476365360 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265fa98e8b6b995a server_cookie=ee1b3c4743bb87e5 in_seq=0 out_seq=0 2023-10-06T21:20:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.935+0000 7fe0b57fa700 10 client.26743 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 client.16545 shutdown 2023-10-06T21:20:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 msgr2=0x562976ab51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9088016970 tx=0x7f908800d040 comp rx=0 tx=0).stop 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 shutdown_connections 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976ad83e0 0x562976ada8a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976ab51a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b0c0 0x562976ab4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/1737918961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b9e0 0x562976ab4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 >> 172.21.15.26:0/1737918961 conn(0x562976a10260 msgr2=0x562976a97040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.935+0000 7f9072ff5700 0 client.16545 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.935+0000 7fe0b57fa700 1 -- 172.21.15.26:0/3199776964 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557476272db0 con 0x557476362f40 2023-10-06T21:20:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0b57fa700 10 client.26743 handle_command_reply: tid=0 2023-10-06T21:20:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 client.26743 shutdown 2023-10-06T21:20:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.936+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 shutdown_connections 2023-10-06T21:20:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.936+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/1737918961 wait complete. 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 msgr2=0x55747632e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x55747632e050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc10fc40 tx=0x7fe0bc009260 comp rx=0 tx=0).stop 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 shutdown_connections 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557476362f40 0x557476365360 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x55747632e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476330dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3199776964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x55747632e050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 >> 172.21.15.26:0/3199776964 conn(0x55747629b200 msgr2=0x557476321c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0b57fa700 0 client.26743 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.936+0000 7f90a8bb7b80 20 client.16545 trim_cache size 0 max 16384 2023-10-06T21:20:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 shutdown_connections 2023-10-06T21:20:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.936+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3199776964 wait complete. 2023-10-06T21:20:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.937+0000 7fe0e7aefb80 20 client.26743 trim_cache size 0 max 16384 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.952+0000 7f90a8bb7b80 1 Processor -- start 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.952+0000 7f90a8bb7b80 1 -- start start 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.952+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 0x562976abe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976ab4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297686f920 con 0x562976a9b0c0 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976ad5a80 con 0x562976a9b9e0 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976abe8c0 con 0x562976ab78a0 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f908f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f908f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38926/0 (socket says 172.21.15.26:38926) 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f908f59e700 1 -- 172.21.15.26:0/2498820232 learned_addr learned my addr 172.21.15.26:0/2498820232 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.953+0000 7f908ed9d700 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 0x562976abe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 -- 172.21.15.26:0/2498820232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 msgr2=0x562976ab19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 -- 172.21.15.26:0/2498820232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 msgr2=0x562976ab4fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976ab4fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 -- 172.21.15.26:0/2498820232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562976840250 con 0x562976a9b0c0 2023-10-06T21:20:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.954+0000 7fe0e7aefb80 1 Processor -- start 2023-10-06T21:20:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.954+0000 7f908ed9d700 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 0x562976abe380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x562976adb1c0 tx=0x7f9080019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf620b49be5877d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.954+0000 7fe0e7aefb80 1 -- start start 2023-10-06T21:20:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476325c00 0x557476365330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557476326520 0x557476328aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574760fa8c0 con 0x5574763313d0 2023-10-06T21:20:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.955+0000 7f9075ffb700 1 -- 172.21.15.26:0/2498820232 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908001d040 con 0x562976a9b0c0 2023-10-06T21:20:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747631e8d0 con 0x557476326520 2023-10-06T21:20:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557476322e40 con 0x557476325c00 2023-10-06T21:20:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.955+0000 7f9075ffb700 1 -- 172.21.15.26:0/2498820232 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9080029070 con 0x562976a9b0c0 2023-10-06T21:20:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.955+0000 7f9075ffb700 1 -- 172.21.15.26:0/2498820232 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90800255b0 con 0x562976a9b0c0 2023-10-06T21:20:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0d279c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0d279c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35698/0 (socket says 172.21.15.26:35698) 2023-10-06T21:20:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.955+0000 7fe0d279c700 1 -- 172.21.15.26:0/3893482242 learned_addr learned my addr 172.21.15.26:0/3893482242 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.956+0000 7fe0d279c700 1 -- 172.21.15.26:0/3893482242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476325c00 msgr2=0x557476365330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.956+0000 7fe0d279c700 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476325c00 0x557476365330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.956+0000 7fe0d279c700 1 -- 172.21.15.26:0/3893482242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557476326520 msgr2=0x557476328aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:20:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.956+0000 7fe0d279c700 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557476326520 0x557476328aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.956+0000 7fe0d279c700 1 -- 172.21.15.26:0/3893482242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574760cb1f0 con 0x5574763313d0 2023-10-06T21:20:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.956+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/2498820232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 msgr2=0x562976abe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.956+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 0x562976abe380 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x562976adb1c0 tx=0x7f9080019040 comp rx=0 tx=0).stop 2023-10-06T21:20:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.957+0000 7fe0d279c700 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4095d60 tx=0x7fe0c4099630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9420052eb3eaf012 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/2498820232 shutdown_connections 2023-10-06T21:20:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976ab78a0 0x562976ab19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976a9b0c0 0x562976abe380 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/2498820232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976ab4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/2498820232 >> 172.21.15.26:0/2498820232 conn(0x562976a10260 msgr2=0x562976aafaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.957+0000 7fe0c0ff9700 1 -- 172.21.15.26:0/3893482242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4099f60 con 0x5574763313d0 2023-10-06T21:20:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.957+0000 7fe0c0ff9700 1 -- 172.21.15.26:0/3893482242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c4099860 con 0x5574763313d0 2023-10-06T21:20:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.957+0000 7fe0c0ff9700 1 -- 172.21.15.26:0/3893482242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40840c0 con 0x5574763313d0 2023-10-06T21:20:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/2498820232 shutdown_connections 2023-10-06T21:20:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.957+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/2498820232 wait complete. 2023-10-06T21:20:10.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.958+0000 7f90a8bb7b80 1 -- start start 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3893482242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 msgr2=0x55747632d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4095d60 tx=0x7fe0c4099630 comp rx=0 tx=0).stop 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3893482242 shutdown_connections 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5574763313d0 0x55747632d890 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476325c00 0x557476365330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/3893482242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557476326520 0x557476328aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3893482242 >> 172.21.15.26:0/3893482242 conn(0x55747629b200 msgr2=0x5574763293d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.958+0000 7f90a8bb7b80 1 Processor -- start 2023-10-06T21:20:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3893482242 shutdown_connections 2023-10-06T21:20:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.958+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/3893482242 wait complete. 2023-10-06T21:20:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.959+0000 7fe0e7aefb80 1 -- start start 2023-10-06T21:20:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976abc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976b2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976b2b830 con 0x562976ab78a0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976b2b970 con 0x562976a9b9e0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562976b2bae0 con 0x562976a9b0c0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f90a8bb7b80 10 client.0 _ll_register_callbacks cb 0x7f908800af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f908ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f908ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38942/0 (socket says 172.21.15.26:38942) 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f908e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976abc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f908ed9d700 1 -- 172.21.15.26:0/4148586577 learned_addr learned my addr 172.21.15.26:0/4148586577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.959+0000 7fe0e7aefb80 1 Processor -- start 2023-10-06T21:20:10.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.959+0000 7f908f59e700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976b2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 -- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 msgr2=0x562976abc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976abc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 -- 172.21.15.26:0/4148586577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 msgr2=0x562976b2b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976b2b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 -- 172.21.15.26:0/4148586577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9090114bf0 con 0x562976a9b0c0 2023-10-06T21:20:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476348ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476347310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x557476347850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f908ed9d700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f908000bc30 tx=0x7f9080025230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1daf63fb552e03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.960+0000 7f9072ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557476347f60 con 0x557476325c00 2023-10-06T21:20:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574763480d0 con 0x5574763313d0 2023-10-06T21:20:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574763b6d80 con 0x557476326520 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0e7aefb80 10 client.0 _ll_register_callbacks cb 0x7fe0bc010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0d179a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476347310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476348ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0d279c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x557476347850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0d1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476348ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35718/0 (socket says 172.21.15.26:35718) 2023-10-06T21:20:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.960+0000 7fe0d1f9b700 1 -- 172.21.15.26:0/2130669581 learned_addr learned my addr 172.21.15.26:0/2130669581 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.961+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908001d070 con 0x562976a9b0c0 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.961+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f90800259f0 con 0x562976a9b0c0 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.961+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9084099730 con 0x562976a9b0c0 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.962+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90800255a0 con 0x562976a9b0c0 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.963+0000 7f90a8bb7b80 10 client.26749 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:10.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.963+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5629768182b0 con 0x562976a9b0c0 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.961+0000 7fe0d279c700 1 -- 172.21.15.26:0/2130669581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 msgr2=0x557476347310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.961+0000 7fe0d279c700 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476347310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.961+0000 7fe0d279c700 1 -- 172.21.15.26:0/2130669581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 msgr2=0x557476348ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.961+0000 7fe0d279c700 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476348ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.961+0000 7fe0d279c700 1 -- 172.21.15.26:0/2130669581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc1aa4a0 con 0x5574763313d0 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.962+0000 7fe0d279c700 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x557476347850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe0c409db70 tx=0x7fe0c4097e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=430f00430f79a7c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.962+0000 7fe0b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.962+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4099cc0 con 0x5574763313d0 2023-10-06T21:20:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.962+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fe0c4082e40 con 0x5574763313d0 2023-10-06T21:20:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.962+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0c801e040 con 0x5574763313d0 2023-10-06T21:20:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.963+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c40ca460 con 0x5574763313d0 2023-10-06T21:20:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.963+0000 7fe0e7aefb80 10 client.26864 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.963+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5574760a3250 con 0x5574763313d0 2023-10-06T21:20:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9084099730 con 0x562976a9b0c0 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9080005410 con 0x562976a9b0c0 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 10 client.26749 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 10 client.26749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 20 client.26749 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976b2ce30 0x562976b2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 4 client.26749 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.964+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629769e7b30 con 0x562976b2ce30 2023-10-06T21:20:10.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.965+0000 7f908e59c700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976b2ce30 0x562976b2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c407d080 con 0x5574763313d0 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe0c409ab90 con 0x5574763313d0 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 10 client.26864 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 20 client.26864 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5574763b81a0 0x5574763ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 4 client.26864 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:20:10.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.965+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557476272ad0 con 0x5574763b81a0 2023-10-06T21:20:10.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.966+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5574763b81a0 0x5574763ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.966+0000 7f908e59c700 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976b2ce30 0x562976b2f250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc4c0e68595f78a3 server_cookie=eb73c305cc45f3bd in_seq=0 out_seq=0 2023-10-06T21:20:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.966+0000 7f9072ff5700 10 client.26749 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.966+0000 7f9072ff5700 1 -- 172.21.15.26:0/4148586577 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629769e7b30 con 0x562976b2ce30 2023-10-06T21:20:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.966+0000 7f9072ff5700 10 client.26749 handle_command_reply: tid=0 2023-10-06T21:20:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.966+0000 7f90a8bb7b80 1 client.26749 shutdown 2023-10-06T21:20:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.967+0000 7fe0d1f9b700 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5574763b81a0 0x5574763ba5c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9674643f0d8504 server_cookie=a0331c86f661c3e4 in_seq=0 out_seq=0 2023-10-06T21:20:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.967+0000 7fe0b57fa700 10 client.26864 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:10.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 msgr2=0x562976abbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f908000bc30 tx=0x7f9080025230 comp rx=0 tx=0).stop 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 shutdown_connections 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x562976b2ce30 0x562976b2f250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562976ab78a0 0x562976b2b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562976a9b0c0 0x562976abbf50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 --2- 172.21.15.26:0/4148586577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562976a9b9e0 0x562976abc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f9072ff5700 0 client.26749 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 >> 172.21.15.26:0/4148586577 conn(0x562976a10260 msgr2=0x562976ab4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.969+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 shutdown_connections 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90a8bb7b80 1 -- 172.21.15.26:0/4148586577 wait complete. 2023-10-06T21:20:10.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90a8bb7b80 20 client.26749 trim_cache size 0 max 16384 2023-10-06T21:20:10.980 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:10.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 msgr2=0x7f90780a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 0x7f90780a3cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9088009fd0 tx=0x7f9088004be0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 msgr2=0x7f90901a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.970+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f908400b730 comp rx=0 tx=0).stop 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 shutdown_connections 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9090111e50 0x7f909019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f90780a1800 0x7f90780a3cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9090073250 0x7f90901a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 --2- 172.21.15.26:0/1832234472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9090073b70 0x7f90901a3a60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 >> 172.21.15.26:0/1832234472 conn(0x7f90900fc950 msgr2=0x7f90900fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 shutdown_connections 2023-10-06T21:20:10.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:10.971+0000 7f90951ff700 1 -- 172.21.15.26:0/1832234472 wait complete. 2023-10-06T21:20:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.993+0000 7fe0b57fa700 1 -- 172.21.15.26:0/2130669581 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557476272ad0 con 0x5574763b81a0 2023-10-06T21:20:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.993+0000 7fe0b57fa700 10 client.26864 handle_command_reply: tid=0 2023-10-06T21:20:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.993+0000 7fe0e7aefb80 1 client.26864 shutdown 2023-10-06T21:20:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.994+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 msgr2=0x557476347850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.994+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x557476347850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe0c409db70 tx=0x7fe0c4097e10 comp rx=0 tx=0).stop 2023-10-06T21:20:10.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 shutdown_connections 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5574763b81a0 0x5574763ba5c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574763313d0 0x557476347850 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557476325c00 0x557476348ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 --2- 172.21.15.26:0/2130669581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557476326520 0x557476347310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 >> 172.21.15.26:0/2130669581 conn(0x55747629b200 msgr2=0x55747632f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0b57fa700 0 client.26864 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 shutdown_connections 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 1 -- 172.21.15.26:0/2130669581 wait complete. 2023-10-06T21:20:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.996+0000 7fe0e7aefb80 20 client.26864 trim_cache size 0 max 16384 2023-10-06T21:20:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:20:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:20:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:20:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 msgr2=0x7fe0b80a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 0x7fe0b80a3bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc004790 tx=0x7fe0bc010430 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 msgr2=0x7fe0cc1a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc1a9570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4007f10 tx=0x7fe0c400d3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 shutdown_connections 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0cc101d80 0x7fe0cc1a9570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe0b80a1730 0x7fe0b80a3bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0cc0ff6e0 0x7fe0cc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 --2- 172.21.15.26:0/1649136429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc1000a0 0x7fe0cc10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 >> 172.21.15.26:0/1649136429 conn(0x7fe0cc0762e0 msgr2=0x7fe0cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 shutdown_connections 2023-10-06T21:20:11.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:10.997+0000 7fe0d41ff700 1 -- 172.21.15.26:0/1649136429 wait complete. 2023-10-06T21:20:11.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:20:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.369+0000 7fa4f19ff700 1 -- 172.21.15.26:0/1985399838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec114000 msgr2=0x7fa4ec1163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.369+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/1985399838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec114000 0x7fa4ec1163f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400b0a0 tx=0x7fa4e400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 -- 172.21.15.26:0/1985399838 shutdown_connections 2023-10-06T21:20:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/1985399838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec114000 0x7fa4ec1163f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/1985399838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/1985399838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec1087d0 0x7fa4ec113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 -- 172.21.15.26:0/1985399838 >> 172.21.15.26:0/1985399838 conn(0x7fa4ec06d1a0 msgr2=0x7fa4ec06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 -- 172.21.15.26:0/1985399838 shutdown_connections 2023-10-06T21:20:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.370+0000 7fa4f19ff700 1 -- 172.21.15.26:0/1985399838 wait complete. 2023-10-06T21:20:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 Processor -- start 2023-10-06T21:20:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 -- start start 2023-10-06T21:20:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec1087d0 0x7fa4ec1a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec114000 0x7fa4ec1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1a1ff0 con 0x7fa4ec1087d0 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1a2160 con 0x7fa4ec107eb0 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.372+0000 7fa4f19ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec1a2320 con 0x7fa4ec114000 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52502/0 (socket says 172.21.15.26:52502) 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2026960297 learned_addr learned my addr 172.21.15.26:0/2026960297 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec1087d0 0x7fa4ec1a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec114000 0x7fa4ec1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2026960297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec114000 msgr2=0x7fa4ec1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec114000 0x7fa4ec1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec1087d0 msgr2=0x7fa4ec1a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec1087d0 0x7fa4ec1a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2026960297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4009d00 con 0x7fa4ec107eb0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e9a0 tx=0x7fa4dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4f09fd700 1 -- 172.21.15.26:0/2026960297 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc009790 con 0x7fa4ec107eb0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4f09fd700 1 -- 172.21.15.26:0/2026960297 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4dc010430 con 0x7fa4ec107eb0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.373+0000 7fa4f09fd700 1 -- 172.21.15.26:0/2026960297 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc018680 con 0x7fa4ec107eb0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.374+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec1a2600 con 0x7fa4ec107eb0 2023-10-06T21:20:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.374+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec1ae900 con 0x7fa4ec107eb0 2023-10-06T21:20:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.381+0000 7fa4f09fd700 1 -- 172.21.15.26:0/2026960297 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa4dc0105a0 con 0x7fa4ec107eb0 2023-10-06T21:20:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa505227b80 1 Processor -- start 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa4f09fd700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 0x7fa4d40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa505227b80 1 -- start start 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa4f09fd700 1 -- 172.21.15.26:0/2026960297 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa4dc014070 con 0x7fa4ec107eb0 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 0x7fa4d40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.382+0000 7fa505227b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa505227b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa505227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa697ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa505227b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa68da460 con 0x560fa69710c0 2023-10-06T21:20:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa505227b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa68cbc20 con 0x560fa69719e0 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa505227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa68c9540 con 0x560fa697c890 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52516/0 (socket says 172.21.15.26:52516) 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1064765488 learned_addr learned my addr 172.21.15.26:0/1064765488 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa697ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1064765488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 msgr2=0x560fa697ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa697ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1064765488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 msgr2=0x560fa69714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.383+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1064765488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fa66efde0 con 0x560fa69719e0 2023-10-06T21:20:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.385+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 0x7fa4d40a3ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1a8610 tx=0x7fa4e400bd80 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.386+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009ca0 tx=0x7fa4e0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adec1e3d19bcb439 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.386+0000 7fa4d1ffb700 1 -- 172.21.15.26:0/1064765488 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0011d70 con 0x560fa69719e0 2023-10-06T21:20:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.386+0000 7fa4d1ffb700 1 -- 172.21.15.26:0/1064765488 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4e00198f0 con 0x560fa69719e0 2023-10-06T21:20:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.386+0000 7fa4d1ffb700 1 -- 172.21.15.26:0/1064765488 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000e850 con 0x560fa69719e0 2023-10-06T21:20:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 -- 172.21.15.26:0/1064765488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 msgr2=0x560fa697c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009ca0 tx=0x7fa4e0011440 comp rx=0 tx=0).stop 2023-10-06T21:20:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 -- 172.21.15.26:0/1064765488 shutdown_connections 2023-10-06T21:20:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa697ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 --2- 172.21.15.26:0/1064765488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa697c350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 -- 172.21.15.26:0/1064765488 >> 172.21.15.26:0/1064765488 conn(0x560fa68e6260 msgr2=0x560fa696d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 -- 172.21.15.26:0/1064765488 shutdown_connections 2023-10-06T21:20:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.387+0000 7fa505227b80 1 -- 172.21.15.26:0/1064765488 wait complete. 2023-10-06T21:20:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 -- start start 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 Processor -- start 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa69719e0 0x560fa6979c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa68c4e30 con 0x560fa69710c0 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa6861710 con 0x560fa697c890 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa68613b0 con 0x560fa69719e0 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.388+0000 7fa505227b80 10 client.0 _ll_register_callbacks cb 0x560fa66be800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52532/0 (socket says 172.21.15.26:52532) 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 -- 172.21.15.26:0/142460989 learned_addr learned my addr 172.21.15.26:0/142460989 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa69719e0 0x560fa6979c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 -- 172.21.15.26:0/142460989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa69719e0 msgr2=0x560fa6979c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa69719e0 0x560fa6979c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 -- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 msgr2=0x560fa69796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69796d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 -- 172.21.15.26:0/142460989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e0009900 con 0x560fa697c890 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d010 tx=0x7fa4e400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54bb702868be4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.389+0000 7fa4caffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400e040 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fa6964f10 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa4e4021cd0 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4e4021e90 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa505227b80 10 client.26876 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560fa66d8c60 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4e4036020 con 0x560fa697c890 2023-10-06T21:20:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.390+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa4e4005040 con 0x560fa697c890 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 10 client.26876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 20 client.26876 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa698c3d0 0x560fa698e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 4 client.26876 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.392+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fa68bde10 con 0x560fa698c3d0 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.393+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa698c3d0 0x560fa698e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.394+0000 7fa4caffd700 10 client.26876 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:11.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.394+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa698c3d0 0x560fa698e7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50e2ca616f1fbe40 server_cookie=611e4d331445159b in_seq=0 out_seq=0 2023-10-06T21:20:11.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa4caffd700 1 -- 172.21.15.26:0/142460989 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560fa68bde10 con 0x560fa698c3d0 2023-10-06T21:20:11.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa4caffd700 10 client.26876 handle_command_reply: tid=0 2023-10-06T21:20:11.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 client.26876 shutdown 2023-10-06T21:20:11.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 msgr2=0x560fa697a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d010 tx=0x7fa4e400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 shutdown_connections 2023-10-06T21:20:11.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa697c890 0x560fa697a150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa698c3d0 0x560fa698e7f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 --2- 172.21.15.26:0/142460989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa69719e0 0x560fa6979c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 >> 172.21.15.26:0/142460989 conn(0x560fa68e6260 msgr2=0x560fa696cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa4caffd700 0 client.26876 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 shutdown_connections 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 1 -- 172.21.15.26:0/142460989 wait complete. 2023-10-06T21:20:11.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.395+0000 7fa505227b80 20 client.26876 trim_cache size 0 max 16384 2023-10-06T21:20:11.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 Processor -- start 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 -- start start 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa6745920 con 0x560fa69710c0 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa697aa90 con 0x560fa69719e0 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa505227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa696ee10 con 0x560fa697c890 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4eaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52534/0 (socket says 172.21.15.26:52534) 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4eb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39006/0 (socket says 172.21.15.26:39006) 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.413+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1488464264 learned_addr learned my addr 172.21.15.26:0/1488464264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1488464264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 msgr2=0x560fa69749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1488464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 msgr2=0x560fa69833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 -- 172.21.15.26:0/1488464264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fa6969140 con 0x560fa69719e0 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:11.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:11.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.414+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e0009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af641d56edf6c13f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.415+0000 7fa4d27fc700 1 -- 172.21.15.26:0/1488464264 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0011d70 con 0x560fa69719e0 2023-10-06T21:20:11.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.415+0000 7fa4d27fc700 1 -- 172.21.15.26:0/1488464264 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4e0019a60 con 0x560fa69719e0 2023-10-06T21:20:11.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.415+0000 7fa4d27fc700 1 -- 172.21.15.26:0/1488464264 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003680 con 0x560fa69719e0 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 -- 172.21.15.26:0/1488464264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 msgr2=0x560fa6979f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e0009380 comp rx=0 tx=0).stop 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 -- 172.21.15.26:0/1488464264 shutdown_connections 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69710c0 0x560fa69833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 --2- 172.21.15.26:0/1488464264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69719e0 0x560fa6979f80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.416+0000 7fa505227b80 1 -- 172.21.15.26:0/1488464264 >> 172.21.15.26:0/1488464264 conn(0x560fa68e6260 msgr2=0x560fa698e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.417+0000 7fa505227b80 1 -- 172.21.15.26:0/1488464264 shutdown_connections 2023-10-06T21:20:11.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.417+0000 7fa505227b80 1 -- 172.21.15.26:0/1488464264 wait complete. 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.417+0000 7fa505227b80 1 -- start start 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.417+0000 7fa505227b80 1 Processor -- start 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 0x560fa6980f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa69df350 con 0x560fa69719e0 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa69df4c0 con 0x560fa69710c0 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa69df680 con 0x560fa697c890 2023-10-06T21:20:11.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa505227b80 10 client.0 _ll_register_callbacks cb 0x7fa4e4003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa4eaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa4eaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52546/0 (socket says 172.21.15.26:52546) 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2715068641 learned_addr learned my addr 172.21.15.26:0/2715068641 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa4eb7fe700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.418+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 0x560fa6980f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2715068641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 msgr2=0x560fa69814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 msgr2=0x560fa6980f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 0x560fa6980f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 -- 172.21.15.26:0/2715068641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc0cc040 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 0x560fa6980f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4eaffd700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c2d0 tx=0x7fa4dc096660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de557f2c04e8ea0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4caffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc098640 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4e00198f0 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fa4dc010880 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0b8340 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa505227b80 10 client.26882 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.419+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560fa66d8c60 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4e00198f0 con 0x560fa69710c0 2023-10-06T21:20:11.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa4dc0b8060 con 0x560fa69710c0 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 10 client.26882 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 10 client.26882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 20 client.26882 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa69e0dd0 0x560fa69e31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 4 client.26882 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fa68bdb30 con 0x560fa69e0dd0 2023-10-06T21:20:11.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.420+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa69e0dd0 0x560fa69e31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.421+0000 7fa4ea7fc700 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa69e0dd0 0x560fa69e31c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9029ca1d8e83e472 server_cookie=c801f54cd11a6b42 in_seq=0 out_seq=0 2023-10-06T21:20:11.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.422+0000 7fa4caffd700 10 client.26882 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:11.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.423+0000 7fa4caffd700 1 -- 172.21.15.26:0/2715068641 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fa68bdb30 con 0x560fa69e0dd0 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.423+0000 7fa4caffd700 10 client.26882 handle_command_reply: tid=0 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.423+0000 7fa505227b80 1 client.26882 shutdown 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 msgr2=0x560fa6980a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c2d0 tx=0x7fa4dc096660 comp rx=0 tx=0).stop 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 shutdown_connections 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x560fa69e0dd0 0x560fa69e31c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560fa697c890 0x560fa69814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fa69710c0 0x560fa6980a50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 --2- 172.21.15.26:0/2715068641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560fa69719e0 0x560fa6980f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 >> 172.21.15.26:0/2715068641 conn(0x560fa68e6260 msgr2=0x560fa69749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.424+0000 7fa4caffd700 0 client.26882 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.425+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 shutdown_connections 2023-10-06T21:20:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.425+0000 7fa505227b80 1 -- 172.21.15.26:0/2715068641 wait complete. 2023-10-06T21:20:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.425+0000 7fa505227b80 20 client.26882 trim_cache size 0 max 16384 2023-10-06T21:20:11.429 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 msgr2=0x7fa4d40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 0x7fa4d40a3ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec1a8610 tx=0x7fa4e400bd80 comp rx=0 tx=0).stop 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 msgr2=0x7fa4ec1a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e9a0 tx=0x7fa4dc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 shutdown_connections 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4ec114000 0x7fa4ec1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4ec107eb0 0x7fa4ec1a7910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4d40a15f0 0x7fa4d40a3ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 --2- 172.21.15.26:0/2026960297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4ec1087d0 0x7fa4ec1a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 >> 172.21.15.26:0/2026960297 conn(0x7fa4ec06d1a0 msgr2=0x7fa4ec109850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 shutdown_connections 2023-10-06T21:20:11.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.426+0000 7fa4f19ff700 1 -- 172.21.15.26:0/2026960297 wait complete. 2023-10-06T21:20:11.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:20:11.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.769+0000 7f70a99ff700 1 -- 172.21.15.26:0/4286538271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41050b0 msgr2=0x7f70a410cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.769+0000 7f70a99ff700 1 --2- 172.21.15.26:0/4286538271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41050b0 0x7f70a410cc30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.770+0000 7f70a99ff700 1 -- 172.21.15.26:0/4286538271 shutdown_connections 2023-10-06T21:20:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.770+0000 7f70a99ff700 1 --2- 172.21.15.26:0/4286538271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41050b0 0x7f70a410cc30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.770+0000 7f70a99ff700 1 --2- 172.21.15.26:0/4286538271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a40fff90 0x7f70a4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.770+0000 7f70a99ff700 1 --2- 172.21.15.26:0/4286538271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a4100940 0x7f70a4104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.770+0000 7f70a99ff700 1 -- 172.21.15.26:0/4286538271 >> 172.21.15.26:0/4286538271 conn(0x7f70a40fb830 msgr2=0x7f70a40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.771+0000 7f70a99ff700 1 -- 172.21.15.26:0/4286538271 shutdown_connections 2023-10-06T21:20:11.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.771+0000 7f70a99ff700 1 -- 172.21.15.26:0/4286538271 wait complete. 2023-10-06T21:20:11.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.772+0000 7f70a99ff700 1 Processor -- start 2023-10-06T21:20:11.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.772+0000 7f70a99ff700 1 -- start start 2023-10-06T21:20:11.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.772+0000 7f70a99ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.772+0000 7f70a99ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a41050b0 0x7f70a4101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.772+0000 7f70a99ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41024a0 0x7f70a41a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a99ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a59b0 con 0x7f70a41050b0 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a99ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a5b20 con 0x7f70a40fff90 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a99ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a5ce0 con 0x7f70a41024a0 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52558/0 (socket says 172.21.15.26:52558) 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 -- 172.21.15.26:0/581131628 learned_addr learned my addr 172.21.15.26:0/581131628 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a27fc700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a41050b0 0x7f70a4101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a37fe700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41024a0 0x7f70a41a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 -- 172.21.15.26:0/581131628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41024a0 msgr2=0x7f70a41a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41024a0 0x7f70a41a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 -- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a41050b0 msgr2=0x7f70a4101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a41050b0 0x7f70a4101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 -- 172.21.15.26:0/581131628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009d00 con 0x7f70a40fff90 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.773+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f709400c920 tx=0x7f709400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.774+0000 7f70a89fd700 1 -- 172.21.15.26:0/581131628 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094004a30 con 0x7f70a40fff90 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.774+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a41a5fc0 con 0x7f70a40fff90 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.774+0000 7f70a89fd700 1 -- 172.21.15.26:0/581131628 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70940054e0 con 0x7f70a40fff90 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.774+0000 7f70a89fd700 1 -- 172.21.15.26:0/581131628 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094007c10 con 0x7f70a40fff90 2023-10-06T21:20:11.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.774+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a41a6490 con 0x7f70a40fff90 2023-10-06T21:20:11.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.781+0000 7f70a89fd700 1 -- 172.21.15.26:0/581131628 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7094005000 con 0x7f70a40fff90 2023-10-06T21:20:11.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 Processor -- start 2023-10-06T21:20:11.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 -- start start 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf900a0 0x55c19cf92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cee5460 con 0x55c19cf900a0 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19ced6c20 con 0x55c19cf7c0c0 2023-10-06T21:20:11.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.783+0000 7f70bd274b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19ced4540 con 0x55c19cf7c9e0 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52570/0 (socket says 172.21.15.26:52570) 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 -- 172.21.15.26:0/1626354821 learned_addr learned my addr 172.21.15.26:0/1626354821 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a89fd700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 0x7f708c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 -- 172.21.15.26:0/1626354821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c9e0 msgr2=0x55c19cf8fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 -- 172.21.15.26:0/1626354821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf900a0 msgr2=0x55c19cf92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf900a0 0x55c19cf92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 -- 172.21.15.26:0/1626354821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c19ccfade0 con 0x55c19cf7c0c0 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a89fd700 1 -- 172.21.15.26:0/581131628 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f70940ccfc0 con 0x7f70a40fff90 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.784+0000 7f70a37fe700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 0x7f708c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.785+0000 7f70a37fe700 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c0092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e83b621786b396 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.785+0000 7f7089ffb700 1 -- 172.21.15.26:0/1626354821 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c00e040 con 0x55c19cf7c0c0 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.785+0000 7f7089ffb700 1 -- 172.21.15.26:0/1626354821 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c022070 con 0x55c19cf7c0c0 2023-10-06T21:20:11.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.785+0000 7f7089ffb700 1 -- 172.21.15.26:0/1626354821 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c005040 con 0x55c19cf7c0c0 2023-10-06T21:20:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 -- 172.21.15.26:0/1626354821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 msgr2=0x55c19cf7c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70a37fe700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 0x7f708c0a3b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f709c028000 tx=0x7f709c0053f0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c0092a0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 -- 172.21.15.26:0/1626354821 shutdown_connections 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf900a0 0x55c19cf92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c0c0 0x55c19cf7c4a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1626354821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 -- 172.21.15.26:0/1626354821 >> 172.21.15.26:0/1626354821 conn(0x55c19cef1260 msgr2=0x55c19cf78450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 -- 172.21.15.26:0/1626354821 shutdown_connections 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.786+0000 7f70bd274b80 1 -- 172.21.15.26:0/1626354821 wait complete. 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.787+0000 7f70bd274b80 1 -- start start 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.787+0000 7f70bd274b80 1 Processor -- start 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.787+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cecfe30 con 0x55c19cf7c9e0 2023-10-06T21:20:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19ce6c710 con 0x55c19cf900a0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19ce6c3b0 con 0x55c19cf7c0c0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70bd274b80 10 client.0 _ll_register_callbacks cb 0x55c19ccc9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39062/0 (socket says 172.21.15.26:39062) 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a2ffd700 1 -- 172.21.15.26:0/458470695 learned_addr learned my addr 172.21.15.26:0/458470695 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a27fc700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 -- 172.21.15.26:0/458470695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 msgr2=0x55c19cf8ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 -- 172.21.15.26:0/458470695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 msgr2=0x55c19cf8d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.788+0000 7f70a37fe700 1 -- 172.21.15.26:0/458470695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a410d4a0 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f70a37fe700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8d8c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f709c0061e0 tx=0x7f709c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec68efaa0903730 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f7082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c005040 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c19cf6ff10 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f709c01ecf0 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709c01e670 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f70bd274b80 10 client.26894 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c19cce3c60 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.789+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c03d080 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f709c0077a0 con 0x55c19cf900a0 2023-10-06T21:20:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 10 client.26894 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 20 client.26894 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cf9f910 0x55c19cfa1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 4 client.26894 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c19cec8e10 con 0x55c19cf9f910 2023-10-06T21:20:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.790+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cf9f910 0x55c19cfa1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.791+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cf9f910 0x55c19cfa1d30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7485a43811d6b34 server_cookie=3937603a60407e66 in_seq=0 out_seq=0 2023-10-06T21:20:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.792+0000 7f7082ffd700 10 client.26894 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.792+0000 7f7082ffd700 1 -- 172.21.15.26:0/458470695 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c19cec8e10 con 0x55c19cf9f910 2023-10-06T21:20:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.792+0000 7f7082ffd700 10 client.26894 handle_command_reply: tid=0 2023-10-06T21:20:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.793+0000 7f70bd274b80 1 client.26894 shutdown 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.793+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 msgr2=0x55c19cf8d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.793+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8d8c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f709c0061e0 tx=0x7f709c019000 comp rx=0 tx=0).stop 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 shutdown_connections 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cf9f910 0x55c19cfa1d30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8d8c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 --2- 172.21.15.26:0/458470695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 >> 172.21.15.26:0/458470695 conn(0x55c19cef1260 msgr2=0x55c19cf77fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f7082ffd700 0 client.26894 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 shutdown_connections 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 1 -- 172.21.15.26:0/458470695 wait complete. 2023-10-06T21:20:11.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.794+0000 7f70bd274b80 20 client.26894 trim_cache size 0 max 16384 2023-10-06T21:20:11.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 Processor -- start 2023-10-06T21:20:11.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 -- start start 2023-10-06T21:20:11.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cd50920 con 0x55c19cf7c9e0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cf6fae0 con 0x55c19cf900a0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70bd274b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cf77240 con 0x55c19cf7c0c0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.813+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39064/0 (socket says 172.21.15.26:39064) 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52586/0 (socket says 172.21.15.26:52586) 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 -- 172.21.15.26:0/1886298055 learned_addr learned my addr 172.21.15.26:0/1886298055 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a27fc700 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 -- 172.21.15.26:0/1886298055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 msgr2=0x55c19cf8a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 -- 172.21.15.26:0/1886298055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 msgr2=0x55c19cf8bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 -- 172.21.15.26:0/1886298055 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c19cf74140 con 0x55c19cf7c0c0 2023-10-06T21:20:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a37fe700 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7094099ef0 tx=0x7f70940cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=285828d946638668 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f708a7fc700 1 -- 172.21.15.26:0/1886298055 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709409e0b0 con 0x55c19cf7c0c0 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f708a7fc700 1 -- 172.21.15.26:0/1886298055 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70940a22f0 con 0x55c19cf7c0c0 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.814+0000 7f708a7fc700 1 -- 172.21.15.26:0/1886298055 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094007d70 con 0x55c19cf7c0c0 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 -- 172.21.15.26:0/1886298055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 msgr2=0x55c19cf8d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7094099ef0 tx=0x7f70940cf040 comp rx=0 tx=0).stop 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 -- 172.21.15.26:0/1886298055 shutdown_connections 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf900a0 0x55c19cf8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf7c0c0 0x55c19cf8d710 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 --2- 172.21.15.26:0/1886298055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c9e0 0x55c19cf8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 -- 172.21.15.26:0/1886298055 >> 172.21.15.26:0/1886298055 conn(0x55c19cef1260 msgr2=0x55c19cf8b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 -- 172.21.15.26:0/1886298055 shutdown_connections 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.815+0000 7f70bd274b80 1 -- 172.21.15.26:0/1886298055 wait complete. 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 -- start start 2023-10-06T21:20:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 Processor -- start 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 0x55c19cf94cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 0x55c19cf95230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cf95770 con 0x55c19cf7c0c0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cf958e0 con 0x55c19cf7c9e0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.816+0000 7f70bd274b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c19cf95a50 con 0x55c19cf900a0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70bd274b80 10 client.0 _ll_register_callbacks cb 0x7f709c009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 0x55c19cf94cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 0x55c19cf95230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35820/0 (socket says 172.21.15.26:35820) 2023-10-06T21:20:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a27fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 0x55c19cf94cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52588/0 (socket says 172.21.15.26:52588) 2023-10-06T21:20:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.817+0000 7f70a2ffd700 1 -- 172.21.15.26:0/103264524 learned_addr learned my addr 172.21.15.26:0/103264524 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.819+0000 7f70a37fe700 1 -- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 msgr2=0x55c19cf94cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.819+0000 7f70a37fe700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 0x55c19cf94cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.819+0000 7f70a37fe700 1 -- 172.21.15.26:0/103264524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 msgr2=0x55c19cf969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.819+0000 7f70a37fe700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.819+0000 7f70a37fe700 1 -- 172.21.15.26:0/103264524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c19cce3c60 con 0x55c19cf900a0 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.820+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.820+0000 7f70a37fe700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 0x55c19cf95230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f709c009150 tx=0x7f709c00afe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d77df0ddbe6f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:11.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.820+0000 7f7082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.820+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c040070 con 0x55c19cf900a0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.820+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f709c0076b0 con 0x55c19cf900a0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.821+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f709c022070 con 0x55c19cf900a0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.821+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709c007870 con 0x55c19cf900a0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.821+0000 7f70bd274b80 10 client.26782 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.821+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f70a41a6490 con 0x55c19cf900a0 2023-10-06T21:20:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.821+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c022070 con 0x55c19cf900a0 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f709c043020 con 0x55c19cf900a0 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 10 client.26782 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 10 client.26782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 20 client.26782 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cff4680 0x55c19cff6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:11.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 4 client.26782 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c19cec8b30 con 0x55c19cff4680 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.822+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cff4680 0x55c19cff6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.823+0000 7f70a2ffd700 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cff4680 0x55c19cff6a70 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513115fc1671a8b9 server_cookie=ba5caf05cc5d7616 in_seq=0 out_seq=0 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.823+0000 7f7082ffd700 10 client.26782 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.823+0000 7f7082ffd700 1 -- 172.21.15.26:0/103264524 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c19cec8b30 con 0x55c19cff4680 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.823+0000 7f7082ffd700 10 client.26782 handle_command_reply: tid=0 2023-10-06T21:20:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.823+0000 7f70bd274b80 1 client.26782 shutdown 2023-10-06T21:20:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 msgr2=0x55c19cf95230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 0x55c19cf95230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f709c009150 tx=0x7f709c00afe0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 shutdown_connections 2023-10-06T21:20:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c19cf900a0 0x55c19cf95230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c19cf7c0c0 0x55c19cf969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c19cff4680 0x55c19cff6a70 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 --2- 172.21.15.26:0/103264524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c19cf7c9e0 0x55c19cf94cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 >> 172.21.15.26:0/103264524 conn(0x55c19cef1260 msgr2=0x55c19cf8bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f7082ffd700 0 client.26782 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 shutdown_connections 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.824+0000 7f70bd274b80 1 -- 172.21.15.26:0/103264524 wait complete. 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70bd274b80 20 client.26782 trim_cache size 0 max 16384 2023-10-06T21:20:11.829 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 msgr2=0x7f708c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 0x7f708c0a3b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f709c028000 tx=0x7f709c0053f0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 msgr2=0x7f70a4101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f709400c920 tx=0x7f709400cc30 comp rx=0 tx=0).stop 2023-10-06T21:20:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 shutdown_connections 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a41024a0 0x7f70a41a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70a40fff90 0x7f70a4101a20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f708c0a1670 0x7f708c0a3b30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 --2- 172.21.15.26:0/581131628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a41050b0 0x7f70a4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.825+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 >> 172.21.15.26:0/581131628 conn(0x7f70a40fb830 msgr2=0x7f70a40680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.826+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 shutdown_connections 2023-10-06T21:20:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:11.826+0000 7f70a99ff700 1 -- 172.21.15.26:0/581131628 wait complete. 2023-10-06T21:20:12.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 570.0 s, 1289.9 tps, lat 77.357 ms stddev 84.849 2023-10-06T21:20:12.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:12 smithi167 ceph-mon[134494]: pgmap v307: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 958 op/s 2023-10-06T21:20:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:12 smithi026 ceph-mon[98490]: pgmap v307: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 958 op/s 2023-10-06T21:20:12.854 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:20:12.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:20:12.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:12 smithi078 ceph-mon[134896]: pgmap v307: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 14 MiB/s wr, 958 op/s 2023-10-06T21:20:13.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.210+0000 7f05883ff700 1 -- 172.21.15.26:0/487116654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0580104da0 msgr2=0x7f0580105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.210+0000 7f05883ff700 1 --2- 172.21.15.26:0/487116654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0580104da0 0x7f0580105180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f057c009990 tx=0x7f057c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 -- 172.21.15.26:0/487116654 shutdown_connections 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 --2- 172.21.15.26:0/487116654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580101280 0x7f058010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 --2- 172.21.15.26:0/487116654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0580104da0 0x7f0580105180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 --2- 172.21.15.26:0/487116654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05801008c0 0x7f0580100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 -- 172.21.15.26:0/487116654 >> 172.21.15.26:0/487116654 conn(0x7f0580076ed0 msgr2=0x7f05800fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 -- 172.21.15.26:0/487116654 shutdown_connections 2023-10-06T21:20:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.211+0000 7f05883ff700 1 -- 172.21.15.26:0/487116654 wait complete. 2023-10-06T21:20:13.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.212+0000 7f05883ff700 1 Processor -- start 2023-10-06T21:20:13.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.212+0000 7f05883ff700 1 -- start start 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0580101280 0x7f058019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 0x7f05801a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05801a2510 con 0x7f0580104da0 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05801a2680 con 0x7f05801008c0 2023-10-06T21:20:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f05883ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05801a2840 con 0x7f0580101280 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f058619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f058599a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0580101280 0x7f058019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f058619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52616/0 (socket says 172.21.15.26:52616) 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.213+0000 7f058619b700 1 -- 172.21.15.26:0/3251741023 learned_addr learned my addr 172.21.15.26:0/3251741023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058699c700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 0x7f05801a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 -- 172.21.15.26:0/3251741023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0580101280 msgr2=0x7f058019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0580101280 0x7f058019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 -- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 msgr2=0x7f05801a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 0x7f05801a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 -- 172.21.15.26:0/3251741023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0095f0 con 0x7f05801008c0 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058699c700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 0x7f05801a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f058619b700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f057c004730 tx=0x7f057c004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f05777fe700 1 -- 172.21.15.26:0/3251741023 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c01d070 con 0x7f05801008c0 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.214+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05801a2ac0 con 0x7f05801008c0 2023-10-06T21:20:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.215+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05801aa360 con 0x7f05801008c0 2023-10-06T21:20:13.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.215+0000 7f05777fe700 1 -- 172.21.15.26:0/3251741023 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057c004c90 con 0x7f05801008c0 2023-10-06T21:20:13.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.215+0000 7f05777fe700 1 -- 172.21.15.26:0/3251741023 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c005490 con 0x7f05801008c0 2023-10-06T21:20:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.219+0000 7f05777fe700 1 -- 172.21.15.26:0/3251741023 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f057c0047d0 con 0x7f05801008c0 2023-10-06T21:20:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.221+0000 7f059bca1b80 1 Processor -- start 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.221+0000 7f059bca1b80 1 -- start start 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a26a140 0x55900a26a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a275910 0x55900a277d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a1d34e0 con 0x55900a275910 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a1c4ca0 con 0x55900a26a140 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f059bca1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a1c25c0 con 0x55900a26aa60 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f058619b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f058619b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39082/0 (socket says 172.21.15.26:39082) 2023-10-06T21:20:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.222+0000 7f058619b700 1 -- 172.21.15.26:0/4026071131 learned_addr learned my addr 172.21.15.26:0/4026071131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058699c700 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a26a140 0x55900a26a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058619b700 1 -- 172.21.15.26:0/4026071131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a26a140 msgr2=0x55900a26a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058619b700 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a26a140 0x55900a26a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058619b700 1 -- 172.21.15.26:0/4026071131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a275910 msgr2=0x55900a277d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058619b700 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a275910 0x55900a277d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.223+0000 7f058619b700 1 -- 172.21.15.26:0/4026071131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559009fe8e60 con 0x55900a26aa60 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.224+0000 7f05777fe700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 0x7f056c0a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.225+0000 7f05777fe700 1 -- 172.21.15.26:0/3251741023 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f057c0ce2e0 con 0x7f05801008c0 2023-10-06T21:20:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.225+0000 7f058699c700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 0x7f056c0a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.226+0000 7f058699c700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 0x7f056c0a3ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0570009cf0 tx=0x7f0570009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.226+0000 7f058619b700 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c0cb900 tx=0x7f057c0d6540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29e2b1e31cc28ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.226+0000 7f0574bfe700 1 -- 172.21.15.26:0/4026071131 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c097620 con 0x55900a26aa60 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.226+0000 7f0574bfe700 1 -- 172.21.15.26:0/4026071131 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057c09b0b0 con 0x55900a26aa60 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.226+0000 7f0574bfe700 1 -- 172.21.15.26:0/4026071131 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c004e00 con 0x55900a26aa60 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.227+0000 7f059bca1b80 1 -- 172.21.15.26:0/4026071131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 msgr2=0x55900a2753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.227+0000 7f059bca1b80 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c0cb900 tx=0x7f057c0d6540 comp rx=0 tx=0).stop 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 -- 172.21.15.26:0/4026071131 shutdown_connections 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a275910 0x55900a277d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a26a140 0x55900a26a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 --2- 172.21.15.26:0/4026071131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a2753d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 -- 172.21.15.26:0/4026071131 >> 172.21.15.26:0/4026071131 conn(0x55900a1df2e0 msgr2=0x55900a2664d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 -- 172.21.15.26:0/4026071131 shutdown_connections 2023-10-06T21:20:13.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 -- 172.21.15.26:0/4026071131 wait complete. 2023-10-06T21:20:13.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.228+0000 7f059bca1b80 1 -- start start 2023-10-06T21:20:13.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.229+0000 7f059bca1b80 1 Processor -- start 2023-10-06T21:20:13.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.229+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a276230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a276770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a1bdeb0 con 0x55900a26a140 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a15a790 con 0x55900a275910 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a15a430 con 0x55900a26aa60 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f059bca1b80 10 client.0 _ll_register_callbacks cb 0x559009fb7880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35830/0 (socket says 172.21.15.26:35830) 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 -- 172.21.15.26:0/3978344157 learned_addr learned my addr 172.21.15.26:0/3978344157 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058699c700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a276770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058599a700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a276230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 -- 172.21.15.26:0/3978344157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 msgr2=0x55900a276230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a276230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 -- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 msgr2=0x55900a276770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a276770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058619b700 1 -- 172.21.15.26:0/3978344157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05801aa360 con 0x55900a26a140 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.230+0000 7f058699c700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a276770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f058619b700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f057c0a30b0 tx=0x7f057c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbc86d1d17f3e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f0569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0d1800 con 0x55900a26a140 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55900a25df90 con 0x55900a26a140 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f057c004e00 con 0x55900a26a140 2023-10-06T21:20:13.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c09ae20 con 0x55900a26a140 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f059bca1b80 10 client.16599 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559009fd1ce0 con 0x55900a26a140 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.231+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55900a25df90 con 0x55900a26a140 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f057c0cbc40 con 0x55900a26a140 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 20 client.16599 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a284cb0 0x55900a2870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 4 client.16599 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:13.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55900a1b6e90 con 0x55900a284cb0 2023-10-06T21:20:13.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.232+0000 7f058599a700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a284cb0 0x55900a2870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.233+0000 7f058599a700 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a284cb0 0x55900a2870d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f427a428dca92a server_cookie=3064a8b97f70670f in_seq=0 out_seq=0 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.233+0000 7f0569ffb700 10 client.16599 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f0569ffb700 1 -- 172.21.15.26:0/3978344157 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55900a1b6e90 con 0x55900a284cb0 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f0569ffb700 10 client.16599 handle_command_reply: tid=0 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 client.16599 shutdown 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 msgr2=0x55900a275cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f057c0a30b0 tx=0x7f057c005ef0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 shutdown_connections 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a284cb0 0x55900a2870d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a276770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a275cf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 --2- 172.21.15.26:0/3978344157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a276230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.234+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 >> 172.21.15.26:0/3978344157 conn(0x55900a1df2e0 msgr2=0x55900a266060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.235+0000 7f0569ffb700 0 client.16599 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.235+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 shutdown_connections 2023-10-06T21:20:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.235+0000 7f059bca1b80 1 -- 172.21.15.26:0/3978344157 wait complete. 2023-10-06T21:20:13.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.235+0000 7f059bca1b80 20 client.16599 trim_cache size 0 max 16384 2023-10-06T21:20:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 Processor -- start 2023-10-06T21:20:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 -- start start 2023-10-06T21:20:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a279d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a03e9a0 con 0x55900a26a140 2023-10-06T21:20:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a26b360 con 0x55900a275910 2023-10-06T21:20:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.260+0000 7f059bca1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a274dd0 con 0x55900a26aa60 2023-10-06T21:20:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39100/0 (socket says 172.21.15.26:39100) 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 -- 172.21.15.26:0/2761442551 learned_addr learned my addr 172.21.15.26:0/2761442551 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058699c700 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 -- 172.21.15.26:0/2761442551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 msgr2=0x55900a2720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 -- 172.21.15.26:0/2761442551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 msgr2=0x55900a279d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a279d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.261+0000 7f058599a700 1 -- 172.21.15.26:0/2761442551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55900a28bcc0 con 0x55900a26aa60 2023-10-06T21:20:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.262+0000 7f058599a700 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0578009d40 tx=0x7f0578004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda9c7c93452ac7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.262+0000 7f05753ff700 1 -- 172.21.15.26:0/2761442551 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801b070 con 0x55900a26aa60 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.262+0000 7f05753ff700 1 -- 172.21.15.26:0/2761442551 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057800f040 con 0x55900a26aa60 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.262+0000 7f05753ff700 1 -- 172.21.15.26:0/2761442551 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801cc00 con 0x55900a26aa60 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.263+0000 7f059bca1b80 1 -- 172.21.15.26:0/2761442551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 msgr2=0x55900a271b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.263+0000 7f059bca1b80 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0578009d40 tx=0x7f0578004b40 comp rx=0 tx=0).stop 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.263+0000 7f059bca1b80 1 -- 172.21.15.26:0/2761442551 shutdown_connections 2023-10-06T21:20:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.263+0000 7f059bca1b80 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.263+0000 7f059bca1b80 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26a140 0x55900a279d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.264+0000 7f059bca1b80 1 --2- 172.21.15.26:0/2761442551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26aa60 0x55900a271b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.264+0000 7f059bca1b80 1 -- 172.21.15.26:0/2761442551 >> 172.21.15.26:0/2761442551 conn(0x55900a1df2e0 msgr2=0x55900a26fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.264+0000 7f059bca1b80 1 -- 172.21.15.26:0/2761442551 shutdown_connections 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.264+0000 7f059bca1b80 1 -- 172.21.15.26:0/2761442551 wait complete. 2023-10-06T21:20:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.264+0000 7f059bca1b80 1 -- start start 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.265+0000 7f059bca1b80 1 Processor -- start 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.265+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 0x55900a2d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a2cd130 con 0x55900a26aa60 2023-10-06T21:20:13.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a2cd2a0 con 0x55900a275910 2023-10-06T21:20:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900a2cd460 con 0x55900a26a140 2023-10-06T21:20:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f059bca1b80 10 client.0 _ll_register_callbacks cb 0x7f057c0d60e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058619b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058599a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 0x55900a2d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52650/0 (socket says 172.21.15.26:52650) 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058619b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39104/0 (socket says 172.21.15.26:39104) 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058599a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 0x55900a2d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35834/0 (socket says 172.21.15.26:35834) 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 -- 172.21.15.26:0/1917606197 learned_addr learned my addr 172.21.15.26:0/1917606197 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 -- 172.21.15.26:0/1917606197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 msgr2=0x55900a2d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 -- 172.21.15.26:0/1917606197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 msgr2=0x55900a2d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 0x55900a2d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058699c700 1 -- 172.21.15.26:0/1917606197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0578009a20 con 0x55900a275910 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.266+0000 7f058619b700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f058699c700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0570011000 tx=0x7f0570014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500a87418642f622 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f0569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570022040 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f057c09b0b0 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f057001b660 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0570009ee0 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f059bca1b80 10 client.26918 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.267+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55900a26f770 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057c09b0b0 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0570025070 con 0x55900a275910 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 20 client.26918 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a2da040 0x55900a2dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 4 client.26918 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55900a1b6bb0 con 0x55900a2da040 2023-10-06T21:20:13.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.268+0000 7f058619b700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a2da040 0x55900a2dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.269+0000 7f058619b700 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a2da040 0x55900a2dc460 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0aaf9ccaf4330bb server_cookie=5892c3e234aae86d in_seq=0 out_seq=0 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.269+0000 7f0569ffb700 10 client.26918 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f0569ffb700 1 -- 172.21.15.26:0/1917606197 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55900a1b6bb0 con 0x55900a2da040 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f0569ffb700 10 client.26918 handle_command_reply: tid=0 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 client.26918 shutdown 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 msgr2=0x55900a2cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0570011000 tx=0x7f0570014040 comp rx=0 tx=0).stop 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 shutdown_connections 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55900a2da040 0x55900a2dc460 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55900a275910 0x55900a2cca30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900a26a140 0x55900a2d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 --2- 172.21.15.26:0/1917606197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55900a26aa60 0x55900a2d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 >> 172.21.15.26:0/1917606197 conn(0x55900a1df2e0 msgr2=0x55900a26c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f0569ffb700 0 client.26918 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.271+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 shutdown_connections 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f059bca1b80 1 -- 172.21.15.26:0/1917606197 wait complete. 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "2bc51b4d-6ed3-49e0-baac-60a9da1c43f9", 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:20:13.275 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f059bca1b80 20 client.26918 trim_cache size 0 max 16384 2023-10-06T21:20:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 msgr2=0x7f056c0a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 0x7f056c0a3ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0570009cf0 tx=0x7f0570009260 comp rx=0 tx=0).stop 2023-10-06T21:20:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 msgr2=0x7f058019d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.272+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f057c004730 tx=0x7f057c004760 comp rx=0 tx=0).stop 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 shutdown_connections 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f056c0a16e0 0x7f056c0a3ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0580104da0 0x7f05801a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05801008c0 0x7f058019d530 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 --2- 172.21.15.26:0/3251741023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0580101280 0x7f058019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 >> 172.21.15.26:0/3251741023 conn(0x7f0580076ed0 msgr2=0x7f0580103570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 shutdown_connections 2023-10-06T21:20:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.273+0000 7f05883ff700 1 -- 172.21.15.26:0/3251741023 wait complete. 2023-10-06T21:20:13.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 -- 172.21.15.26:0/2139670343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a41083f0 msgr2=0x7f94a4108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 --2- 172.21.15.26:0/2139670343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a41083f0 0x7f94a4108870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9494009990 tx=0x7f9494009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 -- 172.21.15.26:0/2139670343 shutdown_connections 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 --2- 172.21.15.26:0/2139670343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a411a690 0x7f94a411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 --2- 172.21.15.26:0/2139670343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a410f640 0x7f94a4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 --2- 172.21.15.26:0/2139670343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a41083f0 0x7f94a4108870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 -- 172.21.15.26:0/2139670343 >> 172.21.15.26:0/2139670343 conn(0x7f94a406d1a0 msgr2=0x7f94a406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 -- 172.21.15.26:0/2139670343 shutdown_connections 2023-10-06T21:20:13.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.685+0000 7f94a89ff700 1 -- 172.21.15.26:0/2139670343 wait complete. 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 Processor -- start 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 -- start start 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a41083f0 0x7f94a41a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a410f640 0x7f94a41a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41aaed0 con 0x7f94a410f640 2023-10-06T21:20:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41ab040 con 0x7f94a411a690 2023-10-06T21:20:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.687+0000 7f94a89ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41ab200 con 0x7f94a41083f0 2023-10-06T21:20:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.688+0000 7f94a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.688+0000 7f94a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52670/0 (socket says 172.21.15.26:52670) 2023-10-06T21:20:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.688+0000 7f94a2d9d700 1 -- 172.21.15.26:0/816130388 learned_addr learned my addr 172.21.15.26:0/816130388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f9499d9b700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a410f640 0x7f94a41a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 -- 172.21.15.26:0/816130388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a41083f0 msgr2=0x7f94a41a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a41083f0 0x7f94a41a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 -- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a410f640 msgr2=0x7f94a41a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a410f640 0x7f94a41a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 -- 172.21.15.26:0/816130388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940095f0 con 0x7f94a411a690 2023-10-06T21:20:13.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.689+0000 7f94a2d9d700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b9e0 tx=0x7f949c00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.690+0000 7f949bfff700 1 -- 172.21.15.26:0/816130388 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00c930 con 0x7f94a411a690 2023-10-06T21:20:13.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.690+0000 7f949bfff700 1 -- 172.21.15.26:0/816130388 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f949c014440 con 0x7f94a411a690 2023-10-06T21:20:13.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.690+0000 7f949bfff700 1 -- 172.21.15.26:0/816130388 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0125a0 con 0x7f94a411a690 2023-10-06T21:20:13.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.690+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a41ab4e0 con 0x7f94a411a690 2023-10-06T21:20:13.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.690+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a41b2d20 con 0x7f94a411a690 2023-10-06T21:20:13.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.697+0000 7f94bc40ab80 1 Processor -- start 2023-10-06T21:20:13.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.698+0000 7f949bfff700 1 -- 172.21.15.26:0/816130388 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f949c0145b0 con 0x7f94a411a690 2023-10-06T21:20:13.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.698+0000 7f94bc40ab80 1 -- start start 2023-10-06T21:20:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.698+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e23f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e24cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e1a8460 con 0x560a4e24a890 2023-10-06T21:20:13.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e199c20 con 0x560a4e23f0c0 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e197540 con 0x560a4e23f9e0 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94a259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94a259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39128/0 (socket says 172.21.15.26:39128) 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e23f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f94a259c700 1 -- 172.21.15.26:0/1695663322 learned_addr learned my addr 172.21.15.26:0/1695663322 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.699+0000 7f9499d9b700 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e24cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.700+0000 7f949bfff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 0x7f94900a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f949bfff700 1 -- 172.21.15.26:0/816130388 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f949c006e40 con 0x7f94a411a690 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f94a259c700 1 -- 172.21.15.26:0/1695663322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 msgr2=0x560a4e23f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f94a259c700 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e23f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f94a259c700 1 -- 172.21.15.26:0/1695663322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 msgr2=0x560a4e24cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f94a259c700 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e24cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.701+0000 7f94a259c700 1 -- 172.21.15.26:0/1695663322 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a4dfbdde0 con 0x560a4e23f9e0 2023-10-06T21:20:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.702+0000 7f94a2d9d700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 0x7f94900a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.703+0000 7f94a2d9d700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 0x7f94900a3c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f949c0c9ef0 tx=0x7f949c00cf90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.726+0000 7f94a259c700 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bb90 tx=0x7f948c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0e7d3ed24794257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.733+0000 7f9498d99700 1 -- 172.21.15.26:0/1695663322 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00d890 con 0x560a4e23f9e0 2023-10-06T21:20:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.733+0000 7f9498d99700 1 -- 172.21.15.26:0/1695663322 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f948c004830 con 0x560a4e23f9e0 2023-10-06T21:20:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.733+0000 7f9498d99700 1 -- 172.21.15.26:0/1695663322 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0052f0 con 0x560a4e23f9e0 2023-10-06T21:20:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 -- 172.21.15.26:0/1695663322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 msgr2=0x560a4e24a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bb90 tx=0x7f948c00bea0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 -- 172.21.15.26:0/1695663322 shutdown_connections 2023-10-06T21:20:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e24cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e23f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/1695663322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e24a350 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.734+0000 7f94bc40ab80 1 -- 172.21.15.26:0/1695663322 >> 172.21.15.26:0/1695663322 conn(0x560a4e1b4260 msgr2=0x560a4e23b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.735+0000 7f94bc40ab80 1 -- 172.21.15.26:0/1695663322 shutdown_connections 2023-10-06T21:20:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.735+0000 7f94bc40ab80 1 -- 172.21.15.26:0/1695663322 wait complete. 2023-10-06T21:20:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.735+0000 7f94bc40ab80 1 -- start start 2023-10-06T21:20:13.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 Processor -- start 2023-10-06T21:20:13.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e2475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e247b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e192e30 con 0x560a4e24a890 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e12f710 con 0x560a4e23f0c0 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e12f3b0 con 0x560a4e23f9e0 2023-10-06T21:20:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.736+0000 7f94bc40ab80 10 client.0 _ll_register_callbacks cb 0x560a4df8c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.738+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.738+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35874/0 (socket says 172.21.15.26:35874) 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.738+0000 7f9499d9b700 1 -- 172.21.15.26:0/958804463 learned_addr learned my addr 172.21.15.26:0/958804463 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.738+0000 7f94a259c700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e247b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.738+0000 7f9499d9b700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e2475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 -- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 msgr2=0x560a4e2475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e2475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 -- 172.21.15.26:0/958804463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 msgr2=0x560a4e248060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 -- 172.21.15.26:0/958804463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c00b7c0 con 0x560a4e23f9e0 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f9499d9b700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f94a259c700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e247b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bfd0 tx=0x7f948c004bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e87d81536d992d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.739+0000 7f9489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:13.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.740+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00daa0 con 0x560a4e23f9e0 2023-10-06T21:20:13.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.740+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f948c00dc00 con 0x560a4e23f9e0 2023-10-06T21:20:13.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.741+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a4e232f10 con 0x560a4e23f9e0 2023-10-06T21:20:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.741+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f948c00ddc0 con 0x560a4e23f9e0 2023-10-06T21:20:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.741+0000 7f94bc40ab80 10 client.26806 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.741+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560a4dfa6c60 con 0x560a4e23f9e0 2023-10-06T21:20:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a4e232f10 con 0x560a4e23f9e0 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f948c003680 con 0x560a4e23f9e0 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 10 client.26806 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 10 client.26806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 20 client.26806 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e25a190 0x560a4e25c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 4 client.26806 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.742+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a4e18be10 con 0x560a4e25a190 2023-10-06T21:20:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.743+0000 7f9499d9b700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e25a190 0x560a4e25c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.743+0000 7f9499d9b700 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e25a190 0x560a4e25c5b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8273241920c38b94 server_cookie=447cb0c31a802e37 in_seq=0 out_seq=0 2023-10-06T21:20:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.743+0000 7f9489ffb700 10 client.26806 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f9489ffb700 1 -- 172.21.15.26:0/958804463 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a4e18be10 con 0x560a4e25a190 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f9489ffb700 10 client.26806 handle_command_reply: tid=0 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 client.26806 shutdown 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 msgr2=0x560a4e247b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e247b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bfd0 tx=0x7f948c004bb0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 shutdown_connections 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e248060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e25a190 0x560a4e25c5b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e2475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/958804463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f9e0 0x560a4e247b20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 >> 172.21.15.26:0/958804463 conn(0x560a4e1b4260 msgr2=0x560a4e23afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f9489ffb700 0 client.26806 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.744+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 shutdown_connections 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.745+0000 7f94bc40ab80 1 -- 172.21.15.26:0/958804463 wait complete. 2023-10-06T21:20:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.745+0000 7f94bc40ab80 20 client.26806 trim_cache size 0 max 16384 2023-10-06T21:20:13.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.769+0000 7f94bc40ab80 1 Processor -- start 2023-10-06T21:20:13.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.769+0000 7f94bc40ab80 1 -- start start 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.769+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f0c0 0x560a4e2513d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e244850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e013920 con 0x560a4e24a890 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e256d50 con 0x560a4e23f9e0 2023-10-06T21:20:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e251910 con 0x560a4e23f0c0 2023-10-06T21:20:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.770+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f0c0 0x560a4e2513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52710/0 (socket says 172.21.15.26:52710) 2023-10-06T21:20:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 -- 172.21.15.26:0/4123887698 learned_addr learned my addr 172.21.15.26:0/4123887698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 -- 172.21.15.26:0/4123887698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f0c0 msgr2=0x560a4e2513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f0c0 0x560a4e2513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.771+0000 7f94a259c700 1 -- 172.21.15.26:0/4123887698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 msgr2=0x560a4e244850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.772+0000 7f94a259c700 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e244850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.772+0000 7f94a259c700 1 -- 172.21.15.26:0/4123887698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a4e237140 con 0x560a4e23f9e0 2023-10-06T21:20:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.772+0000 7f94a259c700 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f948c003a40 tx=0x7f948c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0e14228662eaa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.773+0000 7f949959a700 1 -- 172.21.15.26:0/4123887698 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c005ee0 con 0x560a4e23f9e0 2023-10-06T21:20:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.773+0000 7f949959a700 1 -- 172.21.15.26:0/4123887698 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f948c003680 con 0x560a4e23f9e0 2023-10-06T21:20:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.773+0000 7f949959a700 1 -- 172.21.15.26:0/4123887698 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00d670 con 0x560a4e23f9e0 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 -- 172.21.15.26:0/4123887698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 msgr2=0x560a4e247e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f948c003a40 tx=0x7f948c00bea0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 -- 172.21.15.26:0/4123887698 shutdown_connections 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e24a890 0x560a4e244850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e23f0c0 0x560a4e2513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/4123887698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f9e0 0x560a4e247e90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 -- 172.21.15.26:0/4123887698 >> 172.21.15.26:0/4123887698 conn(0x560a4e1b4260 msgr2=0x560a4e239bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 -- 172.21.15.26:0/4123887698 shutdown_connections 2023-10-06T21:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.774+0000 7f94bc40ab80 1 -- 172.21.15.26:0/4123887698 wait complete. 2023-10-06T21:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.775+0000 7f94bc40ab80 1 -- start start 2023-10-06T21:20:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.775+0000 7f94bc40ab80 1 Processor -- start 2023-10-06T21:20:13.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e23f9e0 0x560a4e24ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e24a890 0x560a4e24f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e2acfc0 con 0x560a4e23f9e0 2023-10-06T21:20:13.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e2ad100 con 0x560a4e23f0c0 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a4e2ad240 con 0x560a4e24a890 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f94bc40ab80 10 client.0 _ll_register_callbacks cb 0x7f9494009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f9499d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52718/0 (socket says 172.21.15.26:52718) 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f9499d9b700 1 -- 172.21.15.26:0/3026543302 learned_addr learned my addr 172.21.15.26:0/3026543302 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.776+0000 7f9499d9b700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e24a890 0x560a4e24f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 -- 172.21.15.26:0/3026543302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e24a890 msgr2=0x560a4e24f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e24a890 0x560a4e24f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 -- 172.21.15.26:0/3026543302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e23f9e0 msgr2=0x560a4e24ed80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e23f9e0 0x560a4e24ed80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 -- 172.21.15.26:0/3026543302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a4dfa6c60 con 0x560a4e23f0c0 2023-10-06T21:20:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9499d9b700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9494006010 tx=0x7f9494009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c12cd06bc5a3779 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.777+0000 7f9489ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.778+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949403dad0 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.778+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f949402c080 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.778+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f948c004830 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.779+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f949402c240 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.779+0000 7f94bc40ab80 10 client.26936 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.779+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560a4dfe4250 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f948c004830 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f949400f570 con 0x560a4e23f0c0 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 10 client.26936 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 20 client.26936 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e2aeaa0 0x560a4e2b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 4 client.26936 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:20:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.780+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a4e18bb30 con 0x560a4e2aeaa0 2023-10-06T21:20:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.781+0000 7f94a259c700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e2aeaa0 0x560a4e2b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.783+0000 7f94a259c700 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e2aeaa0 0x560a4e2b0ec0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423b04875861ae8e server_cookie=4508fe4f0ebd3d3f in_seq=0 out_seq=0 2023-10-06T21:20:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.783+0000 7f9489ffb700 10 client.26936 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.783+0000 7f9489ffb700 1 -- 172.21.15.26:0/3026543302 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560a4e18bb30 con 0x560a4e2aeaa0 2023-10-06T21:20:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.783+0000 7f9489ffb700 10 client.26936 handle_command_reply: tid=0 2023-10-06T21:20:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.783+0000 7f94bc40ab80 1 client.26936 shutdown 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.784+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 msgr2=0x560a4e24e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.784+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9494006010 tx=0x7f9494009310 comp rx=0 tx=0).stop 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.784+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 shutdown_connections 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.784+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a4e24a890 0x560a4e24f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.784+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560a4e2aeaa0 0x560a4e2b0ec0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a4e23f0c0 0x560a4e24e840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 1 --2- 172.21.15.26:0/3026543302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a4e23f9e0 0x560a4e24ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 >> 172.21.15.26:0/3026543302 conn(0x560a4e1b4260 msgr2=0x560a4e248ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f9489ffb700 0 client.26936 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 shutdown_connections 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 1 -- 172.21.15.26:0/3026543302 wait complete. 2023-10-06T21:20:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.785+0000 7f94bc40ab80 20 client.26936 trim_cache size 0 max 16384 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "2bc51b4d-6ed3-49e0-baac-60a9da1c43f9": { 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "2bc51b4d-6ed3-49e0-baac-60a9da1c43f9", 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:20:13.789 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.786+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 msgr2=0x7f94900a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.786+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 0x7f94900a3c40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f949c0c9ef0 tx=0x7f949c00cf90 comp rx=0 tx=0).stop 2023-10-06T21:20:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.786+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 msgr2=0x7f94a41aa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.786+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f949c00b9e0 tx=0x7f949c00bda0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 shutdown_connections 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a411a690 0x7f94a41aa890 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94a41083f0 0x7f94a41a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f94900a1780 0x7f94900a3c40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 --2- 172.21.15.26:0/816130388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a410f640 0x7f94a41a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 >> 172.21.15.26:0/816130388 conn(0x7f94a406d1a0 msgr2=0x7f94a4118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 shutdown_connections 2023-10-06T21:20:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:13.787+0000 7f94a89ff700 1 -- 172.21.15.26:0/816130388 wait complete. 2023-10-06T21:20:13.817 INFO:tasks.cephfs.filesystem:scrub status for tag:2bc51b4d-6ed3-49e0-baac-60a9da1c43f9 - {'path': '/', 'tag': '2bc51b4d-6ed3-49e0-baac-60a9da1c43f9', 'options': 'recursive,force'} 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: pgmap v309: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 16 MiB/s wr, 992 op/s 2023-10-06T21:20:14.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:14 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: pgmap v309: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 16 MiB/s wr, 992 op/s 2023-10-06T21:20:14.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:14 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: osdmap e97: 12 total, 12 up, 12 in 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: pgmap v309: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 16 MiB/s wr, 992 op/s 2023-10-06T21:20:14.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:14 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:15 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:15.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:15 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:15 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:15.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:15 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:15 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:15.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:15 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:15 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:15.577 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:15 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:15 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:15.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:15 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:15.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:15 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:15.627 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:15 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:16.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:16 smithi167 ceph-mon[134494]: pgmap v310: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-10-06T21:20:16.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:16 smithi026 ceph-mon[98490]: pgmap v310: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-10-06T21:20:16.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:16 smithi078 ceph-mon[134896]: pgmap v310: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 798 op/s 2023-10-06T21:20:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 575.0 s, 1410.5 tps, lat 71.677 ms stddev 77.942 2023-10-06T21:20:18.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:18 smithi167 ceph-mon[134494]: pgmap v311: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:18.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:18 smithi026 ceph-mon[98490]: pgmap v311: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:18.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:18 smithi078 ceph-mon[134896]: pgmap v311: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:20.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:20 smithi167 ceph-mon[134494]: pgmap v312: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:20 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:20:20.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:20 smithi026 ceph-mon[98490]: pgmap v312: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:20.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:20 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:20:20.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:20 smithi078 ceph-mon[134896]: pgmap v312: 97 pgs: 97 active+clean; 5.7 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 11 MiB/s wr, 735 op/s 2023-10-06T21:20:20.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:20 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:20:21.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:21 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:20:21.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:21 smithi026 ceph-mon[98490]: scrub summary: idle 2023-10-06T21:20:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:21 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:20:21.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:21 smithi078 ceph-mon[134896]: scrub summary: idle 2023-10-06T21:20:22.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:21 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:20:22.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:21 smithi167 ceph-mon[134494]: scrub summary: idle 2023-10-06T21:20:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 580.0 s, 1483.4 tps, lat 67.136 ms stddev 74.804 2023-10-06T21:20:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:22 smithi078 ceph-mon[134896]: pgmap v313: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 811 op/s 2023-10-06T21:20:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:22 smithi167 ceph-mon[134494]: pgmap v313: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 811 op/s 2023-10-06T21:20:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:22 smithi026 ceph-mon[98490]: pgmap v313: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 811 op/s 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.215+0000 7f4b407ff700 1 -- 172.21.15.26:0/3690344703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 msgr2=0x7f4b38068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.215+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3690344703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b38068750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009990 tx=0x7f4b28009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 -- 172.21.15.26:0/3690344703 shutdown_connections 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3690344703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 0x7f4b3810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3690344703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b38106f60 0x7f4b38107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3690344703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b38068750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 -- 172.21.15.26:0/3690344703 >> 172.21.15.26:0/3690344703 conn(0x7f4b38076b70 msgr2=0x7f4b38076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 -- 172.21.15.26:0/3690344703 shutdown_connections 2023-10-06T21:20:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.216+0000 7f4b407ff700 1 -- 172.21.15.26:0/3690344703 wait complete. 2023-10-06T21:20:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.217+0000 7f4b407ff700 1 Processor -- start 2023-10-06T21:20:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.218+0000 7f4b407ff700 1 -- start start 2023-10-06T21:20:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 0x7f4b381a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b38106f60 0x7f4b3810c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3810cfd0 con 0x7f4b38106f60 2023-10-06T21:20:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3810d140 con 0x7f4b380682d0 2023-10-06T21:20:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b407ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3810d300 con 0x7f4b38068c90 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33988/0 (socket says 172.21.15.26:33988) 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3623366259 learned_addr learned my addr 172.21.15.26:0/3623366259 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 0x7f4b381a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3623366259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 msgr2=0x7f4b381a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 0x7f4b381a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b38106f60 msgr2=0x7f4b3810c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b38106f60 0x7f4b3810c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.219+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3623366259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b280095f0 con 0x7f4b380682d0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400e9a0 tx=0x7f4b3400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b2f7fe700 1 -- 172.21.15.26:0/3623366259 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34009790 con 0x7f4b380682d0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b2f7fe700 1 -- 172.21.15.26:0/3623366259 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b3400cbd0 con 0x7f4b380682d0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b2f7fe700 1 -- 172.21.15.26:0/3623366259 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34018680 con 0x7f4b380682d0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3810d5e0 con 0x7f4b380682d0 2023-10-06T21:20:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.220+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b381aa560 con 0x7f4b380682d0 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.225+0000 7f4b5421ab80 1 Processor -- start 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.225+0000 7f4b2f7fe700 1 -- 172.21.15.26:0/3623366259 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4b3400cd40 con 0x7f4b380682d0 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 -- start start 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6046fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad60547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5fb0400 con 0x55bad6046c00 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5fa1bc0 con 0x55bad6047520 2023-10-06T21:20:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.226+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5f9f4e0 con 0x55bad60523d0 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.227+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 -- 172.21.15.26:0/1275038315 learned_addr learned my addr 172.21.15.26:0/1275038315 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b2f7fe700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 0x7f4b240a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 -- 172.21.15.26:0/1275038315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 msgr2=0x55bad60547c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad60547c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 -- 172.21.15.26:0/1275038315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 msgr2=0x55bad6046fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6046fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.228+0000 7f4b3e59b700 1 -- 172.21.15.26:0/1275038315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad5dc5d80 con 0x55bad6047520 2023-10-06T21:20:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.229+0000 7f4b2f7fe700 1 -- 172.21.15.26:0/3623366259 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4b34014070 con 0x7f4b380682d0 2023-10-06T21:20:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.229+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4b3409a060 tx=0x7f4b3409a240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f92d059c10fb0a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.229+0000 7f4b2cff9700 1 -- 172.21.15.26:0/1275038315 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3409a360 con 0x55bad6047520 2023-10-06T21:20:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.229+0000 7f4b2cff9700 1 -- 172.21.15.26:0/1275038315 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b3409e4b0 con 0x55bad6047520 2023-10-06T21:20:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.229+0000 7f4b2cff9700 1 -- 172.21.15.26:0/1275038315 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3409a5c0 con 0x55bad6047520 2023-10-06T21:20:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.230+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 0x7f4b240a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 -- 172.21.15.26:0/1275038315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 msgr2=0x55bad6051e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4b3409a060 tx=0x7f4b3409a240 comp rx=0 tx=0).stop 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 -- 172.21.15.26:0/1275038315 shutdown_connections 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad60547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6046fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/1275038315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6047520 0x55bad6051e90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 -- 172.21.15.26:0/1275038315 >> 172.21.15.26:0/1275038315 conn(0x55bad5fbc200 msgr2=0x55bad6043080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 0x7f4b240a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009690 tx=0x7f4b300095e0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 -- 172.21.15.26:0/1275038315 shutdown_connections 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.231+0000 7f4b5421ab80 1 -- 172.21.15.26:0/1275038315 wait complete. 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.232+0000 7f4b5421ab80 1 -- start start 2023-10-06T21:20:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.232+0000 7f4b5421ab80 1 Processor -- start 2023-10-06T21:20:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6051e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad604f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad604f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5f9add0 con 0x55bad6046c00 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5f376b0 con 0x55bad60523d0 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5f37350 con 0x55bad6047520 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b5421ab80 10 client.0 _ll_register_callbacks cb 0x55bad5d947a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6051e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b3ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad604f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b3dd9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad604f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b3ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad604f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34000/0 (socket says 172.21.15.26:34000) 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.233+0000 7f4b3ed9c700 1 -- 172.21.15.26:0/4092400261 learned_addr learned my addr 172.21.15.26:0/4092400261 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 -- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 msgr2=0x55bad604f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad604f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 -- 172.21.15.26:0/4092400261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 msgr2=0x55bad6051e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6051e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 -- 172.21.15.26:0/4092400261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b381aa560 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad604f0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4b380fe630 tx=0x7f4b2800beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d138b36a7e36c362 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.234+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28004500 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bad603ad50 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4b28004660 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b28004820 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b5421ab80 10 client.26821 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55bad5daec00 con 0x55bad6047520 2023-10-06T21:20:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.235+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b2802f020 con 0x55bad6047520 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4b2800f760 con 0x55bad6047520 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 10 client.26821 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 20 client.26821 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad6061ea0 0x55bad60642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 4 client.26821 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.236+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bad5f93db0 con 0x55bad6061ea0 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.237+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad6061ea0 0x55bad60642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.238+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad6061ea0 0x55bad60642c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482b9f62af2cca99 server_cookie=12a5a3a726a0180d in_seq=0 out_seq=0 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.238+0000 7f4b21ffb700 10 client.26821 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b21ffb700 1 -- 172.21.15.26:0/4092400261 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bad5f93db0 con 0x55bad6061ea0 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b21ffb700 10 client.26821 handle_command_reply: tid=0 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 client.26821 shutdown 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 msgr2=0x55bad604f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad604f0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4b380fe630 tx=0x7f4b2800beb0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 shutdown_connections 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad6061ea0 0x55bad60642c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad604f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6051e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/4092400261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad604f0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 >> 172.21.15.26:0/4092400261 conn(0x55bad5fbc200 msgr2=0x55bad6042c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b21ffb700 0 client.26821 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 shutdown_connections 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 1 -- 172.21.15.26:0/4092400261 wait complete. 2023-10-06T21:20:24.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.239+0000 7f4b5421ab80 20 client.26821 trim_cache size 0 max 16384 2023-10-06T21:20:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.261+0000 7f4b5421ab80 1 Processor -- start 2023-10-06T21:20:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.261+0000 7f4b5421ab80 1 -- start start 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 0x55bad605d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad604a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad5e1b8c0 con 0x55bad6047520 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad603a8c0 con 0x55bad6046c00 2023-10-06T21:20:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.262+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad6043dc0 con 0x55bad60523d0 2023-10-06T21:20:24.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34008/0 (socket says 172.21.15.26:34008) 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3166906631 learned_addr learned my addr 172.21.15.26:0/3166906631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 0x55bad605d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad604a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3166906631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 msgr2=0x55bad604a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad604a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3166906631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 msgr2=0x55bad605d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 0x55bad605d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 -- 172.21.15.26:0/3166906631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad5dec1f0 con 0x55bad6046c00 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 0x55bad605d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.263+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b34096860 tx=0x7f4b3400efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f883c1dd45c3f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.264+0000 7f4b2d7fa700 1 -- 172.21.15.26:0/3166906631 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3409a360 con 0x55bad6046c00 2023-10-06T21:20:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.264+0000 7f4b2d7fa700 1 -- 172.21.15.26:0/3166906631 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b340a26f0 con 0x55bad6046c00 2023-10-06T21:20:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.264+0000 7f4b2d7fa700 1 -- 172.21.15.26:0/3166906631 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34099ab0 con 0x55bad6046c00 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.264+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3166906631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 msgr2=0x55bad6058e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.264+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b34096860 tx=0x7f4b3400efa0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3166906631 shutdown_connections 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad60523d0 0x55bad604a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad6046c00 0x55bad6058e50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3166906631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6047520 0x55bad605d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3166906631 >> 172.21.15.26:0/3166906631 conn(0x55bad5fbc200 msgr2=0x55bad6051a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3166906631 shutdown_connections 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3166906631 wait complete. 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 -- start start 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.265+0000 7f4b5421ab80 1 Processor -- start 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6054e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad6055370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad60b4d90 con 0x55bad6046c00 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad60b4f00 con 0x55bad60523d0 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad60b50c0 con 0x55bad6047520 2023-10-06T21:20:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b5421ab80 10 client.0 _ll_register_callbacks cb 0x7f4b28009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.266+0000 7f4b3e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6054e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34022/0 (socket says 172.21.15.26:34022) 2023-10-06T21:20:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 -- 172.21.15.26:0/3746736778 learned_addr learned my addr 172.21.15.26:0/3746736778 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad6055370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 -- 172.21.15.26:0/3746736778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 msgr2=0x55bad6055370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad6055370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 -- 172.21.15.26:0/3746736778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 msgr2=0x55bad6054e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6054e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 -- 172.21.15.26:0/3746736778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bad5dc5d80 con 0x55bad60523d0 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3dd9a700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad6055370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6054e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b3ed9c700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b30015e20 tx=0x7f4b30009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c32597bca8be9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.267+0000 7f4b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.269+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001c070 con 0x55bad60523d0 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.269+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4b30013d70 con 0x55bad60523d0 2023-10-06T21:20:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.269+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b3409e4b0 con 0x55bad60523d0 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.270+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b30023480 con 0x55bad60523d0 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.270+0000 7f4b5421ab80 10 client.26954 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.270+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55bad5dc4250 con 0x55bad60523d0 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b3409e4b0 con 0x55bad60523d0 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4b30022640 con 0x55bad60523d0 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 10 client.26954 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:24.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 20 client.26954 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad60b6940 0x55bad60b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 4 client.26954 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bad5f93ad0 con 0x55bad60b6940 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.271+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad60b6940 0x55bad60b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.272+0000 7f4b3e59b700 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad60b6940 0x55bad60b8d60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330204d3c70e98e7 server_cookie=b32fbfb1bae84472 in_seq=0 out_seq=0 2023-10-06T21:20:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.272+0000 7f4b21ffb700 10 client.26954 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.294+0000 7f4b21ffb700 1 -- 172.21.15.26:0/3746736778 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bad5f93ad0 con 0x55bad60b6940 2023-10-06T21:20:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.294+0000 7f4b21ffb700 10 client.26954 handle_command_reply: tid=0 2023-10-06T21:20:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.294+0000 7f4b5421ab80 1 client.26954 shutdown 2023-10-06T21:20:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.294+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 msgr2=0x55bad60558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.294+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b30015e20 tx=0x7f4b30009bb0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 shutdown_connections 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bad60b6940 0x55bad60b8d60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bad60523d0 0x55bad60558b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bad6046c00 0x55bad6054e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 --2- 172.21.15.26:0/3746736778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bad6047520 0x55bad6055370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 >> 172.21.15.26:0/3746736778 conn(0x55bad5fbc200 msgr2=0x55bad604a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.295+0000 7f4b21ffb700 0 client.26954 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.296+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 shutdown_connections 2023-10-06T21:20:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.296+0000 7f4b5421ab80 1 -- 172.21.15.26:0/3746736778 wait complete. 2023-10-06T21:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.297+0000 7f4b5421ab80 20 client.26954 trim_cache size 0 max 16384 2023-10-06T21:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:20:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 msgr2=0x7f4b240a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 0x7f4b240a3b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009690 tx=0x7f4b300095e0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 msgr2=0x7f4b381a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400e9a0 tx=0x7f4b3400ed60 comp rx=0 tx=0).stop 2023-10-06T21:20:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 shutdown_connections 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4b240a1670 0x7f4b240a3b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b38106f60 0x7f4b3810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b380682d0 0x7f4b381a2440 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 --2- 172.21.15.26:0/3623366259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b38068c90 0x7f4b381a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 >> 172.21.15.26:0/3623366259 conn(0x7f4b38076b70 msgr2=0x7f4b380feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 shutdown_connections 2023-10-06T21:20:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:24.298+0000 7f4b407ff700 1 -- 172.21.15.26:0/3623366259 wait complete. 2023-10-06T21:20:24.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:24 smithi167 ceph-mon[134494]: pgmap v314: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:20:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:24 smithi026 ceph-mon[98490]: pgmap v314: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:20:24.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:24 smithi078 ceph-mon[134896]: pgmap v314: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 11 MiB/s wr, 792 op/s 2023-10-06T21:20:26.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:26 smithi167 ceph-mon[134494]: pgmap v315: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.4 MiB/s wr, 686 op/s 2023-10-06T21:20:26.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:26 smithi026 ceph-mon[98490]: pgmap v315: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.4 MiB/s wr, 686 op/s 2023-10-06T21:20:26.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:26 smithi078 ceph-mon[134896]: pgmap v315: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.4 MiB/s wr, 686 op/s 2023-10-06T21:20:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 585.0 s, 1348.6 tps, lat 73.177 ms stddev 81.966 2023-10-06T21:20:28.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:28 smithi167 ceph-mon[134494]: pgmap v316: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 915 op/s 2023-10-06T21:20:28.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:28 smithi026 ceph-mon[98490]: pgmap v316: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 915 op/s 2023-10-06T21:20:28.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:28 smithi078 ceph-mon[134896]: pgmap v316: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 915 op/s 2023-10-06T21:20:30.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:30 smithi167 ceph-mon[134494]: pgmap v317: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 572 op/s 2023-10-06T21:20:30.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:30 smithi026 ceph-mon[98490]: pgmap v317: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 572 op/s 2023-10-06T21:20:30.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:30 smithi078 ceph-mon[134896]: pgmap v317: 97 pgs: 97 active+clean; 5.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 572 op/s 2023-10-06T21:20:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 590.0 s, 1333.2 tps, lat 76.428 ms stddev 131.636 2023-10-06T21:20:32.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:32 smithi167 ceph-mon[134494]: pgmap v318: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-06T21:20:32.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:32 smithi026 ceph-mon[98490]: pgmap v318: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-06T21:20:32.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:32 smithi078 ceph-mon[134896]: pgmap v318: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-06T21:20:34.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:34 smithi167 ceph-mon[134494]: pgmap v319: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-06T21:20:34.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:34 smithi026 ceph-mon[98490]: pgmap v319: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-06T21:20:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:34 smithi078 ceph-mon[134896]: pgmap v319: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-06T21:20:36.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:36 smithi026 ceph-mon[98490]: pgmap v320: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2023-10-06T21:20:36.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:36 smithi078 ceph-mon[134896]: pgmap v320: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2023-10-06T21:20:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:36 smithi167 ceph-mon[134494]: pgmap v320: 97 pgs: 97 active+clean; 6.0 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2023-10-06T21:20:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 595.0 s, 1100.4 tps, lat 89.307 ms stddev 95.718 2023-10-06T21:20:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.577+0000 7f8675bff700 1 -- 172.21.15.26:0/3196045859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101ab0 msgr2=0x7f8670105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.577+0000 7f8675bff700 1 --2- 172.21.15.26:0/3196045859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101ab0 0x7f8670105af0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8664009990 tx=0x7f8664009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 -- 172.21.15.26:0/3196045859 shutdown_connections 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 --2- 172.21.15.26:0/3196045859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f867010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 --2- 172.21.15.26:0/3196045859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8670101100 0x7f86701014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 --2- 172.21.15.26:0/3196045859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101ab0 0x7f8670105af0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 -- 172.21.15.26:0/3196045859 >> 172.21.15.26:0/3196045859 conn(0x7f86700fc9a0 msgr2=0x7f86700fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 -- 172.21.15.26:0/3196045859 shutdown_connections 2023-10-06T21:20:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.578+0000 7f8675bff700 1 -- 172.21.15.26:0/3196045859 wait complete. 2023-10-06T21:20:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 Processor -- start 2023-10-06T21:20:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 -- start start 2023-10-06T21:20:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101100 0x7f8670199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f867019ddd0 0x7f867019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867019e850 con 0x7f867019ddd0 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867019e9c0 con 0x7f8670101100 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.579+0000 7f8675bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867019eb80 con 0x7f8670106220 2023-10-06T21:20:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101100 0x7f8670199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55350/0 (socket says 172.21.15.26:55350) 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 -- 172.21.15.26:0/2136647792 learned_addr learned my addr 172.21.15.26:0/2136647792 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866ffff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f867019ddd0 0x7f867019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 -- 172.21.15.26:0/2136647792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101100 msgr2=0x7f8670199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101100 0x7f8670199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 -- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f867019ddd0 msgr2=0x7f867019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f867019ddd0 0x7f867019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 -- 172.21.15.26:0/2136647792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640095f0 con 0x7f8670106220 2023-10-06T21:20:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.580+0000 7f866effd700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8664005950 tx=0x7f8664005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.581+0000 7f866cff9700 1 -- 172.21.15.26:0/2136647792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866401d070 con 0x7f8670106220 2023-10-06T21:20:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.581+0000 7f866cff9700 1 -- 172.21.15.26:0/2136647792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8664021410 con 0x7f8670106220 2023-10-06T21:20:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.581+0000 7f866cff9700 1 -- 172.21.15.26:0/2136647792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664017650 con 0x7f8670106220 2023-10-06T21:20:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.581+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86701a6080 con 0x7f8670106220 2023-10-06T21:20:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.581+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86701a6450 con 0x7f8670106220 2023-10-06T21:20:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.588+0000 7f86895ddb80 1 Processor -- start 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.588+0000 7f86895ddb80 1 -- start start 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bfb9400 con 0x56338c050520 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bfaabc0 con 0x56338c04fc00 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f86895ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bfa84e0 con 0x56338c063be0 2023-10-06T21:20:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55516/0 (socket says 172.21.15.26:55516) 2023-10-06T21:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 -- 172.21.15.26:0/2487504198 learned_addr learned my addr 172.21.15.26:0/2487504198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866f7fe700 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 -- 172.21.15.26:0/2487504198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 msgr2=0x56338c065fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c065fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 -- 172.21.15.26:0/2487504198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 msgr2=0x56338c0636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.589+0000 7f866ffff700 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f866ffff700 1 -- 172.21.15.26:0/2487504198 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338bdced80 con 0x56338c04fc00 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f866f7fe700 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f866ffff700 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8668011680 tx=0x7f8668009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a246e30331fdba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f8655ffb700 1 -- 172.21.15.26:0/2487504198 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866800e040 con 0x56338c04fc00 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f8655ffb700 1 -- 172.21.15.26:0/2487504198 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f866800f040 con 0x56338c04fc00 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f8655ffb700 1 -- 172.21.15.26:0/2487504198 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668004890 con 0x56338c04fc00 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.590+0000 7f866cff9700 1 -- 172.21.15.26:0/2136647792 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8664021580 con 0x7f8670106220 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 -- 172.21.15.26:0/2487504198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 msgr2=0x56338c04ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8668011680 tx=0x7f8668009920 comp rx=0 tx=0).stop 2023-10-06T21:20:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 -- 172.21.15.26:0/2487504198 shutdown_connections 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c04fc00 0x56338c04ffe0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2487504198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 -- 172.21.15.26:0/2487504198 >> 172.21.15.26:0/2487504198 conn(0x56338bfc5200 msgr2=0x56338c04c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 -- 172.21.15.26:0/2487504198 shutdown_connections 2023-10-06T21:20:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.591+0000 7f86895ddb80 1 -- 172.21.15.26:0/2487504198 wait complete. 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.592+0000 7f866cff9700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 0x7f86580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.592+0000 7f86895ddb80 1 -- start start 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.592+0000 7f866cff9700 1 -- 172.21.15.26:0/2136647792 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f86640ce550 con 0x7f8670106220 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.592+0000 7f866ffff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 0x7f86580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.592+0000 7f86895ddb80 1 Processor -- start 2023-10-06T21:20:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f866ffff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 0x7f86580a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8668000f80 tx=0x7f8668011fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c04fc00 0x56338c063630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c063be0 0x56338c060f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bfa3dd0 con 0x56338c050520 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bf406b0 con 0x56338c063be0 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338bf40350 con 0x56338c04fc00 2023-10-06T21:20:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.593+0000 7f86895ddb80 10 client.0 _ll_register_callbacks cb 0x56338bd9d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.595+0000 7f866effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.595+0000 7f866effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49964/0 (socket says 172.21.15.26:49964) 2023-10-06T21:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.595+0000 7f866effd700 1 -- 172.21.15.26:0/1459492570 learned_addr learned my addr 172.21.15.26:0/1459492570 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.595+0000 7f866f7fe700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c04fc00 0x56338c063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 -- 172.21.15.26:0/1459492570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c04fc00 msgr2=0x56338c063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c04fc00 0x56338c063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 -- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c063be0 msgr2=0x56338c060f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c063be0 0x56338c060f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 -- 172.21.15.26:0/1459492570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8668011330 con 0x56338c050520 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f866effd700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f866409b250 tx=0x7f8664097280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f445be124c765d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.596+0000 7f864effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866409b550 con 0x56338c050520 2023-10-06T21:20:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f866409f6a0 con 0x56338c050520 2023-10-06T21:20:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56338c043d50 con 0x56338c050520 2023-10-06T21:20:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86640a3850 con 0x56338c050520 2023-10-06T21:20:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f86895ddb80 10 client.16638 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.597+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56338bdb7c00 con 0x56338c050520 2023-10-06T21:20:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.598+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56338c043d50 con 0x56338c050520 2023-10-06T21:20:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f86640ce050 con 0x56338c050520 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 10 client.16638 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 20 client.16638 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0732f0 0x56338c075710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 4 client.16638 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.599+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56338bf9cdb0 con 0x56338c0732f0 2023-10-06T21:20:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.600+0000 7f866f7fe700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0732f0 0x56338c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.600+0000 7f866f7fe700 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0732f0 0x56338c075710 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0c3ed773b5e50c server_cookie=412c1a3e682e995e in_seq=0 out_seq=0 2023-10-06T21:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.600+0000 7f864effd700 10 client.16638 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.601+0000 7f864effd700 1 -- 172.21.15.26:0/1459492570 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56338bf9cdb0 con 0x56338c0732f0 2023-10-06T21:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.601+0000 7f864effd700 10 client.16638 handle_command_reply: tid=0 2023-10-06T21:20:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.601+0000 7f86895ddb80 1 client.16638 shutdown 2023-10-06T21:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 msgr2=0x56338c0608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f866409b250 tx=0x7f8664097280 comp rx=0 tx=0).stop 2023-10-06T21:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 shutdown_connections 2023-10-06T21:20:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0732f0 0x56338c075710 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c063be0 0x56338c060f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c04fc00 0x56338c063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 --2- 172.21.15.26:0/1459492570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c050520 0x56338c0608a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 >> 172.21.15.26:0/1459492570 conn(0x56338bfc5200 msgr2=0x56338c04bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f864effd700 0 client.16638 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 shutdown_connections 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 1 -- 172.21.15.26:0/1459492570 wait complete. 2023-10-06T21:20:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.602+0000 7f86895ddb80 20 client.16638 trim_cache size 0 max 16384 2023-10-06T21:20:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 Processor -- start 2023-10-06T21:20:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 -- start start 2023-10-06T21:20:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c060f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338be248c0 con 0x56338c04fc00 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338c070070 con 0x56338c050520 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f86895ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338c0438c0 con 0x56338c063be0 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f866effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f866ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f866f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c060f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f866ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55366/0 (socket says 172.21.15.26:55366) 2023-10-06T21:20:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.621+0000 7f866ffff700 1 -- 172.21.15.26:0/2602672910 learned_addr learned my addr 172.21.15.26:0/2602672910 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 -- 172.21.15.26:0/2602672910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 msgr2=0x56338c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 -- 172.21.15.26:0/2602672910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 msgr2=0x56338c060f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c060f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 -- 172.21.15.26:0/2602672910 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670112560 con 0x56338c063be0 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866f7fe700 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c060f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.622+0000 7f866ffff700 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8668004420 tx=0x7f8668009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eafed34e53cdc1ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86567fc700 1 -- 172.21.15.26:0/2602672910 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668003a00 con 0x56338c063be0 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86567fc700 1 -- 172.21.15.26:0/2602672910 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8668007500 con 0x56338c063be0 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86567fc700 1 -- 172.21.15.26:0/2602672910 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668015450 con 0x56338c063be0 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 -- 172.21.15.26:0/2602672910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 msgr2=0x56338c05fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8668004420 tx=0x7f8668009920 comp rx=0 tx=0).stop 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 -- 172.21.15.26:0/2602672910 shutdown_connections 2023-10-06T21:20:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c05fe50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c060f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 --2- 172.21.15.26:0/2602672910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.623+0000 7f86895ddb80 1 -- 172.21.15.26:0/2602672910 >> 172.21.15.26:0/2602672910 conn(0x56338bfc5200 msgr2=0x56338c06f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.624+0000 7f86895ddb80 1 -- 172.21.15.26:0/2602672910 shutdown_connections 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.624+0000 7f86895ddb80 1 -- 172.21.15.26:0/2602672910 wait complete. 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.624+0000 7f86895ddb80 1 -- start start 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.624+0000 7f86895ddb80 1 Processor -- start 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338c06a7a0 con 0x56338c04fc00 2023-10-06T21:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338c0c6660 con 0x56338c050520 2023-10-06T21:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338c0c67a0 con 0x56338c063be0 2023-10-06T21:20:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f86895ddb80 10 client.0 _ll_register_callbacks cb 0x7f8664097890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49994/0 (socket says 172.21.15.26:49994) 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 -- 172.21.15.26:0/3664494948 learned_addr learned my addr 172.21.15.26:0/3664494948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866ffff700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 -- 172.21.15.26:0/3664494948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 msgr2=0x56338c0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c0c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 -- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 msgr2=0x56338c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.625+0000 7f866f7fe700 1 -- 172.21.15.26:0/3664494948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86600042e0 con 0x56338c04fc00 2023-10-06T21:20:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f866effd700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f866f7fe700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8660005950 tx=0x7f866000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e286a956666d27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f864effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660017040 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f866800f040 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f866000bdc0 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.626+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8660003680 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.627+0000 7f86895ddb80 10 client.16644 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.627+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56338bdced80 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.627+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f866800f040 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.627+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8660011920 con 0x56338c04fc00 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 10 client.16644 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 20 client.16644 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0c7fb0 0x56338c0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 4 client.16644 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.628+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56338bf9cad0 con 0x56338c0c7fb0 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.629+0000 7f866effd700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0c7fb0 0x56338c0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.630+0000 7f866effd700 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0c7fb0 0x56338c0ca3d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f64451b02597eb7 server_cookie=5503d8fd0b278def in_seq=0 out_seq=0 2023-10-06T21:20:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.630+0000 7f864effd700 10 client.16644 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.732+0000 7f864effd700 1 -- 172.21.15.26:0/3664494948 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56338bf9cad0 con 0x56338c0c7fb0 2023-10-06T21:20:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.732+0000 7f864effd700 10 client.16644 handle_command_reply: tid=0 2023-10-06T21:20:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.732+0000 7f86895ddb80 1 client.16644 shutdown 2023-10-06T21:20:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.733+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 msgr2=0x56338c069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.733+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8660005950 tx=0x7f866000b710 comp rx=0 tx=0).stop 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 shutdown_connections 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56338c063be0 0x56338c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56338c04fc00 0x56338c069950 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56338c0c7fb0 0x56338c0ca3d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 --2- 172.21.15.26:0/3664494948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56338c050520 0x56338c069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 >> 172.21.15.26:0/3664494948 conn(0x56338bfc5200 msgr2=0x56338c06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f864effd700 0 client.16644 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 shutdown_connections 2023-10-06T21:20:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 1 -- 172.21.15.26:0/3664494948 wait complete. 2023-10-06T21:20:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.734+0000 7f86895ddb80 20 client.16644 trim_cache size 0 max 16384 2023-10-06T21:20:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:20:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:20:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:20:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:20:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.735+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 msgr2=0x7f86580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.735+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 0x7f86580a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8668000f80 tx=0x7f8668011fd0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 msgr2=0x7f8670199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8664005950 tx=0x7f8664005740 comp rx=0 tx=0).stop 2023-10-06T21:20:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 shutdown_connections 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f86580a1500 0x7f86580a39c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f867019ddd0 0x7f867019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8670101100 0x7f8670199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 --2- 172.21.15.26:0/2136647792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8670106220 0x7f8670199760 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 >> 172.21.15.26:0/2136647792 conn(0x7f86700fc9a0 msgr2=0x7f86700fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 shutdown_connections 2023-10-06T21:20:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:37.736+0000 7f8675bff700 1 -- 172.21.15.26:0/2136647792 wait complete. 2023-10-06T21:20:38.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:38 smithi026 ceph-mon[98490]: pgmap v321: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2023-10-06T21:20:38.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:38 smithi078 ceph-mon[134896]: pgmap v321: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2023-10-06T21:20:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:38 smithi167 ceph-mon[134494]: pgmap v321: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 28 MiB/s wr, 1.84k op/s 2023-10-06T21:20:40.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:40 smithi078 ceph-mon[134896]: pgmap v322: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2023-10-06T21:20:41.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:40 smithi167 ceph-mon[134494]: pgmap v322: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2023-10-06T21:20:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:40 smithi026 ceph-mon[98490]: pgmap v322: 97 pgs: 97 active+clean; 6.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2023-10-06T21:20:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 600.0 s, 1101.4 tps, lat 91.596 ms stddev 93.960 2023-10-06T21:20:43.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:42 smithi167 ceph-mon[134494]: pgmap v323: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 1.99k op/s 2023-10-06T21:20:43.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:42 smithi026 ceph-mon[98490]: pgmap v323: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 1.99k op/s 2023-10-06T21:20:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:42 smithi078 ceph-mon[134896]: pgmap v323: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 1.99k op/s 2023-10-06T21:20:43.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:20:43.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:43 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:44.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:43 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:44.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:43 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:20:44.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 -- 172.21.15.26:0/1725485671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06811a690 msgr2=0x7fb06811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/1725485671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06811a690 0x7fb06811ca80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb050009990 tx=0x7fb050009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 -- 172.21.15.26:0/1725485671 shutdown_connections 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/1725485671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06811a690 0x7fb06811ca80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/1725485671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 0x7fb068107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/1725485671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0681083f0 0x7fb068108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.258+0000 7fb06d7ff700 1 -- 172.21.15.26:0/1725485671 >> 172.21.15.26:0/1725485671 conn(0x7fb06806d1a0 msgr2=0x7fb06806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.259+0000 7fb06d7ff700 1 -- 172.21.15.26:0/1725485671 shutdown_connections 2023-10-06T21:20:44.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.259+0000 7fb06d7ff700 1 -- 172.21.15.26:0/1725485671 wait complete. 2023-10-06T21:20:44.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.260+0000 7fb06d7ff700 1 Processor -- start 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.260+0000 7fb06d7ff700 1 -- start start 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 0x7fb0681a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb06811a690 0x7fb0681aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681aae60 con 0x7fb0681083f0 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681aafd0 con 0x7fb06810f640 2023-10-06T21:20:44.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb06d7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681ab190 con 0x7fb06811a690 2023-10-06T21:20:44.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33998/0 (socket says 172.21.15.26:33998) 2023-10-06T21:20:44.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 -- 172.21.15.26:0/3962614499 learned_addr learned my addr 172.21.15.26:0/3962614499 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.261+0000 7fb0667fc700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 0x7fb0681a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb0677fe700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb06811a690 0x7fb0681aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 -- 172.21.15.26:0/3962614499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb06811a690 msgr2=0x7fb0681aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb06811a690 0x7fb0681aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 -- 172.21.15.26:0/3962614499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 msgr2=0x7fb0681a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 0x7fb0681a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.262+0000 7fb066ffd700 1 -- 172.21.15.26:0/3962614499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500095f0 con 0x7fb0681083f0 2023-10-06T21:20:44.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.263+0000 7fb066ffd700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba50 tx=0x7fb06000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.263+0000 7fb05ffff700 1 -- 172.21.15.26:0/3962614499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000c930 con 0x7fb0681083f0 2023-10-06T21:20:44.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.263+0000 7fb05ffff700 1 -- 172.21.15.26:0/3962614499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb060014440 con 0x7fb0681083f0 2023-10-06T21:20:44.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.263+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0681ab470 con 0x7fb0681083f0 2023-10-06T21:20:44.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.264+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681b2cb0 con 0x7fb0681083f0 2023-10-06T21:20:44.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.264+0000 7fb05ffff700 1 -- 172.21.15.26:0/3962614499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600075d0 con 0x7fb0681083f0 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.271+0000 7fb081120b80 1 Processor -- start 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 -- start start 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9340c0 0x56516a9344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a93f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a941c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a89d460 con 0x56516a93f890 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a88ec20 con 0x56516a9349e0 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.272+0000 7fb081120b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a88c540 con 0x56516a9340c0 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.273+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a93f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.273+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a93f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58100/0 (socket says 172.21.15.26:58100) 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.273+0000 7fb066ffd700 1 -- 172.21.15.26:0/1114092326 learned_addr learned my addr 172.21.15.26:0/1114092326 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.273+0000 7fb0677fe700 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9340c0 0x56516a9344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.273+0000 7fb0667fc700 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a941c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 -- 172.21.15.26:0/1114092326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9340c0 msgr2=0x56516a9344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9340c0 0x56516a9344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 -- 172.21.15.26:0/1114092326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 msgr2=0x56516a93f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a93f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 -- 172.21.15.26:0/1114092326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516a6b2de0 con 0x56516a93f890 2023-10-06T21:20:44.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.278+0000 7fb0667fc700 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a941c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb058005950 tx=0x7fb05800aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c0bdea16e86936 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.279+0000 7fb05ffff700 1 -- 172.21.15.26:0/3962614499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb06000ca90 con 0x7fb0681083f0 2023-10-06T21:20:44.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.280+0000 7fb05d7fa700 1 -- 172.21.15.26:0/1114092326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800fdb0 con 0x56516a93f890 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.280+0000 7fb05d7fa700 1 -- 172.21.15.26:0/1114092326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0580044e0 con 0x56516a93f890 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.280+0000 7fb05d7fa700 1 -- 172.21.15.26:0/1114092326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580186a0 con 0x56516a93f890 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 -- 172.21.15.26:0/1114092326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 msgr2=0x56516a941c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a941c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb058005950 tx=0x7fb05800aeb0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 -- 172.21.15.26:0/1114092326 shutdown_connections 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a941c80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9340c0 0x56516a9344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 --2- 172.21.15.26:0/1114092326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a93f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 -- 172.21.15.26:0/1114092326 >> 172.21.15.26:0/1114092326 conn(0x56516a8a9260 msgr2=0x56516a930450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 -- 172.21.15.26:0/1114092326 shutdown_connections 2023-10-06T21:20:44.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.281+0000 7fb081120b80 1 -- 172.21.15.26:0/1114092326 wait complete. 2023-10-06T21:20:44.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.282+0000 7fb081120b80 1 -- start start 2023-10-06T21:20:44.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.283+0000 7fb05ffff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 0x7fb0540a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.283+0000 7fb081120b80 1 Processor -- start 2023-10-06T21:20:44.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.283+0000 7fb05ffff700 1 -- 172.21.15.26:0/3962614499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb060006e40 con 0x7fb0681083f0 2023-10-06T21:20:44.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb0677fe700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 0x7fb0540a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a93cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a93d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a887e30 con 0x56516a93f890 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a824710 con 0x56516a9340c0 2023-10-06T21:20:44.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a8243b0 con 0x56516a9349e0 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.284+0000 7fb081120b80 10 client.0 _ll_register_callbacks cb 0x56516a681800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58108/0 (socket says 172.21.15.26:58108) 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 -- 172.21.15.26:0/2451988442 learned_addr learned my addr 172.21.15.26:0/2451988442 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a93cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 -- 172.21.15.26:0/2451988442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 msgr2=0x56516a93cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a93cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb0677fe700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a93d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 -- 172.21.15.26:0/2451988442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 msgr2=0x56516a93d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a93d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 -- 172.21.15.26:0/2451988442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05800a830 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb0677fe700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a93d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb0677fe700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 0x7fb0540a3b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb050003800 tx=0x7fb05000b540 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb066ffd700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb060095bd0 tx=0x7fb060095a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223aa5d467d011b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.285+0000 7fb04e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060099210 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56516a927f10 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb060099dd0 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb060099030 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 10 client.26978 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56516a69bc60 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56516a927f10 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb060098bc0 con 0x56516a9340c0 2023-10-06T21:20:44.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 10 client.26978 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:44.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:44.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 20 client.26978 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:44.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a94f3e0 0x56516a951800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 4 client.26978 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:44.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.286+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56516a880e10 con 0x56516a94f3e0 2023-10-06T21:20:44.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.287+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a94f3e0 0x56516a951800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.287+0000 7fb04e7fc700 10 client.26978 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.287+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a94f3e0 0x56516a951800 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3340d7c0ff26d8a9 server_cookie=2494253361dc828f in_seq=0 out_seq=0 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb04e7fc700 1 -- 172.21.15.26:0/2451988442 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56516a880e10 con 0x56516a94f3e0 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb04e7fc700 10 client.26978 handle_command_reply: tid=0 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 client.26978 shutdown 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 msgr2=0x56516a93c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb060095bd0 tx=0x7fb060095a90 comp rx=0 tx=0).stop 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 shutdown_connections 2023-10-06T21:20:44.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a93d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a94f3e0 0x56516a951800 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a93c650 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 --2- 172.21.15.26:0/2451988442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a93cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 >> 172.21.15.26:0/2451988442 conn(0x56516a8a9260 msgr2=0x56516a92ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.288+0000 7fb04e7fc700 0 client.26978 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.289+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 shutdown_connections 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.289+0000 7fb081120b80 1 -- 172.21.15.26:0/2451988442 wait complete. 2023-10-06T21:20:44.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.289+0000 7fb081120b80 20 client.26978 trim_cache size 0 max 16384 2023-10-06T21:20:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.320+0000 7fb081120b80 1 Processor -- start 2023-10-06T21:20:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 -- start start 2023-10-06T21:20:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a936f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a946430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a9398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a708920 con 0x56516a93f890 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a931280 con 0x56516a9340c0 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.321+0000 7fb081120b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a94bd70 con 0x56516a9349e0 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.322+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a936f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.322+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a936f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58110/0 (socket says 172.21.15.26:58110) 2023-10-06T21:20:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.322+0000 7fb066ffd700 1 -- 172.21.15.26:0/2889169624 learned_addr learned my addr 172.21.15.26:0/2889169624 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.322+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a946430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.322+0000 7fb0677fe700 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a9398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 -- 172.21.15.26:0/2889169624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 msgr2=0x56516a936f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a936f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 -- 172.21.15.26:0/2889169624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 msgr2=0x56516a9398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a9398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 -- 172.21.15.26:0/2889169624 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56516a6d9250 con 0x56516a9349e0 2023-10-06T21:20:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.323+0000 7fb0667fc700 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a946430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40fe2b7cd9ec924 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.324+0000 7fb05dffb700 1 -- 172.21.15.26:0/2889169624 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058028070 con 0x56516a9349e0 2023-10-06T21:20:44.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.324+0000 7fb05dffb700 1 -- 172.21.15.26:0/2889169624 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0580037a0 con 0x56516a9349e0 2023-10-06T21:20:44.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.324+0000 7fb05dffb700 1 -- 172.21.15.26:0/2889169624 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580052a0 con 0x56516a9349e0 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 -- 172.21.15.26:0/2889169624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 msgr2=0x56516a946430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a946430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058003c90 comp rx=0 tx=0).stop 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 -- 172.21.15.26:0/2889169624 shutdown_connections 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a93f890 0x56516a9398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9340c0 0x56516a936f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 --2- 172.21.15.26:0/2889169624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a9349e0 0x56516a946430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 -- 172.21.15.26:0/2889169624 >> 172.21.15.26:0/2889169624 conn(0x56516a8a9260 msgr2=0x56516a937930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.325+0000 7fb081120b80 1 -- 172.21.15.26:0/2889169624 shutdown_connections 2023-10-06T21:20:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.326+0000 7fb081120b80 1 -- 172.21.15.26:0/2889169624 wait complete. 2023-10-06T21:20:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.326+0000 7fb081120b80 1 -- start start 2023-10-06T21:20:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.326+0000 7fb081120b80 1 Processor -- start 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a944210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a93f890 0x56516a944750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a944c90 con 0x56516a9340c0 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a944e00 con 0x56516a9349e0 2023-10-06T21:20:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516a944fc0 con 0x56516a93f890 2023-10-06T21:20:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb081120b80 10 client.0 _ll_register_callbacks cb 0x56516a681800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb066ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34042/0 (socket says 172.21.15.26:34042) 2023-10-06T21:20:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.327+0000 7fb066ffd700 1 -- 172.21.15.26:0/182516935 learned_addr learned my addr 172.21.15.26:0/182516935 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0677fe700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a93f890 0x56516a944750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a944210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 -- 172.21.15.26:0/182516935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a93f890 msgr2=0x56516a944750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a93f890 0x56516a944750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 -- 172.21.15.26:0/182516935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 msgr2=0x56516a945f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 -- 172.21.15.26:0/182516935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56516a6b2de0 con 0x56516a9349e0 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb066ffd700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb0667fc700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a944210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56516a93d9c0 tx=0x7fb058003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6a1b90accb94c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.328+0000 7fb04e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.329+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058028040 con 0x56516a9349e0 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.329+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb058003f00 con 0x56516a9349e0 2023-10-06T21:20:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.329+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0580044e0 con 0x56516a9349e0 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.330+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0580040c0 con 0x56516a9349e0 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.330+0000 7fb081120b80 10 client.26984 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.330+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56516a6b12b0 con 0x56516a9349e0 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0580044e0 con 0x56516a9349e0 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb058034070 con 0x56516a9349e0 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 10 client.26984 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 20 client.26984 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a9b5290 0x56516a9b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 4 client.26984 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56516a880b30 con 0x56516a9b5290 2023-10-06T21:20:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.331+0000 7fb066ffd700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a9b5290 0x56516a9b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb04e7fc700 10 client.26984 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb066ffd700 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a9b5290 0x56516a9b76b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b76cd2f7b73109 server_cookie=be621227b7a6880a in_seq=0 out_seq=0 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb04e7fc700 1 -- 172.21.15.26:0/182516935 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56516a880b30 con 0x56516a9b5290 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb04e7fc700 10 client.26984 handle_command_reply: tid=0 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 client.26984 shutdown 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 msgr2=0x56516a944210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a944210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x56516a93d9c0 tx=0x7fb058003910 comp rx=0 tx=0).stop 2023-10-06T21:20:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 shutdown_connections 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56516a93f890 0x56516a944750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56516a9340c0 0x56516a945f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56516a9b5290 0x56516a9b76b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 --2- 172.21.15.26:0/182516935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56516a9349e0 0x56516a944210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 >> 172.21.15.26:0/182516935 conn(0x56516a8a9260 msgr2=0x56516a93abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.332+0000 7fb04e7fc700 0 client.26984 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 shutdown_connections 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb081120b80 1 -- 172.21.15.26:0/182516935 wait complete. 2023-10-06T21:20:44.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb081120b80 20 client.26984 trim_cache size 0 max 16384 2023-10-06T21:20:44.338 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:44.338 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-10-06T21:20:44.338 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-10-06T21:20:44.338 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 msgr2=0x7fb0540a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 0x7fb0540a3b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb050003800 tx=0x7fb05000b540 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 msgr2=0x7fb0681a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.333+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba50 tx=0x7fb06000bd60 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 shutdown_connections 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb06811a690 0x7fb0681aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb0540a1640 0x7fb0540a3b00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0681083f0 0x7fb0681a5e80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 --2- 172.21.15.26:0/3962614499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb06810f640 0x7fb0681a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 >> 172.21.15.26:0/3962614499 conn(0x7fb06806d1a0 msgr2=0x7fb068118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 shutdown_connections 2023-10-06T21:20:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.334+0000 7fb06d7ff700 1 -- 172.21.15.26:0/3962614499 wait complete. 2023-10-06T21:20:44.368 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-06T21:20:44.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 -- 172.21.15.26:0/2393518209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 msgr2=0x7f67cc10ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2393518209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10ff60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67c400b0a0 tx=0x7f67c400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 -- 172.21.15.26:0/2393518209 shutdown_connections 2023-10-06T21:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2393518209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10ff60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2393518209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2393518209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc103c70 0x7f67cc107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 -- 172.21.15.26:0/2393518209 >> 172.21.15.26:0/2393518209 conn(0x7f67cc0feb00 msgr2=0x7f67cc100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.761+0000 7f67d47ff700 1 -- 172.21.15.26:0/2393518209 shutdown_connections 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.762+0000 7f67d47ff700 1 -- 172.21.15.26:0/2393518209 wait complete. 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 Processor -- start 2023-10-06T21:20:44.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 -- start start 2023-10-06T21:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc10ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 0x7f67cc1b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1b2260 con 0x7f67cc10f4f0 2023-10-06T21:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1b23d0 con 0x7f67cc1032c0 2023-10-06T21:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.763+0000 7f67d47ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc1b2590 con 0x7f67cc1083e0 2023-10-06T21:20:44.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.764+0000 7f67d1d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.764+0000 7f67d1d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49466/0 (socket says 172.21.15.26:49466) 2023-10-06T21:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.764+0000 7f67d1d9a700 1 -- 172.21.15.26:0/2056198302 learned_addr learned my addr 172.21.15.26:0/2056198302 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.764+0000 7f67d259b700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc10ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.764+0000 7f67d2d9c700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 0x7f67cc1b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.765+0000 7f67d2d9c700 1 -- 172.21.15.26:0/2056198302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 msgr2=0x7f67cc10efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.765+0000 7f67d2d9c700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.765+0000 7f67d2d9c700 1 -- 172.21.15.26:0/2056198302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 msgr2=0x7f67cc10ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.766+0000 7f67d2d9c700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc10ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.766+0000 7f67d2d9c700 1 -- 172.21.15.26:0/2056198302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4009d00 con 0x7f67cc10f4f0 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.766+0000 7f67d259b700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:44.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.766+0000 7f67d2d9c700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 0x7f67cc1b1c90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67c400b070 tx=0x7f67c400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.767+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400e040 con 0x7f67cc10f4f0 2023-10-06T21:20:44.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.767+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c4007a50 con 0x7f67cc10f4f0 2023-10-06T21:20:44.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.767+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c401b9b0 con 0x7f67cc10f4f0 2023-10-06T21:20:44.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.768+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc1b2810 con 0x7f67cc10f4f0 2023-10-06T21:20:44.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.768+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc1b2c40 con 0x7f67cc10f4f0 2023-10-06T21:20:44.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.769+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc068120 con 0x7f67cc10f4f0 2023-10-06T21:20:44.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.773+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f67c401bb10 con 0x7f67cc10f4f0 2023-10-06T21:20:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.776+0000 7f67c37fe700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 0x7f67b80a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:44.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.776+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f67c40cf1e0 con 0x7f67cc10f4f0 2023-10-06T21:20:44.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.778+0000 7f67d259b700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 0x7f67b80a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.781+0000 7f67d259b700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 0x7f67b80a3c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67c800afd0 tx=0x7f67c800a380 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:44.781+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f67c4098720 con 0x7f67cc10f4f0 2023-10-06T21:20:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:44 smithi167 ceph-mon[134494]: pgmap v324: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-10-06T21:20:45.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:44 smithi026 ceph-mon[98490]: pgmap v324: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-10-06T21:20:45.130 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:44 smithi078 ceph-mon[134896]: pgmap v324: 97 pgs: 97 active+clean; 6.1 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.02k op/s 2023-10-06T21:20:45.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.223+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67cc103e80 con 0x7f67cc10f4f0 2023-10-06T21:20:45.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.224+0000 7f67c37fe700 1 -- 172.21.15.26:0/2056198302 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+11314 (secure 0 0 0) 0x7f67c4021090 con 0x7f67cc10f4f0 2023-10-06T21:20:45.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:20:45.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:19:09.459935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:20:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 msgr2=0x7f67b80a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 0x7f67b80a3c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67c800afd0 tx=0x7f67c800a380 comp rx=0 tx=0).stop 2023-10-06T21:20:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 msgr2=0x7f67cc1b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 0x7f67cc1b1c90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f67c400b070 tx=0x7f67c400ce90 comp rx=0 tx=0).stop 2023-10-06T21:20:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 shutdown_connections 2023-10-06T21:20:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f67b80a1780 0x7f67b80a3c40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67cc10f4f0 0x7f67cc1b1c90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67cc1032c0 0x7f67cc10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 --2- 172.21.15.26:0/2056198302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67cc1083e0 0x7f67cc10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.227+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 >> 172.21.15.26:0/2056198302 conn(0x7f67cc0feb00 msgr2=0x7f67cc106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.230+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 shutdown_connections 2023-10-06T21:20:45.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.230+0000 7f67d47ff700 1 -- 172.21.15.26:0/2056198302 wait complete. 2023-10-06T21:20:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-10-06T21:20:45.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 -- 172.21.15.26:0/207898232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda840753e0 msgr2=0x7fda840757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/207898232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda840753e0 0x7fda840757c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda80009990 tx=0x7fda80009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 -- 172.21.15.26:0/207898232 shutdown_connections 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/207898232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84111d00 0x7fda841140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/207898232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda840753e0 0x7fda840757c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/207898232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda84075d00 0x7fda841117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 -- 172.21.15.26:0/207898232 >> 172.21.15.26:0/207898232 conn(0x7fda840febb0 msgr2=0x7fda84100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 -- 172.21.15.26:0/207898232 shutdown_connections 2023-10-06T21:20:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.631+0000 7fda8b7ff700 1 -- 172.21.15.26:0/207898232 wait complete. 2023-10-06T21:20:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.633+0000 7fda8b7ff700 1 Processor -- start 2023-10-06T21:20:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.633+0000 7fda8b7ff700 1 -- start start 2023-10-06T21:20:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84075d00 0x7fda8419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda84111d00 0x7fda841a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a25a0 con 0x7fda84111d00 2023-10-06T21:20:45.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a2710 con 0x7fda840753e0 2023-10-06T21:20:45.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8b7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841a28d0 con 0x7fda84075d00 2023-10-06T21:20:45.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda88d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84075d00 0x7fda8419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58160/0 (socket says 172.21.15.26:58160) 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 -- 172.21.15.26:0/1757785045 learned_addr learned my addr 172.21.15.26:0/1757785045 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 -- 172.21.15.26:0/1757785045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84075d00 msgr2=0x7fda8419db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84075d00 0x7fda8419db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 -- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda84111d00 msgr2=0x7fda841a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda84111d00 0x7fda841a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.634+0000 7fda8959b700 1 -- 172.21.15.26:0/1757785045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74009520 con 0x7fda840753e0 2023-10-06T21:20:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.635+0000 7fda8959b700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fda80000c00 tx=0x7fda80004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.640+0000 7fda7a7fc700 1 -- 172.21.15.26:0/1757785045 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8001d070 con 0x7fda840753e0 2023-10-06T21:20:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.640+0000 7fda7a7fc700 1 -- 172.21.15.26:0/1757785045 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda80021410 con 0x7fda840753e0 2023-10-06T21:20:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.640+0000 7fda7a7fc700 1 -- 172.21.15.26:0/1757785045 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80017650 con 0x7fda840753e0 2023-10-06T21:20:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.640+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda800095f0 con 0x7fda840753e0 2023-10-06T21:20:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.641+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841aa210 con 0x7fda840753e0 2023-10-06T21:20:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 Processor -- start 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 -- start start 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e50360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.647+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d9d460 con 0x559bb4e340c0 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.648+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d8ec20 con 0x559bb4e349e0 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.648+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d8c540 con 0x559bb4e508a0 2023-10-06T21:20:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.649+0000 7fda7a7fc700 1 -- 172.21.15.26:0/1757785045 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fda80021580 con 0x7fda840753e0 2023-10-06T21:20:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.667+0000 7fda88d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.667+0000 7fda88d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49484/0 (socket says 172.21.15.26:49484) 2023-10-06T21:20:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.667+0000 7fda88d9a700 1 -- 172.21.15.26:0/157185246 learned_addr learned my addr 172.21.15.26:0/157185246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.672+0000 7fda89d9c700 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.673+0000 7fda8959b700 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e50360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.675+0000 7fda7a7fc700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 0x7fda700a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.675+0000 7fda7a7fc700 1 -- 172.21.15.26:0/1757785045 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fda800ce350 con 0x7fda840753e0 2023-10-06T21:20:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.680+0000 7fda88d9a700 1 -- 172.21.15.26:0/157185246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 msgr2=0x559bb4e50360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.680+0000 7fda88d9a700 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e50360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.680+0000 7fda88d9a700 1 -- 172.21.15.26:0/157185246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 msgr2=0x559bb4e344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.680+0000 7fda88d9a700 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.680+0000 7fda88d9a700 1 -- 172.21.15.26:0/157185246 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bb4bb2de0 con 0x559bb4e508a0 2023-10-06T21:20:45.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.682+0000 7fda88d9a700 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda74009eb0 tx=0x7fda74011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d63673b5ca8cb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.683+0000 7fda6f7fe700 1 -- 172.21.15.26:0/157185246 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400e670 con 0x559bb4e508a0 2023-10-06T21:20:45.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.683+0000 7fda6f7fe700 1 -- 172.21.15.26:0/157185246 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda7400ec90 con 0x559bb4e508a0 2023-10-06T21:20:45.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.683+0000 7fda6f7fe700 1 -- 172.21.15.26:0/157185246 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74019da0 con 0x559bb4e508a0 2023-10-06T21:20:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.684+0000 7fda89d9c700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 0x7fda700a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.684+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/157185246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 msgr2=0x559bb4e52c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.684+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda74009eb0 tx=0x7fda74011720 comp rx=0 tx=0).stop 2023-10-06T21:20:45.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.686+0000 7fda89d9c700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 0x7fda700a3c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda84067980 tx=0x7fda7c009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.695+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/157185246 shutdown_connections 2023-10-06T21:20:45.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.695+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e52c90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.695+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.695+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/157185246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e50360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.695+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/157185246 >> 172.21.15.26:0/157185246 conn(0x559bb4da9260 msgr2=0x559bb4e30450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.713+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/157185246 shutdown_connections 2023-10-06T21:20:45.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.713+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/157185246 wait complete. 2023-10-06T21:20:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.714+0000 7fda9f0cfb80 1 -- start start 2023-10-06T21:20:45.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.714+0000 7fda9f0cfb80 1 Processor -- start 2023-10-06T21:20:45.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d87e30 con 0x559bb4e340c0 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d24710 con 0x559bb4e349e0 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.715+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4d243b0 con 0x559bb4e508a0 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.716+0000 7fda9f0cfb80 10 client.0 _ll_register_callbacks cb 0x559bb4b81800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:45.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34104/0 (socket says 172.21.15.26:34104) 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 -- 172.21.15.26:0/2440565809 learned_addr learned my addr 172.21.15.26:0/2440565809 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 -- 172.21.15.26:0/2440565809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 msgr2=0x559bb4e4e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 -- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 msgr2=0x559bb4e4db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 -- 172.21.15.26:0/2440565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda74009b10 con 0x559bb4e340c0 2023-10-06T21:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda8959b700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fda8009b2e0 tx=0x7fda8009b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c7c414b1193944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.717+0000 7fda6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.718+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8009b350 con 0x559bb4e340c0 2023-10-06T21:20:45.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.718+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fda8009f4a0 con 0x559bb4e340c0 2023-10-06T21:20:45.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.720+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bb4e27f10 con 0x559bb4e340c0 2023-10-06T21:20:45.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.721+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda800b1070 con 0x559bb4e340c0 2023-10-06T21:20:45.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.721+0000 7fda9f0cfb80 10 client.16671 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:45.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.721+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559bb4b9bc60 con 0x559bb4e340c0 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bb4e27f10 con 0x559bb4e340c0 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda8009a840 con 0x559bb4e340c0 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 10 client.16671 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 10 client.16671 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 20 client.16671 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4e71070 0x559bb4e73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 4 client.16671 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.722+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bb4d80e10 con 0x559bb4e71070 2023-10-06T21:20:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.723+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4e71070 0x559bb4e73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.724+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4e71070 0x559bb4e73490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a17dfa07255e8ddd server_cookie=8fc1dc0f41a7e9fc in_seq=0 out_seq=0 2023-10-06T21:20:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.724+0000 7fda6cff9700 10 client.16671 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.725+0000 7fda6cff9700 1 -- 172.21.15.26:0/2440565809 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559bb4d80e10 con 0x559bb4e71070 2023-10-06T21:20:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.725+0000 7fda6cff9700 10 client.16671 handle_command_reply: tid=0 2023-10-06T21:20:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.730+0000 7fda9f0cfb80 1 client.16671 shutdown 2023-10-06T21:20:45.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 msgr2=0x559bb4e4d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fda8009b2e0 tx=0x7fda8009b220 comp rx=0 tx=0).stop 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 shutdown_connections 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4e71070 0x559bb4e73490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e340c0 0x559bb4e4d640 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fda8009b2e0 tx=0x7fda8009b220 comp rx=0 tx=0).stop 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2440565809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 >> 172.21.15.26:0/2440565809 conn(0x559bb4da9260 msgr2=0x559bb4e2ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.732+0000 7fda6cff9700 0 client.16671 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.734+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 shutdown_connections 2023-10-06T21:20:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.734+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2440565809 wait complete. 2023-10-06T21:20:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.734+0000 7fda9f0cfb80 20 client.16671 trim_cache size 0 max 16384 2023-10-06T21:20:45.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 Processor -- start 2023-10-06T21:20:45.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 -- start start 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e30670 0x559bb4e4a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4c08920 con 0x559bb4e30670 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4e2f2e0 con 0x559bb4e349e0 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.759+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4e31320 con 0x559bb4e508a0 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58204/0 (socket says 172.21.15.26:58204) 2023-10-06T21:20:45.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda88d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 -- 172.21.15.26:0/340139237 learned_addr learned my addr 172.21.15.26:0/340139237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda89d9c700 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e30670 0x559bb4e4a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 -- 172.21.15.26:0/340139237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 msgr2=0x559bb4e4c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 -- 172.21.15.26:0/340139237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e30670 msgr2=0x559bb4e4a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e30670 0x559bb4e4a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 -- 172.21.15.26:0/340139237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bb4e780c0 con 0x559bb4e349e0 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda88d9a700 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:45.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.760+0000 7fda8959b700 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda800d0040 tx=0x7fda800ce280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87c7e68a8af61d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.761+0000 7fda6ffff700 1 -- 172.21.15.26:0/340139237 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800b1070 con 0x559bb4e349e0 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.761+0000 7fda6ffff700 1 -- 172.21.15.26:0/340139237 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda8009acb0 con 0x559bb4e349e0 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.761+0000 7fda6ffff700 1 -- 172.21.15.26:0/340139237 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8009a9a0 con 0x559bb4e349e0 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.761+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/340139237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 msgr2=0x559bb4e4df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.761+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda800d0040 tx=0x7fda800ce280 comp rx=0 tx=0).stop 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/340139237 shutdown_connections 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e30670 0x559bb4e4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e349e0 0x559bb4e4df20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/340139237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/340139237 >> 172.21.15.26:0/340139237 conn(0x559bb4da9260 msgr2=0x559bb4e4bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/340139237 shutdown_connections 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/340139237 wait complete. 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.762+0000 7fda9f0cfb80 1 -- start start 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 Processor -- start 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 0x559bb4e553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e55920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4e579c0 con 0x559bb4e349e0 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4e55e60 con 0x559bb4e30670 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bb4e55fd0 con 0x559bb4e508a0 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda9f0cfb80 10 client.0 _ll_register_callbacks cb 0x559bb4b81800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda88d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 0x559bb4e553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda8959b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58220/0 (socket says 172.21.15.26:58220) 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.763+0000 7fda8959b700 1 -- 172.21.15.26:0/2547094399 learned_addr learned my addr 172.21.15.26:0/2547094399 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda89d9c700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e55920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 -- 172.21.15.26:0/2547094399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 msgr2=0x559bb4e55920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e55920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 -- 172.21.15.26:0/2547094399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 msgr2=0x559bb4e553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 0x559bb4e553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 -- 172.21.15.26:0/2547094399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bb4bb2de0 con 0x559bb4e30670 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 0x559bb4e553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda8959b700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda80012040 tx=0x7fda8000b2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162fcb70deef4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.764+0000 7fda6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800b1070 con 0x559bb4e30670 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fda800b9070 con 0x559bb4e30670 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bb4e6de20 con 0x559bb4e30670 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda800a3650 con 0x559bb4e30670 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda9f0cfb80 10 client.27002 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.766+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559bb4b9bc60 con 0x559bb4e30670 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.767+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bb4e6de20 con 0x559bb4e30670 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fda800a1460 con 0x559bb4e30670 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 10 client.27002 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 10 client.27002 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 20 client.27002 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4ec5cc0 0x559bb4ec80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 4 client.27002 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.768+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 --> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fda7400ec90 con 0x559bb4ec5cc0 2023-10-06T21:20:45.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.769+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4ec5cc0 0x559bb4ec80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:45.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.770+0000 7fda88d9a700 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4ec5cc0 0x559bb4ec80b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c523ba66c4b9afd5 server_cookie=4cfd29dd4d9bc819 in_seq=0 out_seq=0 2023-10-06T21:20:45.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.771+0000 7fda6cff9700 10 client.27002 ms_handle_connect on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:45.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.771+0000 7fda6cff9700 1 -- 172.21.15.26:0/2547094399 <== mds.2 v2:172.21.15.78:6836/1931966907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fda7400ec90 con 0x559bb4ec5cc0 2023-10-06T21:20:45.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.771+0000 7fda6cff9700 10 client.27002 handle_command_reply: tid=0 2023-10-06T21:20:45.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.771+0000 7fda9f0cfb80 1 client.27002 shutdown 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 msgr2=0x559bb4e570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda80012040 tx=0x7fda8000b2f0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 shutdown_connections 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:6836/1931966907,v1:172.21.15.78:6837/1931966907] conn(0x559bb4ec5cc0 0x559bb4ec80b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bb4e508a0 0x559bb4e55920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559bb4e30670 0x559bb4e570b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 --2- 172.21.15.26:0/2547094399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559bb4e349e0 0x559bb4e553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 >> 172.21.15.26:0/2547094399 conn(0x559bb4da9260 msgr2=0x559bb4e2fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda6cff9700 0 client.27002 ms_handle_reset on v2:172.21.15.78:6836/1931966907 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.773+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 shutdown_connections 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.774+0000 7fda9f0cfb80 1 -- 172.21.15.26:0/2547094399 wait complete. 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.774+0000 7fda9f0cfb80 20 client.27002 trim_cache size 0 max 16384 2023-10-06T21:20:45.777 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:45.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.774+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 msgr2=0x7fda700a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.774+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 0x7fda700a3c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda84067980 tx=0x7fda7c009920 comp rx=0 tx=0).stop 2023-10-06T21:20:45.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 msgr2=0x7fda8419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:45.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fda80000c00 tx=0x7fda80004790 comp rx=0 tx=0).stop 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 shutdown_connections 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fda700a17c0 0x7fda700a3c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda84111d00 0x7fda841a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda840753e0 0x7fda8419d5c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 --2- 172.21.15.26:0/1757785045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda84075d00 0x7fda8419db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 >> 172.21.15.26:0/1757785045 conn(0x7fda840febb0 msgr2=0x7fda84100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 shutdown_connections 2023-10-06T21:20:45.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:45.775+0000 7fda8b7ff700 1 -- 172.21.15.26:0/1757785045 wait complete. 2023-10-06T21:20:45.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-06T21:20:46.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:45 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2056198302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:46.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:45 smithi167 ceph-mon[134494]: pgmap v325: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 826 op/s 2023-10-06T21:20:46.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:45 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2056198302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:46.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:45 smithi026 ceph-mon[98490]: pgmap v325: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 826 op/s 2023-10-06T21:20:46.126 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:45 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2056198302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:20:46.126 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:45 smithi078 ceph-mon[134896]: pgmap v325: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 15 MiB/s wr, 826 op/s 2023-10-06T21:20:46.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.210+0000 7fc9317ff700 1 -- 172.21.15.26:0/537201401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c1051e0 msgr2=0x7fc92c105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.210+0000 7fc9317ff700 1 --2- 172.21.15.26:0/537201401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c1051e0 0x7fc92c105690 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc924009990 tx=0x7fc924009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 -- 172.21.15.26:0/537201401 shutdown_connections 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 --2- 172.21.15.26:0/537201401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c1051e0 0x7fc92c105690 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 --2- 172.21.15.26:0/537201401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 0x7fc92c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 --2- 172.21.15.26:0/537201401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c100700 0x7fc92c104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 -- 172.21.15.26:0/537201401 >> 172.21.15.26:0/537201401 conn(0x7fc92c0fb830 msgr2=0x7fc92c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.211+0000 7fc9317ff700 1 -- 172.21.15.26:0/537201401 shutdown_connections 2023-10-06T21:20:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.212+0000 7fc9317ff700 1 -- 172.21.15.26:0/537201401 wait complete. 2023-10-06T21:20:46.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.213+0000 7fc9317ff700 1 Processor -- start 2023-10-06T21:20:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.213+0000 7fc9317ff700 1 -- start start 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 0x7fc92c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c1051e0 0x7fc92c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a25a0 con 0x7fc92c0ffde0 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a2710 con 0x7fc92c100700 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc9317ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1a28d0 con 0x7fc92c1051e0 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc923fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc923fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58232/0 (socket says 172.21.15.26:58232) 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc923fff700 1 -- 172.21.15.26:0/3328195466 learned_addr learned my addr 172.21.15.26:0/3328195466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c1051e0 0x7fc92c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.214+0000 7fc92affd700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 0x7fc92c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 -- 172.21.15.26:0/3328195466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c1051e0 msgr2=0x7fc92c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c1051e0 0x7fc92c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 -- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 msgr2=0x7fc92c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 0x7fc92c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 -- 172.21.15.26:0/3328195466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240095f0 con 0x7fc92c100700 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc923fff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc928ff9700 1 -- 172.21.15.26:0/3328195466 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400c940 con 0x7fc92c100700 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc928ff9700 1 -- 172.21.15.26:0/3328195466 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914010430 con 0x7fc92c100700 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc928ff9700 1 -- 172.21.15.26:0/3328195466 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914018680 con 0x7fc92c100700 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.215+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1a2bb0 con 0x7fc92c100700 2023-10-06T21:20:46.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.216+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1aa3f0 con 0x7fc92c100700 2023-10-06T21:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.222+0000 7fc928ff9700 1 -- 172.21.15.26:0/3328195466 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc9140105a0 con 0x7fc92c100700 2023-10-06T21:20:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.223+0000 7fc9451cdb80 1 Processor -- start 2023-10-06T21:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.223+0000 7fc9451cdb80 1 -- start start 2023-10-06T21:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dade350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dae0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2da3c460 con 0x561a2dad39e0 2023-10-06T21:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2da2dc20 con 0x561a2dade890 2023-10-06T21:20:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2da2b540 con 0x561a2dad30c0 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49534/0 (socket says 172.21.15.26:49534) 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc92b7fe700 1 -- 172.21.15.26:0/1746994930 learned_addr learned my addr 172.21.15.26:0/1746994930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.224+0000 7fc923fff700 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dae0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc92affd700 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dade350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc928ff9700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 0x7fc9180a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc923fff700 1 -- 172.21.15.26:0/1746994930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 msgr2=0x561a2dad34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc923fff700 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc923fff700 1 -- 172.21.15.26:0/1746994930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 msgr2=0x561a2dade350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc923fff700 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dade350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc928ff9700 1 -- 172.21.15.26:0/3328195466 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc914014070 con 0x7fc92c100700 2023-10-06T21:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc923fff700 1 -- 172.21.15.26:0/1746994930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a2d851de0 con 0x561a2dade890 2023-10-06T21:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.225+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 0x7fc9180a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.226+0000 7fc923fff700 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dae0c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc9140caa20 tx=0x7fc914010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5637bdc07237ba2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.226+0000 7fc9217fa700 1 -- 172.21.15.26:0/1746994930 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914023f10 con 0x561a2dade890 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.226+0000 7fc9217fa700 1 -- 172.21.15.26:0/1746994930 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9140965f0 con 0x561a2dade890 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.226+0000 7fc9217fa700 1 -- 172.21.15.26:0/1746994930 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140d0d60 con 0x561a2dade890 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.227+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1746994930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 msgr2=0x561a2dae0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.227+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dae0c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc9140caa20 tx=0x7fc914010c80 comp rx=0 tx=0).stop 2023-10-06T21:20:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.227+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 0x7fc9180a3b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ba70 tx=0x7fc924009310 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1746994930 shutdown_connections 2023-10-06T21:20:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dae0c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1746994930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dade350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1746994930 >> 172.21.15.26:0/1746994930 conn(0x561a2da48260 msgr2=0x561a2dacf450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1746994930 shutdown_connections 2023-10-06T21:20:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.230+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1746994930 wait complete. 2023-10-06T21:20:46.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.231+0000 7fc9451cdb80 1 -- start start 2023-10-06T21:20:46.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 Processor -- start 2023-10-06T21:20:46.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dadb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dadbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dadc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2da26e30 con 0x561a2dad39e0 2023-10-06T21:20:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2d9c3710 con 0x561a2dade890 2023-10-06T21:20:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.232+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2d9c33b0 con 0x561a2dad30c0 2023-10-06T21:20:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.233+0000 7fc9451cdb80 10 client.0 _ll_register_callbacks cb 0x561a2d820800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.233+0000 7fc92affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dadb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.233+0000 7fc92affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dadb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49548/0 (socket says 172.21.15.26:49548) 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.233+0000 7fc92affd700 1 -- 172.21.15.26:0/906469140 learned_addr learned my addr 172.21.15.26:0/906469140 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.233+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dadc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc923fff700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dadbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 -- 172.21.15.26:0/906469140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 msgr2=0x561a2dadb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dadb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 -- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 msgr2=0x561a2dadbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dadbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 -- 172.21.15.26:0/906469140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1aa3f0 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dadc010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc92400b2c0 tx=0x7fc924005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1059ebb63d9c9708 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.234+0000 7fc9127fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d040 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a2dac6f10 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc92402c040 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc924004b70 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9451cdb80 10 client.27014 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561a2d83ac60 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.235+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a2dac6f10 con 0x561a2dade890 2023-10-06T21:20:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc92400fcb0 con 0x561a2dade890 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 10 client.27014 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 10 client.27014 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 20 client.27014 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2daee230 0x561a2daf0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 4 client.27014 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a2da1fe10 con 0x561a2daee230 2023-10-06T21:20:46.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.236+0000 7fc92affd700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2daee230 0x561a2daf0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.237+0000 7fc92affd700 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2daee230 0x561a2daf0650 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d0c00a309ccfe28 server_cookie=fc840fad123625cf in_seq=0 out_seq=0 2023-10-06T21:20:46.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.237+0000 7fc9127fc700 10 client.27014 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:46.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9127fc700 1 -- 172.21.15.26:0/906469140 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a2da1fe10 con 0x561a2daee230 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9127fc700 10 client.27014 handle_command_reply: tid=0 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 client.27014 shutdown 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 msgr2=0x561a2dadc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dadc010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc92400b2c0 tx=0x7fc924005fb0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 shutdown_connections 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dade890 0x561a2dadc010 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2daee230 0x561a2daf0650 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dadb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/906469140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad39e0 0x561a2dadbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9127fc700 0 client.27014 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:46.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 >> 172.21.15.26:0/906469140 conn(0x561a2da48260 msgr2=0x561a2dacefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 shutdown_connections 2023-10-06T21:20:46.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.238+0000 7fc9451cdb80 1 -- 172.21.15.26:0/906469140 wait complete. 2023-10-06T21:20:46.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.239+0000 7fc9451cdb80 20 client.27014 trim_cache size 0 max 16384 2023-10-06T21:20:46.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.268+0000 7fc9451cdb80 1 Processor -- start 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 -- start start 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2dadbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2d8a7920 con 0x561a2dade890 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2dad7180 con 0x561a2dad39e0 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.273+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2daea150 con 0x561a2dad30c0 2023-10-06T21:20:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.275+0000 7fc923fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2dadbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.275+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.275+0000 7fc92b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34172/0 (socket says 172.21.15.26:34172) 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.275+0000 7fc92b7fe700 1 -- 172.21.15.26:0/168242540 learned_addr learned my addr 172.21.15.26:0/168242540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 -- 172.21.15.26:0/168242540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 msgr2=0x561a2dad6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 -- 172.21.15.26:0/168242540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 msgr2=0x561a2dadbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2dadbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 -- 172.21.15.26:0/168242540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a2d878250 con 0x561a2dade890 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.276+0000 7fc92b7fe700 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc92400bf60 tx=0x7fc92400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56db761568de8268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.277+0000 7fc921ffb700 1 -- 172.21.15.26:0/168242540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d040 con 0x561a2dade890 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.277+0000 7fc921ffb700 1 -- 172.21.15.26:0/168242540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92401e040 con 0x561a2dade890 2023-10-06T21:20:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.277+0000 7fc921ffb700 1 -- 172.21.15.26:0/168242540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400f6a0 con 0x561a2dade890 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.278+0000 7fc9451cdb80 1 -- 172.21.15.26:0/168242540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 msgr2=0x561a2dada450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.278+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc92400bf60 tx=0x7fc92400bf90 comp rx=0 tx=0).stop 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 -- 172.21.15.26:0/168242540 shutdown_connections 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dade890 0x561a2dada450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dad30c0 0x561a2dad6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/168242540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2dadbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 -- 172.21.15.26:0/168242540 >> 172.21.15.26:0/168242540 conn(0x561a2da48260 msgr2=0x561a2dad9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 -- 172.21.15.26:0/168242540 shutdown_connections 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.281+0000 7fc9451cdb80 1 -- 172.21.15.26:0/168242540 wait complete. 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.282+0000 7fc9451cdb80 1 -- start start 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.282+0000 7fc9451cdb80 1 Processor -- start 2023-10-06T21:20:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2db3b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dade890 0x561a2dae3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2dae38a0 con 0x561a2dad30c0 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2dae3a10 con 0x561a2dad39e0 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2dae3bd0 con 0x561a2dade890 2023-10-06T21:20:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.283+0000 7fc9451cdb80 10 client.0 _ll_register_callbacks cb 0x7fc924005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.285+0000 7fc92affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.286+0000 7fc92affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34174/0 (socket says 172.21.15.26:34174) 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.286+0000 7fc92affd700 1 -- 172.21.15.26:0/1906977183 learned_addr learned my addr 172.21.15.26:0/1906977183 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.287+0000 7fc92affd700 1 -- 172.21.15.26:0/1906977183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dade890 msgr2=0x561a2dae3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.287+0000 7fc92affd700 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dade890 0x561a2dae3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.287+0000 7fc92affd700 1 -- 172.21.15.26:0/1906977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 msgr2=0x561a2db3b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.287+0000 7fc92affd700 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2db3b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.287+0000 7fc92affd700 1 -- 172.21.15.26:0/1906977183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a2d83ac60 con 0x561a2dad30c0 2023-10-06T21:20:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.288+0000 7fc92affd700 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x561a2daf0f30 tx=0x7fc91c01fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb1ffde35649c05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.291+0000 7fc9127fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.292+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c004db0 con 0x561a2dad30c0 2023-10-06T21:20:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.292+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc91c01c040 con 0x561a2dad30c0 2023-10-06T21:20:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.292+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9140965f0 con 0x561a2dad30c0 2023-10-06T21:20:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.293+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91c007860 con 0x561a2dad30c0 2023-10-06T21:20:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.293+0000 7fc9451cdb80 10 client.16695 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.293+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561a2d8502b0 con 0x561a2dad30c0 2023-10-06T21:20:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.293+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9140965f0 con 0x561a2dad30c0 2023-10-06T21:20:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc91c017b00 con 0x561a2dad30c0 2023-10-06T21:20:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 10 client.16695 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 10 client.16695 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 20 client.16695 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2db87110 0x561a2db89530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 4 client.16695 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 --> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a2da1fb30 con 0x561a2db87110 2023-10-06T21:20:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.294+0000 7fc923fff700 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2db87110 0x561a2db89530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.295+0000 7fc923fff700 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2db87110 0x561a2db89530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fed8e4a4a7db0a0 server_cookie=5e3ff84bfa8e113e in_seq=0 out_seq=0 2023-10-06T21:20:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.295+0000 7fc9127fc700 10 client.16695 ms_handle_connect on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.295+0000 7fc9127fc700 1 -- 172.21.15.26:0/1906977183 <== mds.1 v2:172.21.15.26:6840/2891556508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a2da1fb30 con 0x561a2db87110 2023-10-06T21:20:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.296+0000 7fc9127fc700 10 client.16695 handle_command_reply: tid=0 2023-10-06T21:20:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.296+0000 7fc9451cdb80 1 client.16695 shutdown 2023-10-06T21:20:46.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.298+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 msgr2=0x561a2db3b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.298+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x561a2daf0f30 tx=0x7fc91c01fca0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 shutdown_connections 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:6840/2891556508,v1:172.21.15.26:6841/2891556508] conn(0x561a2db87110 0x561a2db89530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a2dade890 0x561a2dae3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561a2dad30c0 0x561a2db3b290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 --2- 172.21.15.26:0/1906977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561a2dad39e0 0x561a2db3b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 >> 172.21.15.26:0/1906977183 conn(0x561a2da48260 msgr2=0x561a2dada5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9127fc700 0 client.16695 ms_handle_reset on v2:172.21.15.26:6840/2891556508 2023-10-06T21:20:46.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 shutdown_connections 2023-10-06T21:20:46.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.299+0000 7fc9451cdb80 1 -- 172.21.15.26:0/1906977183 wait complete. 2023-10-06T21:20:46.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.300+0000 7fc9451cdb80 20 client.16695 trim_cache size 0 max 16384 2023-10-06T21:20:46.303 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 msgr2=0x7fc9180a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 0x7fc9180a3b70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc92400ba70 tx=0x7fc924009310 comp rx=0 tx=0).stop 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 msgr2=0x7fc92c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400ed00 comp rx=0 tx=0).stop 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 shutdown_connections 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc92c1051e0 0x7fc92c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc9180a16b0 0x7fc9180a3b70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c0ffde0 0x7fc92c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 --2- 172.21.15.26:0/3328195466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc92c100700 0x7fc92c19db00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 >> 172.21.15.26:0/3328195466 conn(0x7fc92c0fb830 msgr2=0x7fc92c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.301+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 shutdown_connections 2023-10-06T21:20:46.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.302+0000 7fc9317ff700 1 -- 172.21.15.26:0/3328195466 wait complete. 2023-10-06T21:20:46.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-06T21:20:46.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.708+0000 7fa55b1ff700 1 -- 172.21.15.26:0/548580267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 msgr2=0x7fa554107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.708+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/548580267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554107eb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa548009990 tx=0x7fa548009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 -- 172.21.15.26:0/548580267 shutdown_connections 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/548580267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa55411a690 0x7fa55411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/548580267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554107eb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/548580267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5541083f0 0x7fa554108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 -- 172.21.15.26:0/548580267 >> 172.21.15.26:0/548580267 conn(0x7fa55406d1a0 msgr2=0x7fa55406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 -- 172.21.15.26:0/548580267 shutdown_connections 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.710+0000 7fa55b1ff700 1 -- 172.21.15.26:0/548580267 wait complete. 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 Processor -- start 2023-10-06T21:20:46.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 -- start start 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5541083f0 0x7fa554112a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 0x7fa554116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554116980 con 0x7fa55410f640 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554116af0 con 0x7fa55411a690 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.712+0000 7fa55b1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554116cb0 con 0x7fa5541083f0 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.713+0000 7fa55979c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 0x7fa554116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.713+0000 7fa55979c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 0x7fa554116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58272/0 (socket says 172.21.15.26:58272) 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.713+0000 7fa55979c700 1 -- 172.21.15.26:0/3952055252 learned_addr learned my addr 172.21.15.26:0/3952055252 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.713+0000 7fa553fff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 -- 172.21.15.26:0/3952055252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5541083f0 msgr2=0x7fa554112a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5541083f0 0x7fa554112a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 -- 172.21.15.26:0/3952055252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 msgr2=0x7fa554116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 0x7fa554116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 -- 172.21.15.26:0/3952055252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480095f0 con 0x7fa55410f640 2023-10-06T21:20:46.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa553fff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554112f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.714+0000 7fa551ffb700 1 -- 172.21.15.26:0/3952055252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000cbd0 con 0x7fa55410f640 2023-10-06T21:20:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.715+0000 7fa551ffb700 1 -- 172.21.15.26:0/3952055252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54000cd30 con 0x7fa55410f640 2023-10-06T21:20:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.715+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa554116f90 con 0x7fa55410f640 2023-10-06T21:20:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.715+0000 7fa551ffb700 1 -- 172.21.15.26:0/3952055252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540018680 con 0x7fa55410f640 2023-10-06T21:20:46.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.715+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa554117580 con 0x7fa55410f640 2023-10-06T21:20:46.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.723+0000 7fa56ec09b80 1 Processor -- start 2023-10-06T21:20:46.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.724+0000 7fa551ffb700 1 -- 172.21.15.26:0/3952055252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa540010430 con 0x7fa55410f640 2023-10-06T21:20:46.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.724+0000 7fa56ec09b80 1 -- start start 2023-10-06T21:20:46.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.724+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.724+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f254460 con 0x55964f2f6890 2023-10-06T21:20:46.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f245c20 con 0x55964f2eb9e0 2023-10-06T21:20:46.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f243540 con 0x55964f2eb0c0 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49596/0 (socket says 172.21.15.26:49596) 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 -- 172.21.15.26:0/1153308364 learned_addr learned my addr 172.21.15.26:0/1153308364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa558f9b700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa553fff700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 -- 172.21.15.26:0/1153308364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 msgr2=0x55964f2f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.725+0000 7fa55979c700 1 -- 172.21.15.26:0/1153308364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 msgr2=0x55964f2f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa55979c700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa55979c700 1 -- 172.21.15.26:0/1153308364 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55964f069de0 con 0x55964f2eb0c0 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa553fff700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa55979c700 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009710 tx=0x7fa54c0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d9ca146160b8d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa53effd700 1 -- 172.21.15.26:0/1153308364 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c014610 con 0x55964f2eb0c0 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa53effd700 1 -- 172.21.15.26:0/1153308364 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c00f050 con 0x55964f2eb0c0 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa53effd700 1 -- 172.21.15.26:0/1153308364 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c013960 con 0x55964f2eb0c0 2023-10-06T21:20:46.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.726+0000 7fa551ffb700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 0x7fa5440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa551ffb700 1 -- 172.21.15.26:0/3952055252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa540014070 con 0x7fa55410f640 2023-10-06T21:20:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 -- 172.21.15.26:0/1153308364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 msgr2=0x55964f2eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009710 tx=0x7fa54c0077e0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 -- 172.21.15.26:0/1153308364 shutdown_connections 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2eb4a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/1153308364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa56ec09b80 1 -- 172.21.15.26:0/1153308364 >> 172.21.15.26:0/1153308364 conn(0x55964f260260 msgr2=0x55964f2e7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.727+0000 7fa558f9b700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 0x7fa5440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.728+0000 7fa56ec09b80 1 -- 172.21.15.26:0/1153308364 shutdown_connections 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.728+0000 7fa56ec09b80 1 -- 172.21.15.26:0/1153308364 wait complete. 2023-10-06T21:20:46.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.728+0000 7fa558f9b700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 0x7fa5440a3b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa54800b6f0 tx=0x7fa54800b260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.729+0000 7fa56ec09b80 1 -- start start 2023-10-06T21:20:46.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 Processor -- start 2023-10-06T21:20:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f23ee30 con 0x55964f2f6890 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f1db710 con 0x55964f2eb9e0 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f1db3b0 con 0x55964f2eb0c0 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa56ec09b80 10 client.0 _ll_register_callbacks cb 0x55964f038800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa558f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa558f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49604/0 (socket says 172.21.15.26:49604) 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.730+0000 7fa558f9b700 1 -- 172.21.15.26:0/3092078474 learned_addr learned my addr 172.21.15.26:0/3092078474 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa553fff700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa55979c700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 -- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 msgr2=0x55964f2f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 -- 172.21.15.26:0/3092078474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 msgr2=0x55964f2f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 -- 172.21.15.26:0/3092078474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c009a20 con 0x55964f2eb0c0 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa55979c700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa553fff700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.731+0000 7fa558f9b700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa548005470 tx=0x7fa548009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42d669bb0a3f10b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa533fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54802b070 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55964f2def10 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa548004bd0 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa548004d90 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa56ec09b80 10 client.26896 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55964f052c60 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.732+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54803c020 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa5480244a0 con 0x55964f2eb0c0 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 10 client.26896 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:46.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 10 client.26896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 20 client.26896 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f306330 0x55964f308750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 4 client.26896 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55964f237e10 con 0x55964f306330 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa553fff700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f306330 0x55964f308750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa553fff700 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f306330 0x55964f308750 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cd73ccf9a828f5 server_cookie=f4a49a3d7285db5b in_seq=0 out_seq=0 2023-10-06T21:20:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.733+0000 7fa533fff700 10 client.26896 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.734+0000 7fa533fff700 1 -- 172.21.15.26:0/3092078474 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7fa5400b2e90 con 0x55964f306330 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.734+0000 7fa533fff700 10 client.26896 handle_command_reply: tid=0 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.734+0000 7fa56ec09b80 1 client.26896 shutdown 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.734+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 msgr2=0x55964f2f6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.734+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa548005470 tx=0x7fa548009310 comp rx=0 tx=0).stop 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 shutdown_connections 2023-10-06T21:20:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2f6890 0x55964f2f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2eb0c0 0x55964f2f6330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f306330 0x55964f308750 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/3092078474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 >> 172.21.15.26:0/3092078474 conn(0x55964f260260 msgr2=0x55964f2e6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa533fff700 0 client.26896 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.735+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 shutdown_connections 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.736+0000 7fa56ec09b80 1 -- 172.21.15.26:0/3092078474 wait complete. 2023-10-06T21:20:46.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.740+0000 7fa56ec09b80 20 client.26896 trim_cache size 0 max 16384 2023-10-06T21:20:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.759+0000 7fa56ec09b80 1 Processor -- start 2023-10-06T21:20:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.759+0000 7fa56ec09b80 1 -- start start 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 0x55964f2edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f0bf920 con 0x55964f2eb9e0 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f2ecb30 con 0x55964f2eb0c0 2023-10-06T21:20:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.760+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f2f4280 con 0x55964f2f6890 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa553fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49614/0 (socket says 172.21.15.26:49614) 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa553fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34242/0 (socket says 172.21.15.26:34242) 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 -- 172.21.15.26:0/665418039 learned_addr learned my addr 172.21.15.26:0/665418039 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa558f9b700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 0x55964f2edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 -- 172.21.15.26:0/665418039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 msgr2=0x55964f2edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 0x55964f2edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 -- 172.21.15.26:0/665418039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 msgr2=0x55964f2f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa55979c700 1 -- 172.21.15.26:0/665418039 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55964f090250 con 0x55964f2f6890 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa558f9b700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 0x55964f2edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:20:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.761+0000 7fa553fff700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.762+0000 7fa55979c700 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa54c013550 tx=0x7fa54c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb59000c12c3eb03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.762+0000 7fa53f7fe700 1 -- 172.21.15.26:0/665418039 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0137b0 con 0x55964f2f6890 2023-10-06T21:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.762+0000 7fa53f7fe700 1 -- 172.21.15.26:0/665418039 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c022070 con 0x55964f2f6890 2023-10-06T21:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.762+0000 7fa53f7fe700 1 -- 172.21.15.26:0/665418039 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c014ab0 con 0x55964f2f6890 2023-10-06T21:20:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 -- 172.21.15.26:0/665418039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 msgr2=0x55964f2ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa54c013550 tx=0x7fa54c0091d0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 -- 172.21.15.26:0/665418039 shutdown_connections 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2ee8d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb0c0 0x55964f2edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/665418039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb9e0 0x55964f2f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 -- 172.21.15.26:0/665418039 >> 172.21.15.26:0/665418039 conn(0x55964f260260 msgr2=0x55964f302670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 -- 172.21.15.26:0/665418039 shutdown_connections 2023-10-06T21:20:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.763+0000 7fa56ec09b80 1 -- 172.21.15.26:0/665418039 wait complete. 2023-10-06T21:20:46.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.764+0000 7fa56ec09b80 1 -- start start 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.764+0000 7fa56ec09b80 1 Processor -- start 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb0c0 0x55964f2fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f359200 con 0x55964f2eb0c0 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f359340 con 0x55964f2eb9e0 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964f359500 con 0x55964f2f6890 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa56ec09b80 10 client.0 _ll_register_callbacks cb 0x55964f038800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58290/0 (socket says 172.21.15.26:58290) 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 -- 172.21.15.26:0/422901730 learned_addr learned my addr 172.21.15.26:0/422901730 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa558f9b700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb0c0 0x55964f2fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa55979c700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 -- 172.21.15.26:0/422901730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 msgr2=0x55964f2fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 -- 172.21.15.26:0/422901730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb0c0 msgr2=0x55964f2fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb0c0 0x55964f2fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.765+0000 7fa553fff700 1 -- 172.21.15.26:0/422901730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa554117580 con 0x55964f2eb9e0 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.766+0000 7fa553fff700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa54009a0d0 tx=0x7fa54000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1990608245499ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.766+0000 7fa533fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.766+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54009a3d0 con 0x55964f2eb9e0 2023-10-06T21:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.767+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa54009e520 con 0x55964f2eb9e0 2023-10-06T21:20:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.767+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa54c00f050 con 0x55964f2eb9e0 2023-10-06T21:20:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.772+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5400a2760 con 0x55964f2eb9e0 2023-10-06T21:20:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 10 client.27032 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55964f069de0 con 0x55964f2eb9e0 2023-10-06T21:20:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c00f050 con 0x55964f2eb9e0 2023-10-06T21:20:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa5400a22d0 con 0x55964f2eb9e0 2023-10-06T21:20:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 10 client.27032 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 10 client.27032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 20 client.27032 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f35abf0 0x55964f35d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.773+0000 7fa56ec09b80 4 client.27032 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-06T21:20:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.774+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55964f237b30 con 0x55964f35abf0 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.774+0000 7fa558f9b700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f35abf0 0x55964f35d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa558f9b700 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f35abf0 0x55964f35d010 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3814ac14ca409e0 server_cookie=95eb482a3bf3011c in_seq=0 out_seq=0 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa533fff700 10 client.27032 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa533fff700 1 -- 172.21.15.26:0/422901730 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55964f237b30 con 0x55964f35abf0 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa533fff700 10 client.27032 handle_command_reply: tid=0 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa56ec09b80 1 client.27032 shutdown 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 msgr2=0x55964f2fb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.775+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa54009a0d0 tx=0x7fa54000ef90 comp rx=0 tx=0).stop 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 shutdown_connections 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55964f2f6890 0x55964f2fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55964f2eb0c0 0x55964f2fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55964f35abf0 0x55964f35d010 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 --2- 172.21.15.26:0/422901730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55964f2eb9e0 0x55964f2fb900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 >> 172.21.15.26:0/422901730 conn(0x55964f260260 msgr2=0x55964f2ee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa533fff700 0 client.27032 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 shutdown_connections 2023-10-06T21:20:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 1 -- 172.21.15.26:0/422901730 wait complete. 2023-10-06T21:20:46.783 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.776+0000 7fa56ec09b80 20 client.27032 trim_cache size 0 max 16384 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.777+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 msgr2=0x7fa5440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.777+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 0x7fa5440a3b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa54800b6f0 tx=0x7fa54800b260 comp rx=0 tx=0).stop 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.777+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 msgr2=0x7fa554112f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.777+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554112f70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000edb0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 shutdown_connections 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa55411a690 0x7fa554116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5541083f0 0x7fa554112a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa5440a1670 0x7fa5440a3b30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 --2- 172.21.15.26:0/3952055252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa55410f640 0x7fa554112f70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 >> 172.21.15.26:0/3952055252 conn(0x7fa55406d1a0 msgr2=0x7fa55411aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 shutdown_connections 2023-10-06T21:20:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:46.778+0000 7fa55b1ff700 1 -- 172.21.15.26:0/3952055252 wait complete. 2023-10-06T21:20:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 605.0 s, 1085.2 tps, lat 92.143 ms stddev 94.586 2023-10-06T21:20:47.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-06T21:20:47.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:47 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:47.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:47 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:47 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 -- 172.21.15.26:0/3980987682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 msgr2=0x7faa801128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 --2- 172.21.15.26:0/3980987682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa801128c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faa700098f0 tx=0x7faa70009c00 comp rx=0 tx=0).stop 2023-10-06T21:20:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 -- 172.21.15.26:0/3980987682 shutdown_connections 2023-10-06T21:20:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 --2- 172.21.15.26:0/3980987682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa801151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 --2- 172.21.15.26:0/3980987682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa80109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 --2- 172.21.15.26:0/3980987682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa801128c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 -- 172.21.15.26:0/3980987682 >> 172.21.15.26:0/3980987682 conn(0x7faa8006d400 msgr2=0x7faa8006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 -- 172.21.15.26:0/3980987682 shutdown_connections 2023-10-06T21:20:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.202+0000 7faa881ff700 1 -- 172.21.15.26:0/3980987682 wait complete. 2023-10-06T21:20:48.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.203+0000 7faa881ff700 1 Processor -- start 2023-10-06T21:20:48.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.203+0000 7faa881ff700 1 -- start start 2023-10-06T21:20:48.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa8019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa8019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801adcf0 con 0x7faa80109850 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801ade60 con 0x7faa8010a170 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa881ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801ae020 con 0x7faa80112e00 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa8679c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa8019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa8679c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa8019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49646/0 (socket says 172.21.15.26:49646) 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34252/0 (socket says 172.21.15.26:34252) 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa8679c700 1 -- 172.21.15.26:0/144366538 learned_addr learned my addr 172.21.15.26:0/144366538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.204+0000 7faa8579a700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa8019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.205+0000 7faa85f9b700 1 -- 172.21.15.26:0/144366538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 msgr2=0x7faa8019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.205+0000 7faa85f9b700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa8019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.205+0000 7faa85f9b700 1 -- 172.21.15.26:0/144366538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 msgr2=0x7faa8019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.205+0000 7faa85f9b700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa8019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.205+0000 7faa85f9b700 1 -- 172.21.15.26:0/144366538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0095f0 con 0x7faa80109850 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.210+0000 7faa85f9b700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00eb90 tx=0x7faa7c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.210+0000 7faa76ffd700 1 -- 172.21.15.26:0/144366538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c009970 con 0x7faa80109850 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.210+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa70009520 con 0x7faa80109850 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.210+0000 7faa76ffd700 1 -- 172.21.15.26:0/144366538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7c0044e0 con 0x7faa80109850 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.210+0000 7faa76ffd700 1 -- 172.21.15.26:0/144366538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c003e50 con 0x7faa80109850 2023-10-06T21:20:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.211+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801ae6a0 con 0x7faa80109850 2023-10-06T21:20:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.216+0000 7faa9bbfcb80 1 Processor -- start 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.216+0000 7faa76ffd700 1 -- 172.21.15.26:0/144366538 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faa7c004000 con 0x7faa80109850 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.216+0000 7faa9bbfcb80 1 -- start start 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648a140 0x563e6648a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 0x563e66497d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e663f34e0 con 0x563e6648a140 2023-10-06T21:20:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e663e4ca0 con 0x563e66495910 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e663e25c0 con 0x563e6648aa60 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49650/0 (socket says 172.21.15.26:49650) 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.217+0000 7faa85f9b700 1 -- 172.21.15.26:0/3814490116 learned_addr learned my addr 172.21.15.26:0/3814490116 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa8679c700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648a140 0x563e6648a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa8579a700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 0x563e66497d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 -- 172.21.15.26:0/3814490116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 msgr2=0x563e66497d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 0x563e66497d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 -- 172.21.15.26:0/3814490116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648a140 msgr2=0x563e6648a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648a140 0x563e6648a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 -- 172.21.15.26:0/3814490116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e66208e60 con 0x563e6648aa60 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa8579a700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 0x563e66497d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.218+0000 7faa85f9b700 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0cac10 tx=0x7faa7c0d53d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58e31765e6d093d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.219+0000 7faa6bfff700 1 -- 172.21.15.26:0/3814490116 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0cf0d0 con 0x563e6648aa60 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.219+0000 7faa6bfff700 1 -- 172.21.15.26:0/3814490116 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7c0cd810 con 0x563e6648aa60 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.219+0000 7faa6bfff700 1 -- 172.21.15.26:0/3814490116 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0d9540 con 0x563e6648aa60 2023-10-06T21:20:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.219+0000 7faa76ffd700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 0x7faa6c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.219+0000 7faa76ffd700 1 -- 172.21.15.26:0/144366538 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faa7c014070 con 0x7faa80109850 2023-10-06T21:20:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa8679c700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 0x7faa6c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/3814490116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 msgr2=0x563e664953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0cac10 tx=0x7faa7c0d53d0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/3814490116 shutdown_connections 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e66495910 0x563e66497d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648a140 0x563e6648a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/3814490116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664953d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/3814490116 >> 172.21.15.26:0/3814490116 conn(0x563e663ff2e0 msgr2=0x563e664864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.220+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/3814490116 shutdown_connections 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.221+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/3814490116 wait complete. 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.221+0000 7faa8679c700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 0x7faa6c0a3b30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa78009d90 tx=0x7faa7800b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.221+0000 7faa9bbfcb80 1 -- start start 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.222+0000 7faa9bbfcb80 1 Processor -- start 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 0x563e66492ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e663ddeb0 con 0x563e66495910 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e6637a790 con 0x563e6648aa60 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e6637a430 con 0x563e6648a140 2023-10-06T21:20:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa9bbfcb80 10 client.0 _ll_register_callbacks cb 0x563e661d7880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49656/0 (socket says 172.21.15.26:49656) 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa85f9b700 1 -- 172.21.15.26:0/2335449428 learned_addr learned my addr 172.21.15.26:0/2335449428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.223+0000 7faa8579a700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 0x563e66492ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa8679c700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 -- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 msgr2=0x563e66492ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 0x563e66492ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 -- 172.21.15.26:0/2335449428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 msgr2=0x563e664931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 -- 172.21.15.26:0/2335449428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801ae6a0 con 0x563e6648a140 2023-10-06T21:20:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa8579a700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 0x563e66492ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0cb2a0 tx=0x7faa7c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=355d153f0ec81b3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.224+0000 7faa697fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c096870 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faa7c00ce20 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e6647df90 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa7c09a300 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa9bbfcb80 10 client.26914 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.225+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563e661f1ce0 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e6647df90 con 0x563e6648a140 2023-10-06T21:20:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa7c096c40 con 0x563e6648a140 2023-10-06T21:20:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 10 client.26914 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 20 client.26914 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e664ad750 0x563e664afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 4 client.26914 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.226+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e663d6e90 con 0x563e664ad750 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.227+0000 7faa8579a700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e664ad750 0x563e664afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.228+0000 7faa8579a700 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e664ad750 0x563e664afb70 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1b84aeb2da598f server_cookie=6ac0c2b1ff5b1333 in_seq=0 out_seq=0 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.228+0000 7faa697fa700 10 client.26914 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.228+0000 7faa697fa700 1 -- 172.21.15.26:0/2335449428 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e663d6e90 con 0x563e664ad750 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.228+0000 7faa697fa700 10 client.26914 handle_command_reply: tid=0 2023-10-06T21:20:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.228+0000 7faa9bbfcb80 1 client.26914 shutdown 2023-10-06T21:20:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 msgr2=0x563e66492760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0cb2a0 tx=0x7faa7c00cd50 comp rx=0 tx=0).stop 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 shutdown_connections 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648a140 0x563e66492760 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.229+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e664ad750 0x563e664afb70 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2335449428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648aa60 0x563e66492ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 >> 172.21.15.26:0/2335449428 conn(0x563e663ff2e0 msgr2=0x563e66486060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa697fa700 0 client.26914 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 shutdown_connections 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2335449428 wait complete. 2023-10-06T21:20:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.230+0000 7faa9bbfcb80 20 client.26914 trim_cache size 0 max 16384 2023-10-06T21:20:48.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 Processor -- start 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 -- start start 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 0x563e66493010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 0x563e6648f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e6625e9a0 con 0x563e6648aa60 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e66487310 con 0x563e6648a140 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.258+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e6648e5b0 con 0x563e66495910 2023-10-06T21:20:48.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa8679c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 0x563e6648f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa8579a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 0x563e66493010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa8679c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 0x563e6648f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49660/0 (socket says 172.21.15.26:49660) 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa8679c700 1 -- 172.21.15.26:0/2534172921 learned_addr learned my addr 172.21.15.26:0/2534172921 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58334/0 (socket says 172.21.15.26:58334) 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 -- 172.21.15.26:0/2534172921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 msgr2=0x563e6648f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 0x563e6648f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 -- 172.21.15.26:0/2534172921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 msgr2=0x563e66493010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 0x563e66493010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa85f9b700 1 -- 172.21.15.26:0/2534172921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e664821c0 con 0x563e6648a140 2023-10-06T21:20:48.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.259+0000 7faa8579a700 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 0x563e66493010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:48.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.260+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c5b0 tx=0x7faa7c09e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40005236aa2aef80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.260+0000 7faa74ff9700 1 -- 172.21.15.26:0/2534172921 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0cd660 con 0x563e6648a140 2023-10-06T21:20:48.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.260+0000 7faa74ff9700 1 -- 172.21.15.26:0/2534172921 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7c096040 con 0x563e6648a140 2023-10-06T21:20:48.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.260+0000 7faa74ff9700 1 -- 172.21.15.26:0/2534172921 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c09ac30 con 0x563e6648a140 2023-10-06T21:20:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2534172921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 msgr2=0x563e664919f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00c5b0 tx=0x7faa7c09e7a0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2534172921 shutdown_connections 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e66495910 0x563e6648f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664919f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2534172921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e6648aa60 0x563e66493010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2534172921 >> 172.21.15.26:0/2534172921 conn(0x563e663ff2e0 msgr2=0x563e6648c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2534172921 shutdown_connections 2023-10-06T21:20:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.261+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2534172921 wait complete. 2023-10-06T21:20:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.262+0000 7faa9bbfcb80 1 -- start start 2023-10-06T21:20:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.262+0000 7faa9bbfcb80 1 Processor -- start 2023-10-06T21:20:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e664a4120 con 0x563e66495910 2023-10-06T21:20:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e66501570 con 0x563e6648a140 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e665016b0 con 0x563e6648aa60 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa9bbfcb80 10 client.0 _ll_register_callbacks cb 0x7faa78009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa8679c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa8679c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34300/0 (socket says 172.21.15.26:34300) 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.263+0000 7faa8679c700 1 -- 172.21.15.26:0/2535955357 learned_addr learned my addr 172.21.15.26:0/2535955357 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.264+0000 7faa8679c700 1 -- 172.21.15.26:0/2535955357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 msgr2=0x563e664a32d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.264+0000 7faa8679c700 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664a32d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.264+0000 7faa8679c700 1 -- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 msgr2=0x563e664a4e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.264+0000 7faa8679c700 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664a4e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.264+0000 7faa8679c700 1 -- 172.21.15.26:0/2535955357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa80115cf0 con 0x563e66495910 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.265+0000 7faa8679c700 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa78008b70 tx=0x7faa78004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc96a9b43c7772a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.265+0000 7faa697fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.265+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7800e040 con 0x563e66495910 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.265+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faa78003a00 con 0x563e66495910 2023-10-06T21:20:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.266+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa7c0cd810 con 0x563e66495910 2023-10-06T21:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.267+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa78003bc0 con 0x563e66495910 2023-10-06T21:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.267+0000 7faa9bbfcb80 10 client.16731 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:48.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.267+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x563e661f1ce0 con 0x563e66495910 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.267+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa78038020 con 0x563e66495910 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.267+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faa78034a00 con 0x563e66495910 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 10 client.16731 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 20 client.16731 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e66502eb0 0x563e665052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 4 client.16731 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563e663d6bb0 con 0x563e66502eb0 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.268+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e66502eb0 0x563e665052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.269+0000 7faa697fa700 10 client.16731 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.269+0000 7faa85f9b700 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e66502eb0 0x563e665052d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e433c6ca75dac64 server_cookie=2fe792c5590c3207 in_seq=0 out_seq=0 2023-10-06T21:20:48.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.270+0000 7faa697fa700 1 -- 172.21.15.26:0/2535955357 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563e663d6bb0 con 0x563e66502eb0 2023-10-06T21:20:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.270+0000 7faa697fa700 10 client.16731 handle_command_reply: tid=0 2023-10-06T21:20:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.270+0000 7faa9bbfcb80 1 client.16731 shutdown 2023-10-06T21:20:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 msgr2=0x563e664a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa78008b70 tx=0x7faa78004b40 comp rx=0 tx=0).stop 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 shutdown_connections 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e66495910 0x563e664a3a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563e66502eb0 0x563e665052d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563e6648a140 0x563e664a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 --2- 172.21.15.26:0/2535955357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563e6648aa60 0x563e664a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 >> 172.21.15.26:0/2535955357 conn(0x563e663ff2e0 msgr2=0x563e66494000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa697fa700 0 client.16731 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 shutdown_connections 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.271+0000 7faa9bbfcb80 1 -- 172.21.15.26:0/2535955357 wait complete. 2023-10-06T21:20:48.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.272+0000 7faa9bbfcb80 20 client.16731 trim_cache size 0 max 16384 2023-10-06T21:20:48.277 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:48.277 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-10-06T21:20:48.277 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:20:48.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-10-06T21:20:48.277 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:48.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 msgr2=0x7faa6c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 0x7faa6c0a3b30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa78009d90 tx=0x7faa7800b040 comp rx=0 tx=0).stop 2023-10-06T21:20:48.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 msgr2=0x7faa8019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00eb90 tx=0x7faa7c00eea0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 shutdown_connections 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa80112e00 0x7faa8019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faa6c0a1670 0x7faa6c0a3b30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa80109850 0x7faa8019e4b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 --2- 172.21.15.26:0/144366538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8010a170 0x7faa8019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 >> 172.21.15.26:0/144366538 conn(0x7faa8006d400 msgr2=0x7faa8006f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.273+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 shutdown_connections 2023-10-06T21:20:48.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.274+0000 7faa881ff700 1 -- 172.21.15.26:0/144366538 wait complete. 2023-10-06T21:20:48.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:20:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.777+0000 7f3b40fff700 1 -- 172.21.15.26:0/3995983956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c11a690 msgr2=0x7f3b3c11ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.777+0000 7f3b40fff700 1 --2- 172.21.15.26:0/3995983956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c11a690 0x7f3b3c11ca80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009990 tx=0x7f3b30009ca0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.794+0000 7f3b40fff700 1 -- 172.21.15.26:0/3995983956 shutdown_connections 2023-10-06T21:20:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.794+0000 7f3b40fff700 1 --2- 172.21.15.26:0/3995983956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c11a690 0x7f3b3c11ca80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.794+0000 7f3b40fff700 1 --2- 172.21.15.26:0/3995983956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c10f640 0x7f3b3c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.794+0000 7f3b40fff700 1 --2- 172.21.15.26:0/3995983956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3c1083f0 0x7f3b3c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.794+0000 7f3b40fff700 1 -- 172.21.15.26:0/3995983956 >> 172.21.15.26:0/3995983956 conn(0x7f3b3c06d1a0 msgr2=0x7f3b3c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.805+0000 7f3b40fff700 1 -- 172.21.15.26:0/3995983956 shutdown_connections 2023-10-06T21:20:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.805+0000 7f3b40fff700 1 -- 172.21.15.26:0/3995983956 wait complete. 2023-10-06T21:20:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.807+0000 7f3b40fff700 1 Processor -- start 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.807+0000 7f3b40fff700 1 -- start start 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c1083f0 0x7f3b3c113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3c10f640 0x7f3b3c113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c118a30 con 0x7f3b3c10f640 2023-10-06T21:20:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c118ba0 con 0x7f3b3c1083f0 2023-10-06T21:20:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.808+0000 7f3b40fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c118d60 con 0x7f3b3c11a690 2023-10-06T21:20:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.809+0000 7f3b3ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.809+0000 7f3b3ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49672/0 (socket says 172.21.15.26:49672) 2023-10-06T21:20:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.809+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/2728004805 learned_addr learned my addr 172.21.15.26:0/2728004805 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.811+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/2728004805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c1083f0 msgr2=0x7f3b3c113a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:20:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.811+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c1083f0 0x7f3b3c113a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.811+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3c10f640 msgr2=0x7f3b3c113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.811+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3c10f640 0x7f3b3c113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.811+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/2728004805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300095f0 con 0x7f3b3c11a690 2023-10-06T21:20:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.812+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005750 tx=0x7f3b30009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.812+0000 7f3b2b7fe700 1 -- 172.21.15.26:0/2728004805 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3001d070 con 0x7f3b3c11a690 2023-10-06T21:20:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.812+0000 7f3b2b7fe700 1 -- 172.21.15.26:0/2728004805 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b30021410 con 0x7f3b3c11a690 2023-10-06T21:20:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.812+0000 7f3b2b7fe700 1 -- 172.21.15.26:0/2728004805 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30017650 con 0x7f3b3c11a690 2023-10-06T21:20:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.813+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c118fe0 con 0x7f3b3c11a690 2023-10-06T21:20:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.813+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c1bb2f0 con 0x7f3b3c11a690 2023-10-06T21:20:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.821+0000 7f3b5498eb80 1 Processor -- start 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: pgmap v326: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:20:48.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: scrub queued for path: / 2023-10-06T21:20:48.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:48 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:48.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.826+0000 7f3b2b7fe700 1 -- 172.21.15.26:0/2728004805 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3b3000fac0 con 0x7f3b3c11a690 2023-10-06T21:20:48.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.826+0000 7f3b5498eb80 1 -- start start 2023-10-06T21:20:48.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a89e0 0x5600ec4b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec411460 con 0x5600ec4a89e0 2023-10-06T21:20:48.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec402c20 con 0x5600ec4b3890 2023-10-06T21:20:48.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec400540 con 0x5600ec4a80c0 2023-10-06T21:20:48.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.827+0000 7f3b39d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58358/0 (socket says 172.21.15.26:58358) 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1073223709 learned_addr learned my addr 172.21.15.26:0/1073223709 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a89e0 0x5600ec4b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1073223709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 msgr2=0x5600ec4a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1073223709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a89e0 msgr2=0x5600ec4b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a89e0 0x5600ec4b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.828+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1073223709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ec226de0 con 0x5600ec4b3890 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.829+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c013880 tx=0x7f3b2c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a44ac1a5bebfdd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.829+0000 7f3b28ff9700 1 -- 172.21.15.26:0/1073223709 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0132d0 con 0x5600ec4b3890 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.829+0000 7f3b28ff9700 1 -- 172.21.15.26:0/1073223709 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2c019030 con 0x5600ec4b3890 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.829+0000 7f3b28ff9700 1 -- 172.21.15.26:0/1073223709 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c005550 con 0x5600ec4b3890 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.830+0000 7f3b2b7fe700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 0x7f3b240a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.830+0000 7f3b2b7fe700 1 -- 172.21.15.26:0/2728004805 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b300ce290 con 0x7f3b3c11a690 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 0x7f3b240a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1073223709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 msgr2=0x5600ec4b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c013880 tx=0x7f3b2c0096c0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1073223709 shutdown_connections 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4b5c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1073223709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a89e0 0x5600ec4b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.831+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1073223709 >> 172.21.15.26:0/1073223709 conn(0x5600ec41d260 msgr2=0x5600ec4a4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.834+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1073223709 shutdown_connections 2023-10-06T21:20:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.834+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1073223709 wait complete. 2023-10-06T21:20:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.835+0000 7f3b5498eb80 1 -- start start 2023-10-06T21:20:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.835+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 0x7f3b240a3bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009ad0 tx=0x7f3b34009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.835+0000 7f3b5498eb80 1 Processor -- start 2023-10-06T21:20:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4b3890 0x5600ec4b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec3fbe30 con 0x5600ec4b3890 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec398710 con 0x5600ec4a89e0 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec3983b0 con 0x5600ec4a80c0 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b5498eb80 10 client.0 _ll_register_callbacks cb 0x5600ec1f5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.836+0000 7f3b3a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b3a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49684/0 (socket says 172.21.15.26:49684) 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b3a59c700 1 -- 172.21.15.26:0/4112609306 learned_addr learned my addr 172.21.15.26:0/4112609306 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 -- 172.21.15.26:0/4112609306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 msgr2=0x5600ec4b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 -- 172.21.15.26:0/4112609306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4b3890 msgr2=0x5600ec4b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4b3890 0x5600ec4b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.837+0000 7f3b39d9b700 1 -- 172.21.15.26:0/4112609306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c013560 con 0x5600ec4a89e0 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.838+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4b0570 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c013fa0 tx=0x7f3b2c0094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287b009324da1050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.838+0000 7f3b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.838+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c004460 con 0x5600ec4a89e0 2023-10-06T21:20:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.838+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b2c0056c0 con 0x5600ec4a89e0 2023-10-06T21:20:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.838+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600ec49bf10 con 0x5600ec4a89e0 2023-10-06T21:20:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.839+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b2c004830 con 0x5600ec4a89e0 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.839+0000 7f3b5498eb80 10 client.27053 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.839+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5600ec20fc60 con 0x5600ec4a89e0 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600ec49bf10 con 0x5600ec4a89e0 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3b2c003940 con 0x5600ec4a89e0 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 10 client.27053 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 20 client.27053 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec4c3290 0x5600ec4c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 4 client.27053 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.840+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600ec3f4e10 con 0x5600ec4c3290 2023-10-06T21:20:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.841+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec4c3290 0x5600ec4c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.842+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec4c3290 0x5600ec4c56b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbef923a6b548e0 server_cookie=f200b414505e55cc in_seq=0 out_seq=0 2023-10-06T21:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.842+0000 7f3b21ffb700 10 client.27053 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b21ffb700 1 -- 172.21.15.26:0/4112609306 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5600ec3f4e10 con 0x5600ec4c3290 2023-10-06T21:20:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b21ffb700 10 client.27053 handle_command_reply: tid=0 2023-10-06T21:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 client.27053 shutdown 2023-10-06T21:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 msgr2=0x5600ec4b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4b0570 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c013fa0 tx=0x7f3b2c0094b0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 shutdown_connections 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4b3890 0x5600ec4b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a80c0 0x5600ec4b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec4c3290 0x5600ec4c56b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/4112609306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4b0570 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 >> 172.21.15.26:0/4112609306 conn(0x5600ec41d260 msgr2=0x5600ec4a4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.843+0000 7f3b21ffb700 0 client.27053 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.844+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 shutdown_connections 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.844+0000 7f3b5498eb80 1 -- 172.21.15.26:0/4112609306 wait complete. 2023-10-06T21:20:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.844+0000 7f3b5498eb80 20 client.27053 trim_cache size 0 max 16384 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: pgmap v326: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: scrub queued for path: / 2023-10-06T21:20:48.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:48.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:48 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:48.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.874+0000 7f3b5498eb80 1 Processor -- start 2023-10-06T21:20:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 -- start start 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a89e0 0x5600ec4afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4bf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec27c920 con 0x5600ec4a80c0 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec4c5d30 con 0x5600ec4b3890 2023-10-06T21:20:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.876+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec4b5fe0 con 0x5600ec4a89e0 2023-10-06T21:20:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.877+0000 7f3b3a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.877+0000 7f3b39d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a89e0 0x5600ec4afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.877+0000 7f3b3a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ba3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34338/0 (socket says 172.21.15.26:34338) 2023-10-06T21:20:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.877+0000 7f3b3a59c700 1 -- 172.21.15.26:0/3506275671 learned_addr learned my addr 172.21.15.26:0/3506275671 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.877+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.878+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/3506275671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a89e0 msgr2=0x5600ec4afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.878+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a89e0 0x5600ec4afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.878+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/3506275671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 msgr2=0x5600ec4ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.878+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.878+0000 7f3b3ad9d700 1 -- 172.21.15.26:0/3506275671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ec24d250 con 0x5600ec4b3890 2023-10-06T21:20:48.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.880+0000 7f3b3ad9d700 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4bf710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cb970 tx=0x7f3b3009b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4573b82c4af09a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.880+0000 7f3b297fa700 1 -- 172.21.15.26:0/3506275671 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300ce070 con 0x5600ec4b3890 2023-10-06T21:20:48.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.881+0000 7f3b297fa700 1 -- 172.21.15.26:0/3506275671 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b3009f170 con 0x5600ec4b3890 2023-10-06T21:20:48.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.881+0000 7f3b297fa700 1 -- 172.21.15.26:0/3506275671 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000fd70 con 0x5600ec4b3890 2023-10-06T21:20:48.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 -- 172.21.15.26:0/3506275671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 msgr2=0x5600ec4bf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4bf710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cb970 tx=0x7f3b3009b040 comp rx=0 tx=0).stop 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 -- 172.21.15.26:0/3506275671 shutdown_connections 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4b3890 0x5600ec4bf710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/3506275671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4a89e0 0x5600ec4afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 -- 172.21.15.26:0/3506275671 >> 172.21.15.26:0/3506275671 conn(0x5600ec41d260 msgr2=0x5600ec4ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 -- 172.21.15.26:0/3506275671 shutdown_connections 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.882+0000 7f3b5498eb80 1 -- 172.21.15.26:0/3506275671 wait complete. 2023-10-06T21:20:48.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.883+0000 7f3b5498eb80 1 -- start start 2023-10-06T21:20:48.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.883+0000 7f3b5498eb80 1 Processor -- start 2023-10-06T21:20:48.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4b3890 0x5600ec4ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec4acd30 con 0x5600ec4a80c0 2023-10-06T21:20:48.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec4acea0 con 0x5600ec4a89e0 2023-10-06T21:20:48.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ec4ad060 con 0x5600ec4b3890 2023-10-06T21:20:48.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b5498eb80 10 client.0 _ll_register_callbacks cb 0x7f3b2c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.884+0000 7f3b39d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58382/0 (socket says 172.21.15.26:58382) 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1768979685 learned_addr learned my addr 172.21.15.26:0/1768979685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1768979685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4b3890 msgr2=0x5600ec4ac7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4b3890 0x5600ec4ac7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1768979685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 msgr2=0x5600ec4ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.885+0000 7f3b39d9b700 1 -- 172.21.15.26:0/1768979685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c1bb2f0 con 0x5600ec4a89e0 2023-10-06T21:20:48.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.886+0000 7f3b39d9b700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0092a0 tx=0x7f3b2c009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19e73788ce7616d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:48.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.886+0000 7f3b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:48.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.887+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c004460 con 0x5600ec4a89e0 2023-10-06T21:20:48.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.887+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b2c000e40 con 0x5600ec4a89e0 2023-10-06T21:20:48.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.888+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2c019030 con 0x5600ec4a89e0 2023-10-06T21:20:48.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.889+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b2c0045c0 con 0x5600ec4a89e0 2023-10-06T21:20:48.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.889+0000 7f3b5498eb80 10 client.27059 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:48.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.889+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5600ec2252b0 con 0x5600ec4a89e0 2023-10-06T21:20:48.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2c019030 con 0x5600ec4a89e0 2023-10-06T21:20:48.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3b2c011ac0 con 0x5600ec4a89e0 2023-10-06T21:20:48.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 10 client.27059 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:48.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 10 client.27059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:48.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 20 client.27059 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:48.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec517d70 0x5600ec51a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:48.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 4 client.27059 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:20:48.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.890+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600ec3f4b30 con 0x5600ec517d70 2023-10-06T21:20:48.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.891+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec517d70 0x5600ec51a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:48.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.892+0000 7f3b3a59c700 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec517d70 0x5600ec51a190 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865877fd9afbbd20 server_cookie=110beceb07af1011 in_seq=0 out_seq=0 2023-10-06T21:20:48.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.892+0000 7f3b21ffb700 10 client.27059 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.892+0000 7f3b21ffb700 1 -- 172.21.15.26:0/1768979685 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5600ec3f4b30 con 0x5600ec517d70 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.892+0000 7f3b21ffb700 10 client.27059 handle_command_reply: tid=0 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.892+0000 7f3b5498eb80 1 client.27059 shutdown 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 msgr2=0x5600ec4adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0092a0 tx=0x7f3b2c009d40 comp rx=0 tx=0).stop 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 shutdown_connections 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ec4b3890 0x5600ec4ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600ec4a80c0 0x5600ec4ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5600ec517d70 0x5600ec51a190 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 --2- 172.21.15.26:0/1768979685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600ec4a89e0 0x5600ec4adfd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 >> 172.21.15.26:0/1768979685 conn(0x5600ec41d260 msgr2=0x5600ec4bf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b21ffb700 0 client.27059 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:48.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 shutdown_connections 2023-10-06T21:20:48.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.893+0000 7f3b5498eb80 1 -- 172.21.15.26:0/1768979685 wait complete. 2023-10-06T21:20:48.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b5498eb80 20 client.27059 trim_cache size 0 max 16384 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:20:48.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:20:48.899 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:20:48.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 msgr2=0x7f3b240a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 0x7f3b240a3bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009ad0 tx=0x7f3b34009040 comp rx=0 tx=0).stop 2023-10-06T21:20:48.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 msgr2=0x7f3b3c118430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:48.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005750 tx=0x7f3b30009580 comp rx=0 tx=0).stop 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 shutdown_connections 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3c11a690 0x7f3b3c118430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3c1083f0 0x7f3b3c113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b240a1730 0x7f3b240a3bf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 --2- 172.21.15.26:0/2728004805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3c10f640 0x7f3b3c113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.894+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 >> 172.21.15.26:0/2728004805 conn(0x7f3b3c06d1a0 msgr2=0x7f3b3c11aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.895+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 shutdown_connections 2023-10-06T21:20:48.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:20:48.895+0000 7f3b40fff700 1 -- 172.21.15.26:0/2728004805 wait complete. 2023-10-06T21:20:48.935 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: pgmap v326: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 18 MiB/s wr, 1.05k op/s 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: scrub queued for path: / 2023-10-06T21:20:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:49.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:48 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:49.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:49.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:49 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:50.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:50.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:49 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:20:50.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:20:50.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:49 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:20:50.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:50 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:20:50.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:50 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:20:50.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:50 smithi078 ceph-mon[134896]: pgmap v327: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T21:20:51.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:50 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:20:51.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:50 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:20:51.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:50 smithi167 ceph-mon[134494]: pgmap v327: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T21:20:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 -- 172.21.15.26:0/1122821067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc10c880 msgr2=0x7fccfc114400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1122821067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc10c880 0x7fccfc114400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009fc0 tx=0x7fccf400b330 comp rx=0 tx=0).stop 2023-10-06T21:20:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 -- 172.21.15.26:0/1122821067 shutdown_connections 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1122821067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc10c880 0x7fccfc114400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1122821067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.045+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1122821067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccfc1081f0 0x7fccfc10c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.046+0000 7fcd019ff700 1 -- 172.21.15.26:0/1122821067 >> 172.21.15.26:0/1122821067 conn(0x7fccfc076d50 msgr2=0x7fccfc105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.046+0000 7fcd019ff700 1 -- 172.21.15.26:0/1122821067 shutdown_connections 2023-10-06T21:20:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.046+0000 7fcd019ff700 1 -- 172.21.15.26:0/1122821067 wait complete. 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 Processor -- start 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 -- start start 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc10f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccfc10c880 0x7fccfc10ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc1a14d0 con 0x7fccfc107840 2023-10-06T21:20:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc1a1610 con 0x7fccfc19eba0 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.047+0000 7fcd019ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc1a17d0 con 0x7fccfc10c880 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59994/0 (socket says 172.21.15.26:59994) 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 -- 172.21.15.26:0/1736602898 learned_addr learned my addr 172.21.15.26:0/1736602898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfaffd700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc10f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 -- 172.21.15.26:0/1736602898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccfc10c880 msgr2=0x7fccfc10ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccfc10c880 0x7fccfc10ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 -- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 msgr2=0x7fccfc10f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc10f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 -- 172.21.15.26:0/1736602898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4009c40 con 0x7fccfc19eba0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfaffd700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.048+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009520 tx=0x7fccf4004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.049+0000 7fcd009fd700 1 -- 172.21.15.26:0/1736602898 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400e040 con 0x7fccfc19eba0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.049+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc1a1a50 con 0x7fccfc19eba0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.049+0000 7fcd009fd700 1 -- 172.21.15.26:0/1736602898 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf400bbf0 con 0x7fccfc19eba0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.049+0000 7fcd009fd700 1 -- 172.21.15.26:0/1736602898 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40052a0 con 0x7fccfc19eba0 2023-10-06T21:20:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.049+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc1a1f20 con 0x7fccfc19eba0 2023-10-06T21:20:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.056+0000 7fcd15229b80 1 Processor -- start 2023-10-06T21:20:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.058+0000 7fcd15229b80 1 -- start start 2023-10-06T21:20:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabbc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcab18400 con 0x564bcabaf520 2023-10-06T21:20:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcab09bc0 con 0x564bcabaec00 2023-10-06T21:20:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.059+0000 7fcd15229b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcab074e0 con 0x564bcabba3d0 2023-10-06T21:20:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.060+0000 7fcd009fd700 1 -- 172.21.15.26:0/1736602898 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fccf401d470 con 0x7fccfc19eba0 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.060+0000 7fccfa7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.060+0000 7fccfa7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60006/0 (socket says 172.21.15.26:60006) 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.060+0000 7fccfa7fc700 1 -- 172.21.15.26:0/3743060446 learned_addr learned my addr 172.21.15.26:0/3743060446 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.060+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabbc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 -- 172.21.15.26:0/3743060446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 msgr2=0x564bcabbc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabbc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 -- 172.21.15.26:0/3743060446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 msgr2=0x564bcabb9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 -- 172.21.15.26:0/3743060446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bca92dd80 con 0x564bcabaec00 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.061+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fccec0247f0 tx=0x7fccec01a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb7b6fb4dc058a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fcce1ffb700 1 -- 172.21.15.26:0/3743060446 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0240d0 con 0x564bcabaec00 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fcce1ffb700 1 -- 172.21.15.26:0/3743060446 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccec02a030 con 0x564bcabaec00 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fcce1ffb700 1 -- 172.21.15.26:0/3743060446 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec0053a0 con 0x564bcabaec00 2023-10-06T21:20:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fcd009fd700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 0x7fcce40a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fcd009fd700 1 -- 172.21.15.26:0/1736602898 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccf40ce370 con 0x7fccfc19eba0 2023-10-06T21:20:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.062+0000 7fccfaffd700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 0x7fcce40a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.063+0000 7fcd15229b80 1 -- 172.21.15.26:0/3743060446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 msgr2=0x564bcabaefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.063+0000 7fcd15229b80 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fccec0247f0 tx=0x7fccec01a320 comp rx=0 tx=0).stop 2023-10-06T21:20:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 -- 172.21.15.26:0/3743060446 shutdown_connections 2023-10-06T21:20:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabbc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaec00 0x564bcabaefe0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 --2- 172.21.15.26:0/3743060446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 -- 172.21.15.26:0/3743060446 >> 172.21.15.26:0/3743060446 conn(0x564bcab24200 msgr2=0x564bcabab080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fcd15229b80 1 -- 172.21.15.26:0/3743060446 shutdown_connections 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.065+0000 7fccfaffd700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 0x7fcce40a3bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccf00076a0 tx=0x7fccf0007630 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.066+0000 7fcd15229b80 1 -- 172.21.15.26:0/3743060446 wait complete. 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.066+0000 7fcd15229b80 1 -- start start 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.066+0000 7fcd15229b80 1 Processor -- start 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabb9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 0x564bcabb75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcab02dd0 con 0x564bcabaf520 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcaa9f6b0 con 0x564bcabba3d0 2023-10-06T21:20:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcaa9f350 con 0x564bcabaec00 2023-10-06T21:20:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fcd15229b80 10 client.0 _ll_register_callbacks cb 0x564bca8fc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fccfaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabb9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fccfaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabb9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45182/0 (socket says 172.21.15.26:45182) 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.067+0000 7fccfaffd700 1 -- 172.21.15.26:0/4259869323 learned_addr learned my addr 172.21.15.26:0/4259869323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.068+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 0x564bcabb75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.068+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 -- 172.21.15.26:0/4259869323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 msgr2=0x564bcabb9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabb9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 -- 172.21.15.26:0/4259869323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 msgr2=0x564bcabb6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 -- 172.21.15.26:0/4259869323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec0244d0 con 0x564bcabba3d0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 0x564bcabb75f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fccf40cba50 tx=0x7fccf409aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559ea837eff2a69c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.069+0000 7fccdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf409b370 con 0x564bcabba3d0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccf409f4c0 con 0x564bcabba3d0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bcaba2d50 con 0x564bcabba3d0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccf40a3670 con 0x564bcabba3d0 2023-10-06T21:20:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fcd15229b80 10 client.27071 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.070+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564bca916c00 con 0x564bcabba3d0 2023-10-06T21:20:51.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:50 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:20:51.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:50 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:20:51.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:50 smithi026 ceph-mon[98490]: pgmap v327: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bcaba2d50 con 0x564bcabba3d0 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccf409a860 con 0x564bcabba3d0 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 10 client.27071 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 10 client.27071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 20 client.27071 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcabdac40 0x564bcabdd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 4 client.27071 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.071+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bcaafbdb0 con 0x564bcabdac40 2023-10-06T21:20:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.073+0000 7fccfaffd700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcabdac40 0x564bcabdd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.074+0000 7fccfaffd700 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcabdac40 0x564bcabdd060 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1c9b368db46f9a server_cookie=2f83f56042b6036f in_seq=0 out_seq=0 2023-10-06T21:20:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.074+0000 7fccdaffd700 10 client.27071 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.075+0000 7fccdaffd700 1 -- 172.21.15.26:0/4259869323 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564bcaafbdb0 con 0x564bcabdac40 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.075+0000 7fccdaffd700 10 client.27071 handle_command_reply: tid=0 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.075+0000 7fcd15229b80 1 client.27071 shutdown 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.075+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 msgr2=0x564bcabb75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.075+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 0x564bcabb75f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fccf40cba50 tx=0x7fccf409aeb0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 shutdown_connections 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcabdac40 0x564bcabdd060 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabba3d0 0x564bcabb75f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabb9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 --2- 172.21.15.26:0/4259869323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaf520 0x564bcabb6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 >> 172.21.15.26:0/4259869323 conn(0x564bcab24200 msgr2=0x564bcabaac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fccdaffd700 0 client.27071 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 shutdown_connections 2023-10-06T21:20:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 1 -- 172.21.15.26:0/4259869323 wait complete. 2023-10-06T21:20:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.076+0000 7fcd15229b80 20 client.27071 trim_cache size 0 max 16384 2023-10-06T21:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.107+0000 7fcd15229b80 1 Processor -- start 2023-10-06T21:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 -- start start 2023-10-06T21:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaec00 0x564bcabb1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabb9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bca9838c0 con 0x564bcabaec00 2023-10-06T21:20:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcabac6c0 con 0x564bcabaf520 2023-10-06T21:20:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.108+0000 7fcd15229b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcabd8450 con 0x564bcabba3d0 2023-10-06T21:20:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.109+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.109+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45198/0 (socket says 172.21.15.26:45198) 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.109+0000 7fccfb7fe700 1 -- 172.21.15.26:0/482041185 learned_addr learned my addr 172.21.15.26:0/482041185 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.109+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabb9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.109+0000 7fccfaffd700 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaec00 0x564bcabb1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 -- 172.21.15.26:0/482041185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 msgr2=0x564bcabb9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabb9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 -- 172.21.15.26:0/482041185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaec00 msgr2=0x564bcabb1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaec00 0x564bcabb1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 -- 172.21.15.26:0/482041185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bca9541f0 con 0x564bcabba3d0 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097de0 tx=0x7fccf40ce2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db501bdde36e199f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fcce27fc700 1 -- 172.21.15.26:0/482041185 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf409f4c0 con 0x564bcabba3d0 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fcce27fc700 1 -- 172.21.15.26:0/482041185 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf409b370 con 0x564bcabba3d0 2023-10-06T21:20:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.110+0000 7fcce27fc700 1 -- 172.21.15.26:0/482041185 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf409b5d0 con 0x564bcabba3d0 2023-10-06T21:20:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 -- 172.21.15.26:0/482041185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 msgr2=0x564bcabb4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097de0 tx=0x7fccf40ce2a0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 -- 172.21.15.26:0/482041185 shutdown_connections 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabba3d0 0x564bcabb4290 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabaec00 0x564bcabb1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 --2- 172.21.15.26:0/482041185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabb9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.112+0000 7fcd15229b80 1 -- 172.21.15.26:0/482041185 >> 172.21.15.26:0/482041185 conn(0x564bcab24200 msgr2=0x564bcabb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.113+0000 7fcd15229b80 1 -- 172.21.15.26:0/482041185 shutdown_connections 2023-10-06T21:20:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.113+0000 7fcd15229b80 1 -- 172.21.15.26:0/482041185 wait complete. 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.113+0000 7fcd15229b80 1 -- start start 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.113+0000 7fcd15229b80 1 Processor -- start 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabd00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabd0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcac27f70 con 0x564bcabba3d0 2023-10-06T21:20:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcac280e0 con 0x564bcabaf520 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcac282a0 con 0x564bcabaec00 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fcd15229b80 10 client.0 _ll_register_callbacks cb 0x564bca8fc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56508/0 (socket says 172.21.15.26:56508) 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.114+0000 7fccfb7fe700 1 -- 172.21.15.26:0/2844827663 learned_addr learned my addr 172.21.15.26:0/2844827663 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfa7fc700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabd0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfaffd700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabd00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 -- 172.21.15.26:0/2844827663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 msgr2=0x564bcabd00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabd00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 -- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 msgr2=0x564bcabd0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabd0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 -- 172.21.15.26:0/2844827663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bca916c00 con 0x564bcabba3d0 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccfb7fe700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097870 tx=0x7fccf409aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b367e763077350d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.115+0000 7fccdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.116+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf401d930 con 0x564bcabba3d0 2023-10-06T21:20:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.116+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccf40a3670 con 0x564bcabba3d0 2023-10-06T21:20:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.116+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccec02a030 con 0x564bcabba3d0 2023-10-06T21:20:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.119+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccf409f4c0 con 0x564bcabba3d0 2023-10-06T21:20:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.119+0000 7fcd15229b80 10 client.16755 fetch_fsmap learned FSMap version 49 2023-10-06T21:20:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.119+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564bca92c250 con 0x564bcabba3d0 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.119+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccec02a030 con 0x564bcabba3d0 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccf40b93a0 con 0x564bcabba3d0 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 20 client.16755 populate_metadata read hostname 'smithi026' 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcac2f560 0x564bcac31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 4 client.16755 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.120+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bcaafbad0 con 0x564bcac2f560 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.121+0000 7fccfaffd700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcac2f560 0x564bcac31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:20:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.121+0000 7fccdaffd700 10 client.16755 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.121+0000 7fccfaffd700 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcac2f560 0x564bcac31950 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464ec28820512207 server_cookie=8c52287d1a7a069c in_seq=0 out_seq=0 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fccdaffd700 1 -- 172.21.15.26:0/2844827663 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bcaafbad0 con 0x564bcac2f560 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fccdaffd700 10 client.16755 handle_command_reply: tid=0 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 client.16755 shutdown 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 msgr2=0x564bcac27a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097870 tx=0x7fccf409aeb0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 shutdown_connections 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564bcabba3d0 0x564bcac27a00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bcabaec00 0x564bcabd00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564bcac2f560 0x564bcac31950 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 --2- 172.21.15.26:0/2844827663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bcabaf520 0x564bcabd0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 >> 172.21.15.26:0/2844827663 conn(0x564bcab24200 msgr2=0x564bcabd6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fccdaffd700 0 client.16755 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 shutdown_connections 2023-10-06T21:20:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.143+0000 7fcd15229b80 1 -- 172.21.15.26:0/2844827663 wait complete. 2023-10-06T21:20:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.144+0000 7fcd15229b80 20 client.16755 trim_cache size 0 max 16384 2023-10-06T21:20:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:20:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:20:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:20:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:20:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.145+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 msgr2=0x7fcce40a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.145+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 0x7fcce40a3bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccf00076a0 tx=0x7fccf0007630 comp rx=0 tx=0).stop 2023-10-06T21:20:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.145+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 msgr2=0x7fccfc1a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:20:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009520 tx=0x7fccf4004a30 comp rx=0 tx=0).stop 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 shutdown_connections 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccfc19eba0 0x7fccfc1a0f90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcce40a1730 0x7fcce40a3bf0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccf00076a0 tx=0x7fccf0007630 comp rx=0 tx=0).stop 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccfc107840 0x7fccfc10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 --2- 172.21.15.26:0/1736602898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccfc10c880 0x7fccfc10ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 >> 172.21.15.26:0/1736602898 conn(0x7fccfc076d50 msgr2=0x7fccfc077130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:20:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 shutdown_connections 2023-10-06T21:20:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:20:51.146+0000 7fcd019ff700 1 -- 172.21.15.26:0/1736602898 wait complete. 2023-10-06T21:20:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 610.0 s, 1175.6 tps, lat 85.119 ms stddev 93.945 2023-10-06T21:20:53.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:52 smithi167 ceph-mon[134494]: pgmap v328: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-10-06T21:20:53.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:52 smithi026 ceph-mon[98490]: pgmap v328: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-10-06T21:20:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:52 smithi078 ceph-mon[134896]: pgmap v328: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-10-06T21:20:54.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:54 smithi167 ceph-mon[134494]: pgmap v329: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 765 op/s 2023-10-06T21:20:54.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:54 smithi026 ceph-mon[98490]: pgmap v329: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 765 op/s 2023-10-06T21:20:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:54 smithi078 ceph-mon[134896]: pgmap v329: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 765 op/s 2023-10-06T21:20:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:56 smithi167 ceph-mon[134494]: pgmap v330: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-10-06T21:20:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:56 smithi026 ceph-mon[98490]: pgmap v330: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-10-06T21:20:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:56 smithi078 ceph-mon[134896]: pgmap v330: 97 pgs: 97 active+clean; 6.2 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 701 op/s 2023-10-06T21:20:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 615.0 s, 1329.8 tps, lat 75.031 ms stddev 88.582 2023-10-06T21:20:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:20:58 smithi026 ceph-mon[98490]: pgmap v331: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 15 MiB/s wr, 938 op/s 2023-10-06T21:20:58.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:20:58 smithi078 ceph-mon[134896]: pgmap v331: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 15 MiB/s wr, 938 op/s 2023-10-06T21:20:59.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:20:58 smithi167 ceph-mon[134494]: pgmap v331: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 15 MiB/s wr, 938 op/s 2023-10-06T21:21:00.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:00 smithi078 ceph-mon[134896]: pgmap v332: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 696 op/s 2023-10-06T21:21:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:00 smithi167 ceph-mon[134494]: pgmap v332: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 696 op/s 2023-10-06T21:21:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:00 smithi026 ceph-mon[98490]: pgmap v332: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 696 op/s 2023-10-06T21:21:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 620.0 s, 974.8 tps, lat 101.917 ms stddev 133.938 2023-10-06T21:21:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:02 smithi078 ceph-mon[134896]: pgmap v333: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 980 op/s 2023-10-06T21:21:03.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:02 smithi167 ceph-mon[134494]: pgmap v333: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 980 op/s 2023-10-06T21:21:03.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:02 smithi026 ceph-mon[98490]: pgmap v333: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 980 op/s 2023-10-06T21:21:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 -- 172.21.15.26:0/3581845149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e34075d00 msgr2=0x7f5e34111810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/3581845149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e34075d00 0x7f5e34111810 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009990 tx=0x7f5e24009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 -- 172.21.15.26:0/3581845149 shutdown_connections 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/3581845149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e34111d50 0x7f5e34114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/3581845149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e340753e0 0x7f5e340757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/3581845149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e34075d00 0x7f5e34111810 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009990 tx=0x7f5e24009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 -- 172.21.15.26:0/3581845149 >> 172.21.15.26:0/3581845149 conn(0x7f5e340feb60 msgr2=0x7f5e34100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 -- 172.21.15.26:0/3581845149 shutdown_connections 2023-10-06T21:21:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.403+0000 7f5e3bdff700 1 -- 172.21.15.26:0/3581845149 wait complete. 2023-10-06T21:21:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.405+0000 7f5e3bdff700 1 Processor -- start 2023-10-06T21:21:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.405+0000 7f5e3bdff700 1 -- start start 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e34111d50 0x7f5e341a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3419d700 0x7f5e3419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419e180 con 0x7f5e340753e0 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419e2c0 con 0x7f5e3419d700 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.406+0000 7f5e3bdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419e480 con 0x7f5e34111d50 2023-10-06T21:21:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.407+0000 7f5e3a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3419d700 0x7f5e3419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.407+0000 7f5e39b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.407+0000 7f5e39b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57196/0 (socket says 172.21.15.26:57196) 2023-10-06T21:21:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.407+0000 7f5e39b9b700 1 -- 172.21.15.26:0/2758200699 learned_addr learned my addr 172.21.15.26:0/2758200699 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.408+0000 7f5e39b9b700 1 -- 172.21.15.26:0/2758200699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e34111d50 msgr2=0x7f5e341a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.408+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e34111d50 0x7f5e341a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.408+0000 7f5e39b9b700 1 -- 172.21.15.26:0/2758200699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3419d700 msgr2=0x7f5e3419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.408+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3419d700 0x7f5e3419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.408+0000 7f5e39b9b700 1 -- 172.21.15.26:0/2758200699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240095f0 con 0x7f5e340753e0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.409+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d950 tx=0x7f5e3000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.409+0000 7f5e2affd700 1 -- 172.21.15.26:0/2758200699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30009b40 con 0x7f5e340753e0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.409+0000 7f5e2affd700 1 -- 172.21.15.26:0/2758200699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e30009ca0 con 0x7f5e340753e0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.409+0000 7f5e2affd700 1 -- 172.21.15.26:0/2758200699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000f620 con 0x7f5e340753e0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.409+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e341aa710 con 0x7f5e340753e0 2023-10-06T21:21:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.410+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341aabd0 con 0x7f5e340753e0 2023-10-06T21:21:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.414+0000 7f5e2affd700 1 -- 172.21.15.26:0/2758200699 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f5e30010460 con 0x7f5e340753e0 2023-10-06T21:21:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.415+0000 7f5e4f81eb80 1 Processor -- start 2023-10-06T21:21:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.415+0000 7f5e2affd700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 0x7f5e200a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.415+0000 7f5e2affd700 1 -- 172.21.15.26:0/2758200699 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5e300cd400 con 0x7f5e340753e0 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 -- start start 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e3a39c700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 0x7f5e200a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b1307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b08c400 con 0x55973b123520 2023-10-06T21:21:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b07dbc0 con 0x55973b122c00 2023-10-06T21:21:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.416+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b07b4e0 con 0x55973b12e3d0 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57206/0 (socket says 172.21.15.26:57206) 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 -- 172.21.15.26:0/1906063886 learned_addr learned my addr 172.21.15.26:0/1906063886 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3a39c700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 0x7f5e200a3c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e34068d00 tx=0x7f5e2c007270 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3a39c700 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b1307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 -- 172.21.15.26:0/1906063886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 msgr2=0x55973b1307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b1307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 -- 172.21.15.26:0/1906063886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 msgr2=0x55973b122fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b122fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.417+0000 7f5e3939a700 1 -- 172.21.15.26:0/1906063886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973aea1d80 con 0x55973b123520 2023-10-06T21:21:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.418+0000 7f5e3939a700 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e240098e0 tx=0x7f5e240036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdb2cb15d3b28bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.418+0000 7f5e1ffff700 1 -- 172.21.15.26:0/1906063886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2401d070 con 0x55973b123520 2023-10-06T21:21:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.418+0000 7f5e1ffff700 1 -- 172.21.15.26:0/1906063886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2400be00 con 0x55973b123520 2023-10-06T21:21:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.418+0000 7f5e1ffff700 1 -- 172.21.15.26:0/1906063886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24005330 con 0x55973b123520 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1906063886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 msgr2=0x55973b12de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e240098e0 tx=0x7f5e240036a0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1906063886 shutdown_connections 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b1307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1906063886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b123520 0x55973b12de90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1906063886 >> 172.21.15.26:0/1906063886 conn(0x55973b098200 msgr2=0x55973b11f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1906063886 shutdown_connections 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.419+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1906063886 wait complete. 2023-10-06T21:21:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.420+0000 7f5e4f81eb80 1 -- start start 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.420+0000 7f5e4f81eb80 1 Processor -- start 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b122c00 0x55973b12b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12e3d0 0x55973b12bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b076dd0 con 0x55973b122c00 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b0136b0 con 0x55973b12e3d0 2023-10-06T21:21:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b013350 con 0x55973b123520 2023-10-06T21:21:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e4f81eb80 10 client.0 _ll_register_callbacks cb 0x55973ae707a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e3939a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e3939a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51860/0 (socket says 172.21.15.26:51860) 2023-10-06T21:21:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e3939a700 1 -- 172.21.15.26:0/941738410 learned_addr learned my addr 172.21.15.26:0/941738410 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.421+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b122c00 0x55973b12b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 -- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12e3d0 msgr2=0x55973b12bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12e3d0 0x55973b12bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 -- 172.21.15.26:0/941738410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b122c00 msgr2=0x55973b12b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b122c00 0x55973b12b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 -- 172.21.15.26:0/941738410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34114c30 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e3939a700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000c00 tx=0x7f5e24004fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ddb1fef8303aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2401d050 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55973b116d50 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.422+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5e2400bac0 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e240054e0 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 10 client.26953 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55973ae8ac00 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55973b116d50 con 0x55973b123520 2023-10-06T21:21:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5e2400f460 con 0x55973b123520 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 10 client.26953 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 20 client.26953 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b13dad0 0x55973b13fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 4 client.26953 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.423+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55973b06fdb0 con 0x55973b13dad0 2023-10-06T21:21:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.424+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b13dad0 0x55973b13fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.424+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b13dad0 0x55973b13fef0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1694a2c2977217 server_cookie=f2bd243874392cf5 in_seq=0 out_seq=0 2023-10-06T21:21:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.424+0000 7f5e1d7fa700 10 client.26953 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.425+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/941738410 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7f5e300b2e90 con 0x55973b13dad0 2023-10-06T21:21:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.425+0000 7f5e1d7fa700 10 client.26953 handle_command_reply: tid=0 2023-10-06T21:21:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 client.26953 shutdown 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 msgr2=0x55973b12b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5e24000c00 tx=0x7f5e24004fc0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 shutdown_connections 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b13dad0 0x55973b13fef0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12e3d0 0x55973b12bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b122c00 0x55973b12b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/941738410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b123520 0x55973b12b670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e1d7fa700 0 client.26953 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 >> 172.21.15.26:0/941738410 conn(0x55973b098200 msgr2=0x55973b11ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 shutdown_connections 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.426+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/941738410 wait complete. 2023-10-06T21:21:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.427+0000 7f5e4f81eb80 20 client.26953 trim_cache size 0 max 16384 2023-10-06T21:21:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 Processor -- start 2023-10-06T21:21:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 -- start start 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 0x55973b134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12e3d0 0x55973b126430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 0x55973b126970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973aef78c0 con 0x55973b12e3d0 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b11fdc0 con 0x55973b12bdb0 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.445+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b135400 con 0x55973b122c00 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e3a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 0x55973b126970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 0x55973b134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e3a39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 0x55973b126970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51672/0 (socket says 172.21.15.26:51672) 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e3a39c700 1 -- 172.21.15.26:0/1956444371 learned_addr learned my addr 172.21.15.26:0/1956444371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e3939a700 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12e3d0 0x55973b126430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 -- 172.21.15.26:0/1956444371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 msgr2=0x55973b126970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 0x55973b126970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 -- 172.21.15.26:0/1956444371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12e3d0 msgr2=0x55973b126430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12e3d0 0x55973b126430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 -- 172.21.15.26:0/1956444371 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973aec81f0 con 0x55973b122c00 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.446+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 0x55973b134ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e300cab40 tx=0x7f5e30010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e842b6b946e492e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.447+0000 7f5e28bff700 1 -- 172.21.15.26:0/1956444371 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3009aad0 con 0x55973b122c00 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.447+0000 7f5e28bff700 1 -- 172.21.15.26:0/1956444371 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e300b2e90 con 0x55973b122c00 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.447+0000 7f5e28bff700 1 -- 172.21.15.26:0/1956444371 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3009e120 con 0x55973b122c00 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1956444371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 msgr2=0x55973b134ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 0x55973b134ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e300cab40 tx=0x7f5e30010c40 comp rx=0 tx=0).stop 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1956444371 shutdown_connections 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b12bdb0 0x55973b126970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b122c00 0x55973b134ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/1956444371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12e3d0 0x55973b126430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.448+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1956444371 >> 172.21.15.26:0/1956444371 conn(0x55973b098200 msgr2=0x55973b12b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.449+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1956444371 shutdown_connections 2023-10-06T21:21:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.449+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/1956444371 wait complete. 2023-10-06T21:21:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.449+0000 7f5e4f81eb80 1 -- start start 2023-10-06T21:21:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 Processor -- start 2023-10-06T21:21:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 0x55973b185fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b189ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b18a5a0 con 0x55973b12bdb0 2023-10-06T21:21:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b18a710 con 0x55973b122c00 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973b18a8d0 con 0x55973b12e3d0 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.450+0000 7f5e4f81eb80 10 client.0 _ll_register_callbacks cb 0x7f5e24009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51676/0 (socket says 172.21.15.26:51676) 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 -- 172.21.15.26:0/36242796 learned_addr learned my addr 172.21.15.26:0/36242796 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e3a39c700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b189ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e3939a700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 0x55973b185fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 -- 172.21.15.26:0/36242796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 msgr2=0x55973b189ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b189ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 -- 172.21.15.26:0/36242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 msgr2=0x55973b185fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 0x55973b185fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 -- 172.21.15.26:0/36242796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341aabd0 con 0x55973b122c00 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e3939a700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 0x55973b185fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e39b9b700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e3009a310 tx=0x7f5e300a2850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25168367bdc09bc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.451+0000 7f5e1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e300a6070 con 0x55973b122c00 2023-10-06T21:21:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e2400be00 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5e3009aad0 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e30014020 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 10 client.27086 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55973aea0250 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2400be00 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5e300a9020 con 0x55973b122c00 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 10 client.27086 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 10 client.27086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 20 client.27086 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b192fa0 0x55973b1953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 4 client.27086 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.452+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55973b06fad0 con 0x55973b192fa0 2023-10-06T21:21:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.453+0000 7f5e3939a700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b192fa0 0x55973b1953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.453+0000 7f5e3939a700 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b192fa0 0x55973b1953c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c254f089a3a8011 server_cookie=f6698de9be259289 in_seq=0 out_seq=0 2023-10-06T21:21:04.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.453+0000 7f5e1d7fa700 10 client.27086 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.486+0000 7f5e1d7fa700 1 -- 172.21.15.26:0/36242796 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55973b06fad0 con 0x55973b192fa0 2023-10-06T21:21:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.486+0000 7f5e1d7fa700 10 client.27086 handle_command_reply: tid=0 2023-10-06T21:21:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.486+0000 7f5e4f81eb80 1 client.27086 shutdown 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 msgr2=0x55973b185a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e3009a310 tx=0x7f5e300a2850 comp rx=0 tx=0).stop 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 shutdown_connections 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55973b12e3d0 0x55973b189ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55973b192fa0 0x55973b1953c0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55973b122c00 0x55973b185a60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 --2- 172.21.15.26:0/36242796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55973b12bdb0 0x55973b185fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 >> 172.21.15.26:0/36242796 conn(0x55973b098200 msgr2=0x55973b126430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.488+0000 7f5e1d7fa700 0 client.27086 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.489+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 shutdown_connections 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.489+0000 7f5e4f81eb80 1 -- 172.21.15.26:0/36242796 wait complete. 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.489+0000 7f5e4f81eb80 20 client.27086 trim_cache size 0 max 16384 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:21:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:21:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 msgr2=0x7f5e200a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 0x7f5e200a3c00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e34068d00 tx=0x7f5e2c007270 comp rx=0 tx=0).stop 2023-10-06T21:21:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 msgr2=0x7f5e341a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d950 tx=0x7f5e3000dd10 comp rx=0 tx=0).stop 2023-10-06T21:21:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 shutdown_connections 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e3419d700 0x7f5e3419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f5e200a1740 0x7f5e200a3c00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e340753e0 0x7f5e341a38a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 --2- 172.21.15.26:0/2758200699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e34111d50 0x7f5e341a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 >> 172.21.15.26:0/2758200699 conn(0x7f5e340feb60 msgr2=0x7f5e341129c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 shutdown_connections 2023-10-06T21:21:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:04.490+0000 7f5e3bdff700 1 -- 172.21.15.26:0/2758200699 wait complete. 2023-10-06T21:21:04.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:04 smithi167 ceph-mon[134494]: pgmap v334: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 789 op/s 2023-10-06T21:21:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:04 smithi026 ceph-mon[98490]: pgmap v334: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 789 op/s 2023-10-06T21:21:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:04 smithi078 ceph-mon[134896]: pgmap v334: 97 pgs: 97 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 789 op/s 2023-10-06T21:21:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:06 smithi167 ceph-mon[134494]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:21:06.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:06 smithi167 ceph-mon[134494]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-10-06T21:21:06.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:06 smithi026 ceph-mon[98490]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:21:06.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:06 smithi026 ceph-mon[98490]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-10-06T21:21:06.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:06 smithi078 ceph-mon[134896]: osdmap e98: 12 total, 12 up, 12 in 2023-10-06T21:21:06.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:06 smithi078 ceph-mon[134896]: pgmap v336: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 6.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-10-06T21:21:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 625.0 s, 946.0 tps, lat 105.940 ms stddev 145.306 2023-10-06T21:21:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:08 smithi167 ceph-mon[134494]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:08 smithi026 ceph-mon[98490]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:08 smithi078 ceph-mon[134896]: pgmap v337: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:10.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:10 smithi167 ceph-mon[134494]: pgmap v338: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:10.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:10 smithi026 ceph-mon[98490]: pgmap v338: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:10.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:10 smithi078 ceph-mon[134896]: pgmap v338: 97 pgs: 13 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 64 active+clean; 5.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2023-10-06T21:21:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 630.0 s, 1220.8 tps, lat 82.753 ms stddev 99.161 2023-10-06T21:21:12.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:12 smithi167 ceph-mon[134494]: pgmap v339: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:21:12.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:12 smithi026 ceph-mon[98490]: pgmap v339: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:21:12.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:12 smithi078 ceph-mon[134896]: pgmap v339: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 20 MiB/s wr, 1.61k op/s 2023-10-06T21:21:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:14 smithi167 ceph-mon[134494]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:21:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:14 smithi167 ceph-mon[134494]: pgmap v341: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 23 MiB/s wr, 1.80k op/s 2023-10-06T21:21:14.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:14 smithi026 ceph-mon[98490]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:21:14.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:14 smithi026 ceph-mon[98490]: pgmap v341: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 23 MiB/s wr, 1.80k op/s 2023-10-06T21:21:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:14 smithi078 ceph-mon[134896]: osdmap e99: 12 total, 12 up, 12 in 2023-10-06T21:21:14.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:14 smithi078 ceph-mon[134896]: pgmap v341: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 23 MiB/s wr, 1.80k op/s 2023-10-06T21:21:16.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:16 smithi167 ceph-mon[134494]: pgmap v342: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-10-06T21:21:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:16 smithi026 ceph-mon[98490]: pgmap v342: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-10-06T21:21:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:16 smithi078 ceph-mon[134896]: pgmap v342: 97 pgs: 97 active+clean; 5.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-10-06T21:21:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 635.0 s, 1184.8 tps, lat 84.258 ms stddev 117.371 2023-10-06T21:21:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.730+0000 7f25c4dff700 1 -- 172.21.15.26:0/3077364250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073ba0 msgr2=0x7f25c010d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.730+0000 7f25c4dff700 1 --2- 172.21.15.26:0/3077364250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073ba0 0x7f25c010d1b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f25b40098f0 tx=0x7f25b4009c00 comp rx=0 tx=0).stop 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.731+0000 7f25c4dff700 1 -- 172.21.15.26:0/3077364250 shutdown_connections 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.731+0000 7f25c4dff700 1 --2- 172.21.15.26:0/3077364250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c010d6f0 0x7f25c010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.731+0000 7f25c4dff700 1 --2- 172.21.15.26:0/3077364250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073280 0x7f25c0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.731+0000 7f25c4dff700 1 --2- 172.21.15.26:0/3077364250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073ba0 0x7f25c010d1b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.731+0000 7f25c4dff700 1 -- 172.21.15.26:0/3077364250 >> 172.21.15.26:0/3077364250 conn(0x7f25c00fc940 msgr2=0x7f25c00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.732+0000 7f25c4dff700 1 -- 172.21.15.26:0/3077364250 shutdown_connections 2023-10-06T21:21:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.732+0000 7f25c4dff700 1 -- 172.21.15.26:0/3077364250 wait complete. 2023-10-06T21:21:19.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:19.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:18 smithi078 ceph-mon[134896]: pgmap v343: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:19.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:18 smithi167 ceph-mon[134494]: pgmap v343: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.733+0000 7f25c4dff700 1 Processor -- start 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.733+0000 7f25c4dff700 1 -- start start 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073280 0x7f25c010cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c010d6f0 0x7f25c0106c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01073d0 con 0x7f25c010d6f0 2023-10-06T21:21:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0107540 con 0x7f25c0073ba0 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25c4dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0107700 con 0x7f25c0073280 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.734+0000 7f25bdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25be59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073280 0x7f25c010cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59152/0 (socket says 172.21.15.26:59152) 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2885121800 learned_addr learned my addr 172.21.15.26:0/2885121800 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2885121800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073280 msgr2=0x7f25c010cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073280 0x7f25c010cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c010d6f0 msgr2=0x7f25c0106c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c010d6f0 0x7f25c0106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2885121800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4009520 con 0x7f25c0073ba0 2023-10-06T21:21:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.735+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009810 tx=0x7f25b400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.736+0000 7f25af7fe700 1 -- 172.21.15.26:0/2885121800 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b401c070 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.736+0000 7f25af7fe700 1 -- 172.21.15.26:0/2885121800 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b4005040 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.736+0000 7f25af7fe700 1 -- 172.21.15.26:0/2885121800 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4004ac0 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.736+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c01ae610 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.736+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01ae9e0 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.741+0000 7f25af7fe700 1 -- 172.21.15.26:0/2885121800 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f25b4020070 con 0x7f25c0073ba0 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.742+0000 7f25d8815b80 1 Processor -- start 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.742+0000 7f25d8815b80 1 -- start start 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a31bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a326e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a285400 con 0x55581a31c520 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a276bc0 con 0x55581a3273d0 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25d8815b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a2744e0 con 0x55581a31bc00 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.743+0000 7f25bdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59154/0 (socket says 172.21.15.26:59154) 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2480874689 learned_addr learned my addr 172.21.15.26:0/2480874689 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2480874689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 msgr2=0x55581a31bfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a31bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2480874689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 msgr2=0x55581a326e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a326e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25bdd9b700 1 -- 172.21.15.26:0/2480874689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55581a09ad80 con 0x55581a3273d0 2023-10-06T21:21:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.744+0000 7f25af7fe700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 0x7f25a80a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.745+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25b40cb620 tx=0x7f25b40d1e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdf0e279ab95461 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.746+0000 7f25acff9700 1 -- 172.21.15.26:0/2480874689 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4096c10 con 0x55581a3273d0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.746+0000 7f25acff9700 1 -- 172.21.15.26:0/2480874689 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b409add0 con 0x55581a3273d0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.746+0000 7f25acff9700 1 -- 172.21.15.26:0/2480874689 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b409a750 con 0x55581a3273d0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.746+0000 7f25af7fe700 1 -- 172.21.15.26:0/2885121800 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f25b40ce000 con 0x7f25c0073ba0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.747+0000 7f25d8815b80 1 -- 172.21.15.26:0/2480874689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 msgr2=0x55581a3297c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.747+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25b40cb620 tx=0x7f25b40d1e90 comp rx=0 tx=0).stop 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 0x7f25a80a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 -- 172.21.15.26:0/2480874689 shutdown_connections 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a3297c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a31bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2480874689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a326e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 -- 172.21.15.26:0/2480874689 >> 172.21.15.26:0/2480874689 conn(0x55581a291200 msgr2=0x55581a318080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.748+0000 7f25d8815b80 1 -- 172.21.15.26:0/2480874689 shutdown_connections 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.749+0000 7f25d8815b80 1 -- 172.21.15.26:0/2480874689 wait complete. 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.749+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 0x7f25a80a3bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25c006a4d0 tx=0x7f25b8009670 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.750+0000 7f25d8815b80 1 -- start start 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.750+0000 7f25d8815b80 1 Processor -- start 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a3277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a327cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a26fdd0 con 0x55581a31c520 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a20c6b0 con 0x55581a3273d0 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a20c350 con 0x55581a31bc00 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.751+0000 7f25d8815b80 10 client.0 _ll_register_callbacks cb 0x55581a0697a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59170/0 (socket says 172.21.15.26:59170) 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 -- 172.21.15.26:0/2949070627 learned_addr learned my addr 172.21.15.26:0/2949070627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bdd9b700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a327cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25be59c700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a3277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 -- 172.21.15.26:0/2949070627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 msgr2=0x55581a3277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a3277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 -- 172.21.15.26:0/2949070627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 msgr2=0x55581a327cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a327cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.752+0000 7f25bed9d700 1 -- 172.21.15.26:0/2949070627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c01ae9e0 con 0x55581a3273d0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.753+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f25b80179b0 tx=0x7f25b8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcbf5d27bc02690 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.753+0000 7f25a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.753+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b801c070 con 0x55581a3273d0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.753+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55581a30fd50 con 0x55581a3273d0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.753+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f25b80170d0 con 0x55581a3273d0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8017290 con 0x55581a3273d0 2023-10-06T21:21:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 10 client.27098 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55581a083c00 con 0x55581a3273d0 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b802d020 con 0x55581a3273d0 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f25b80144c0 con 0x55581a3273d0 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 10 client.27098 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 10 client.27098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 20 client.27098 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a336d10 0x55581a339130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 4 client.27098 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.754+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55581a268db0 con 0x55581a336d10 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.755+0000 7f25be59c700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a336d10 0x55581a339130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.755+0000 7f25be59c700 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a336d10 0x55581a339130 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80892c49598cadb server_cookie=8b3b7c1cafb6d019 in_seq=0 out_seq=0 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.755+0000 7f25a5ffb700 10 client.27098 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.759+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2949070627 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55581a268db0 con 0x55581a336d10 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.759+0000 7f25a5ffb700 10 client.27098 handle_command_reply: tid=0 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.759+0000 7f25d8815b80 1 client.27098 shutdown 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 msgr2=0x55581a332830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f25b80179b0 tx=0x7f25b8009600 comp rx=0 tx=0).stop 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 shutdown_connections 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a336d10 0x55581a339130 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a3273d0 0x55581a332830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a31bc00 0x55581a3277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2949070627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a327cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.760+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 >> 172.21.15.26:0/2949070627 conn(0x55581a291200 msgr2=0x55581a317c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.761+0000 7f25a5ffb700 0 client.27098 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.761+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 shutdown_connections 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.761+0000 7f25d8815b80 1 -- 172.21.15.26:0/2949070627 wait complete. 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.761+0000 7f25d8815b80 20 client.27098 trim_cache size 0 max 16384 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.793+0000 7f25d8815b80 1 Processor -- start 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.803+0000 7f25d8815b80 1 -- start start 2023-10-06T21:21:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31bc00 0x55581a332e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31c520 0x55581a331f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a0f08c0 con 0x55581a31bc00 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a322400 con 0x55581a31c520 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.804+0000 7f25d8815b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a339790 con 0x55581a3273d0 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.811+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.811+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45330/0 (socket says 172.21.15.26:45330) 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.811+0000 7f25bed9d700 1 -- 172.21.15.26:0/2161233 learned_addr learned my addr 172.21.15.26:0/2161233 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.812+0000 7f25bed9d700 1 -- 172.21.15.26:0/2161233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31c520 msgr2=0x55581a331f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.812+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31c520 0x55581a331f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.812+0000 7f25bed9d700 1 -- 172.21.15.26:0/2161233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31bc00 msgr2=0x55581a332e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.812+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31bc00 0x55581a332e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.812+0000 7f25bed9d700 1 -- 172.21.15.26:0/2161233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55581a0c11f0 con 0x55581a3273d0 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.813+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25b8014820 tx=0x7f25b8017d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17fbacbc3824be3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.813+0000 7f25ad7fa700 1 -- 172.21.15.26:0/2161233 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b801c070 con 0x55581a3273d0 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.813+0000 7f25ad7fa700 1 -- 172.21.15.26:0/2161233 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b802d040 con 0x55581a3273d0 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.813+0000 7f25ad7fa700 1 -- 172.21.15.26:0/2161233 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8003c60 con 0x55581a3273d0 2023-10-06T21:21:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.814+0000 7f25d8815b80 1 -- 172.21.15.26:0/2161233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 msgr2=0x55581a332490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.814+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f25b8014820 tx=0x7f25b8017d40 comp rx=0 tx=0).stop 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.816+0000 7f25d8815b80 1 -- 172.21.15.26:0/2161233 shutdown_connections 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.816+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a332490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.816+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31bc00 0x55581a332e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.816+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2161233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31c520 0x55581a331f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.816+0000 7f25d8815b80 1 -- 172.21.15.26:0/2161233 >> 172.21.15.26:0/2161233 conn(0x55581a291200 msgr2=0x55581a320de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.817+0000 7f25d8815b80 1 -- 172.21.15.26:0/2161233 shutdown_connections 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.817+0000 7f25d8815b80 1 -- 172.21.15.26:0/2161233 wait complete. 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.817+0000 7f25d8815b80 1 -- start start 2023-10-06T21:21:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.818+0000 7f25d8815b80 1 Processor -- start 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.818+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31bc00 0x55581a32c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a328870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a3292f0 con 0x55581a31c520 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a329460 con 0x55581a31bc00 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55581a3295a0 con 0x55581a3273d0 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.819+0000 7f25d8815b80 10 client.0 _ll_register_callbacks cb 0x7f25b8007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45340/0 (socket says 172.21.15.26:45340) 2023-10-06T21:21:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 -- 172.21.15.26:0/2961974653 learned_addr learned my addr 172.21.15.26:0/2961974653 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25be59c700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31bc00 0x55581a32c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 -- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31bc00 msgr2=0x55581a32c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31bc00 0x55581a32c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 -- 172.21.15.26:0/2961974653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 msgr2=0x55581a328870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a328870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.820+0000 7f25bed9d700 1 -- 172.21.15.26:0/2961974653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55581a083c00 con 0x55581a3273d0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.821+0000 7f25bed9d700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25b8014920 tx=0x7f25b80172e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb663c351c5fe772 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.821+0000 7f25a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b801c040 con 0x55581a3273d0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f25b802e080 con 0x55581a3273d0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25b409add0 con 0x55581a3273d0 2023-10-06T21:21:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8014e50 con 0x55581a3273d0 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25d8815b80 10 client.26974 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.822+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55581a099250 con 0x55581a3273d0 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.826+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b409add0 con 0x55581a3273d0 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.826+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f25b80046d0 con 0x55581a3273d0 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 10 client.26974 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 20 client.26974 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a383ad0 0x55581a385ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 4 client.26974 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55581a268ad0 con 0x55581a383ad0 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.827+0000 7f25be59c700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a383ad0 0x55581a385ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.828+0000 7f25be59c700 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a383ad0 0x55581a385ef0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb47e7d7eea8cb0 server_cookie=a0e61f4344870499 in_seq=0 out_seq=0 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.828+0000 7f25a5ffb700 10 client.26974 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25a5ffb700 1 -- 172.21.15.26:0/2961974653 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55581a268ad0 con 0x55581a383ad0 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25a5ffb700 10 client.26974 handle_command_reply: tid=0 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 client.26974 shutdown 2023-10-06T21:21:19.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 msgr2=0x55581a328db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25b8014920 tx=0x7f25b80172e0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 shutdown_connections 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55581a3273d0 0x55581a328db0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.851+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55581a383ad0 0x55581a385ef0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581a31bc00 0x55581a32c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 1 --2- 172.21.15.26:0/2961974653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55581a31c520 0x55581a328870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 >> 172.21.15.26:0/2961974653 conn(0x55581a291200 msgr2=0x55581a31daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25a5ffb700 0 client.26974 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 shutdown_connections 2023-10-06T21:21:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 1 -- 172.21.15.26:0/2961974653 wait complete. 2023-10-06T21:21:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.852+0000 7f25d8815b80 20 client.26974 trim_cache size 0 max 16384 2023-10-06T21:21:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:21:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:21:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:21:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 msgr2=0x7f25a80a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 0x7f25a80a3bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25c006a4d0 tx=0x7f25b8009670 comp rx=0 tx=0).stop 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 msgr2=0x7f25c010d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009810 tx=0x7f25b400fa70 comp rx=0 tx=0).stop 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 shutdown_connections 2023-10-06T21:21:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f25a80a16f0 0x7f25a80a3bb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25c010d6f0 0x7f25c0106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25c0073280 0x7f25c010cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 --2- 172.21.15.26:0/2885121800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f25c0073ba0 0x7f25c010d140 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 >> 172.21.15.26:0/2885121800 conn(0x7f25c00fc940 msgr2=0x7f25c00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 shutdown_connections 2023-10-06T21:21:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:17.853+0000 7f25c4dff700 1 -- 172.21.15.26:0/2885121800 wait complete. 2023-10-06T21:21:19.153 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:18 smithi026 ceph-mon[98490]: pgmap v343: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:19.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.535+0000 7fa7179ff700 1 -- 172.21.15.26:0/657541711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710101aa0 msgr2=0x7fa710105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.535+0000 7fa7179ff700 1 --2- 172.21.15.26:0/657541711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710101aa0 0x7fa710105ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa7040098f0 tx=0x7fa704009c00 comp rx=0 tx=0).stop 2023-10-06T21:21:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 -- 172.21.15.26:0/657541711 shutdown_connections 2023-10-06T21:21:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 --2- 172.21.15.26:0/657541711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 0x7fa71010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 --2- 172.21.15.26:0/657541711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7101010f0 0x7fa7101014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 --2- 172.21.15.26:0/657541711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710101aa0 0x7fa710105ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 -- 172.21.15.26:0/657541711 >> 172.21.15.26:0/657541711 conn(0x7fa7100fc990 msgr2=0x7fa7100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 -- 172.21.15.26:0/657541711 shutdown_connections 2023-10-06T21:21:19.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.536+0000 7fa7179ff700 1 -- 172.21.15.26:0/657541711 wait complete. 2023-10-06T21:21:19.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.538+0000 7fa7179ff700 1 Processor -- start 2023-10-06T21:21:19.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.538+0000 7fa7179ff700 1 -- start start 2023-10-06T21:21:19.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7101010f0 0x7fa710114e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 0x7fa7101153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71011ceb0 con 0x7fa7101010f0 2023-10-06T21:21:19.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710110df0 con 0x7fa710106210 2023-10-06T21:21:19.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa7179ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710110fb0 con 0x7fa710110430 2023-10-06T21:21:19.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa715f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa715f9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45360/0 (socket says 172.21.15.26:45360) 2023-10-06T21:21:19.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa715f9c700 1 -- 172.21.15.26:0/1901849713 learned_addr learned my addr 172.21.15.26:0/1901849713 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.539+0000 7fa714f9a700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 0x7fa7101153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.540+0000 7fa715f9c700 1 -- 172.21.15.26:0/1901849713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 msgr2=0x7fa7101153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.540+0000 7fa715f9c700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 0x7fa7101153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.540+0000 7fa715f9c700 1 -- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7101010f0 msgr2=0x7fa710114e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.540+0000 7fa715f9c700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7101010f0 0x7fa710114e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.540+0000 7fa715f9c700 1 -- 172.21.15.26:0/1901849713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa704009520 con 0x7fa710110430 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa715f9c700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa708007ee0 tx=0x7fa70800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa7027fc700 1 -- 172.21.15.26:0/1901849713 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70800ddd0 con 0x7fa710110430 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa7027fc700 1 -- 172.21.15.26:0/1901849713 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70800f040 con 0x7fa710110430 2023-10-06T21:21:19.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa7027fc700 1 -- 172.21.15.26:0/1901849713 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080048e0 con 0x7fa710110430 2023-10-06T21:21:19.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa710111290 con 0x7fa710110430 2023-10-06T21:21:19.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.541+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101b3060 con 0x7fa710110430 2023-10-06T21:21:19.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.548+0000 7fa72b25cb80 1 Processor -- start 2023-10-06T21:21:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.548+0000 7fa7027fc700 1 -- 172.21.15.26:0/1901849713 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa708003a00 con 0x7fa710110430 2023-10-06T21:21:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.548+0000 7fa72b25cb80 1 -- start start 2023-10-06T21:21:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 0x558737ef24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737efd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737efd890 0x558737effc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737e5b460 con 0x558737efd890 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737e4cc20 con 0x558737ef20c0 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.549+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737e4a540 con 0x558737ef29e0 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.550+0000 7fa71579b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737efd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.550+0000 7fa71579b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737efd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45366/0 (socket says 172.21.15.26:45366) 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.550+0000 7fa71579b700 1 -- 172.21.15.26:0/4167525446 learned_addr learned my addr 172.21.15.26:0/4167525446 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.551+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 0x558737ef24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.551+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737efd890 0x558737effc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 -- 172.21.15.26:0/4167525446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 msgr2=0x558737efd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737efd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 -- 172.21.15.26:0/4167525446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737efd890 msgr2=0x558737effc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737efd890 0x558737effc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 -- 172.21.15.26:0/4167525446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558737c70de0 con 0x558737ef20c0 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa7027fc700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 0x7fa6fc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 0x558737ef24a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa704009230 tx=0x7fa704019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cedbe380514e2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa7027fc700 1 -- 172.21.15.26:0/1901849713 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa7080cc9f0 con 0x7fa710110430 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.552+0000 7fa71579b700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 0x7fa6fc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.553+0000 7fa6fb7fe700 1 -- 172.21.15.26:0/4167525446 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400f460 con 0x558737ef20c0 2023-10-06T21:21:19.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.553+0000 7fa6fb7fe700 1 -- 172.21.15.26:0/4167525446 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70400b500 con 0x558737ef20c0 2023-10-06T21:21:19.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.553+0000 7fa6fb7fe700 1 -- 172.21.15.26:0/4167525446 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704005290 con 0x558737ef20c0 2023-10-06T21:21:19.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.554+0000 7fa71579b700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 0x7fa6fc0a3ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005b40 tx=0x7fa70c005ad0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4167525446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 msgr2=0x558737ef24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 0x558737ef24a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa704009230 tx=0x7fa704019040 comp rx=0 tx=0).stop 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4167525446 shutdown_connections 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737efd890 0x558737effc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef20c0 0x558737ef24a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4167525446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737efd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.557+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4167525446 >> 172.21.15.26:0/4167525446 conn(0x558737e67260 msgr2=0x558737eee450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.558+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4167525446 shutdown_connections 2023-10-06T21:21:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.559+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4167525446 wait complete. 2023-10-06T21:21:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.559+0000 7fa72b25cb80 1 -- start start 2023-10-06T21:21:19.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.560+0000 7fa72b25cb80 1 Processor -- start 2023-10-06T21:21:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef20c0 0x558737efa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef29e0 0x558737efac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737e45e30 con 0x558737ef29e0 2023-10-06T21:21:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737de2710 con 0x558737efd890 2023-10-06T21:21:19.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737de23b0 con 0x558737ef20c0 2023-10-06T21:21:19.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.561+0000 7fa72b25cb80 10 client.0 _ll_register_callbacks cb 0x558737c3f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.563+0000 7fa715f9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.563+0000 7fa715f9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59206/0 (socket says 172.21.15.26:59206) 2023-10-06T21:21:19.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.563+0000 7fa715f9c700 1 -- 172.21.15.26:0/196692954 learned_addr learned my addr 172.21.15.26:0/196692954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.564+0000 7fa715f9c700 1 -- 172.21.15.26:0/196692954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef20c0 msgr2=0x558737efa6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.564+0000 7fa715f9c700 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef20c0 0x558737efa6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.564+0000 7fa715f9c700 1 -- 172.21.15.26:0/196692954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef29e0 msgr2=0x558737efac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.564+0000 7fa715f9c700 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef29e0 0x558737efac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.564+0000 7fa715f9c700 1 -- 172.21.15.26:0/196692954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70401f040 con 0x558737efd890 2023-10-06T21:21:19.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.565+0000 7fa715f9c700 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7080a1820 tx=0x7fa7080995b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52245e27dab30a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.565+0000 7fa6f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:19.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.565+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70809db10 con 0x558737efd890 2023-10-06T21:21:19.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.565+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa7080a1d50 con 0x558737efd890 2023-10-06T21:21:19.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.565+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558737ee5f10 con 0x558737efd890 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa708099c20 con 0x558737efd890 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 10 client.27110 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558737c59c60 con 0x558737efd890 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558737ee5f10 con 0x558737efd890 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa708098f40 con 0x558737efd890 2023-10-06T21:21:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 10 client.27110 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 20 client.27110 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f0d430 0x558737f0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 4 client.27110 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.566+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558737e3ee10 con 0x558737f0d430 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.568+0000 7fa71579b700 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f0d430 0x558737f0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.569+0000 7fa71579b700 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f0d430 0x558737f0f870 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a682d60f160a16 server_cookie=3568b0ea82cbdd5e in_seq=0 out_seq=0 2023-10-06T21:21:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.570+0000 7fa6f8ff9700 10 client.27110 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/196692954 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558737e3ee10 con 0x558737f0d430 2023-10-06T21:21:19.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa6f8ff9700 10 client.27110 handle_command_reply: tid=0 2023-10-06T21:21:19.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 client.27110 shutdown 2023-10-06T21:21:19.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 msgr2=0x558737efb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7080a1820 tx=0x7fa7080995b0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 shutdown_connections 2023-10-06T21:21:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f0d430 0x558737f0f870 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737efb150 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef20c0 0x558737efa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/196692954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef29e0 0x558737efac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa6f8ff9700 0 client.27110 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.571+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 >> 172.21.15.26:0/196692954 conn(0x558737e67260 msgr2=0x558737eedfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.573+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 shutdown_connections 2023-10-06T21:21:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.574+0000 7fa72b25cb80 1 -- 172.21.15.26:0/196692954 wait complete. 2023-10-06T21:21:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.574+0000 7fa72b25cb80 20 client.27110 trim_cache size 0 max 16384 2023-10-06T21:21:19.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.606+0000 7fa72b25cb80 1 Processor -- start 2023-10-06T21:21:19.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 -- start start 2023-10-06T21:21:19.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737ef5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737f04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737ef7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737cc6920 con 0x558737ef20c0 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737eeaa40 con 0x558737efd890 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.607+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737f0aac0 con 0x558737ef29e0 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa714f9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737f04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa714f9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737f04370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45380/0 (socket says 172.21.15.26:45380) 2023-10-06T21:21:19.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa714f9a700 1 -- 172.21.15.26:0/2416042649 learned_addr learned my addr 172.21.15.26:0/2416042649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737ef7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa71579b700 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737ef5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 -- 172.21.15.26:0/2416042649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 msgr2=0x558737f04370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737f04370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 -- 172.21.15.26:0/2416042649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 msgr2=0x558737ef5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737ef5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.608+0000 7fa715f9c700 1 -- 172.21.15.26:0/2416042649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558737eea140 con 0x558737efd890 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.609+0000 7fa715f9c700 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737ef7980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa708096430 tx=0x7fa708097f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ba5267ea08699f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.609+0000 7fa6fbfff700 1 -- 172.21.15.26:0/2416042649 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080b2290 con 0x558737efd890 2023-10-06T21:21:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.609+0000 7fa6fbfff700 1 -- 172.21.15.26:0/2416042649 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7080a1d50 con 0x558737efd890 2023-10-06T21:21:19.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.609+0000 7fa6fbfff700 1 -- 172.21.15.26:0/2416042649 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708099e20 con 0x558737efd890 2023-10-06T21:21:19.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 -- 172.21.15.26:0/2416042649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 msgr2=0x558737ef7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737ef7980 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa708096430 tx=0x7fa708097f70 comp rx=0 tx=0).stop 2023-10-06T21:21:19.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 -- 172.21.15.26:0/2416042649 shutdown_connections 2023-10-06T21:21:19.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737efd890 0x558737ef7980 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737ef5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/2416042649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737ef29e0 0x558737f04370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.611+0000 7fa72b25cb80 1 -- 172.21.15.26:0/2416042649 >> 172.21.15.26:0/2416042649 conn(0x558737e67260 msgr2=0x558737ef59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.612+0000 7fa72b25cb80 1 -- 172.21.15.26:0/2416042649 shutdown_connections 2023-10-06T21:21:19.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.612+0000 7fa72b25cb80 1 -- 172.21.15.26:0/2416042649 wait complete. 2023-10-06T21:21:19.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.613+0000 7fa72b25cb80 1 -- start start 2023-10-06T21:21:19.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.613+0000 7fa72b25cb80 1 Processor -- start 2023-10-06T21:21:19.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737f02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 0x558737f02a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737efd890 0x558737f5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737f5a840 con 0x558737ef20c0 2023-10-06T21:21:19.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737f5a9b0 con 0x558737ef29e0 2023-10-06T21:21:19.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558737f5ab70 con 0x558737efd890 2023-10-06T21:21:19.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa72b25cb80 10 client.0 _ll_register_callbacks cb 0x7fa708095d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:19.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa714f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 0x558737f02a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa714f9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 0x558737f02a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59238/0 (socket says 172.21.15.26:59238) 2023-10-06T21:21:19.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.614+0000 7fa714f9a700 1 -- 172.21.15.26:0/4177605850 learned_addr learned my addr 172.21.15.26:0/4177605850 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:19.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.615+0000 7fa715f9c700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737efd890 0x558737f5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.615+0000 7fa71579b700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737f02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.616+0000 7fa71579b700 1 -- 172.21.15.26:0/4177605850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737efd890 msgr2=0x558737f5a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.616+0000 7fa71579b700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737efd890 0x558737f5a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.616+0000 7fa71579b700 1 -- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 msgr2=0x558737f02a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:19.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.616+0000 7fa71579b700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 0x558737f02a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.616+0000 7fa71579b700 1 -- 172.21.15.26:0/4177605850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558737c59c60 con 0x558737ef20c0 2023-10-06T21:21:19.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.617+0000 7fa71579b700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737f02550 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x558737f10160 tx=0x7fa70c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d717fcad757744 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:19.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.617+0000 7fa6f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:21:19.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.617+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c022040 con 0x558737ef20c0 2023-10-06T21:21:19.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.617+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa70c025570 con 0x558737ef20c0 2023-10-06T21:21:19.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.617+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70400b500 con 0x558737ef20c0 2023-10-06T21:21:19.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa70c025e70 con 0x558737ef20c0 2023-10-06T21:21:19.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 10 client.16791 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:19.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558737c97250 con 0x558737ef20c0 2023-10-06T21:21:19.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70c03b080 con 0x558737ef20c0 2023-10-06T21:21:19.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fa70c037960 con 0x558737ef20c0 2023-10-06T21:21:19.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 10 client.16791 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:19.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:19.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 20 client.16791 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:19.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f623d0 0x558737f647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:19.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 4 client.16791 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:19.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.618+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558737e3eb30 con 0x558737f623d0 2023-10-06T21:21:19.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.620+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f623d0 0x558737f647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:19.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.622+0000 7fa714f9a700 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f623d0 0x558737f647f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8040967e1fe4db8 server_cookie=8d4fda1835b1af6e in_seq=0 out_seq=0 2023-10-06T21:21:19.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.622+0000 7fa6f8ff9700 10 client.16791 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.623+0000 7fa6f8ff9700 1 -- 172.21.15.26:0/4177605850 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558737e3eb30 con 0x558737f623d0 2023-10-06T21:21:19.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.623+0000 7fa6f8ff9700 10 client.16791 handle_command_reply: tid=0 2023-10-06T21:21:19.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.623+0000 7fa72b25cb80 1 client.16791 shutdown 2023-10-06T21:21:19.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.624+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 msgr2=0x558737f02550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.624+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737f02550 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x558737f10160 tx=0x7fa70c004750 comp rx=0 tx=0).stop 2023-10-06T21:21:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 shutdown_connections 2023-10-06T21:21:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558737efd890 0x558737f5a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558737ef20c0 0x558737f02550 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558737f623d0 0x558737f647f0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 --2- 172.21.15.26:0/4177605850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558737ef29e0 0x558737f02a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 >> 172.21.15.26:0/4177605850 conn(0x558737e67260 msgr2=0x558737ef8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa6f8ff9700 0 client.16791 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:19.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 shutdown_connections 2023-10-06T21:21:19.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.625+0000 7fa72b25cb80 1 -- 172.21.15.26:0/4177605850 wait complete. 2023-10-06T21:21:19.636 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:21:19.636 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:21:19.637 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:21:19.637 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:21:19.637 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:21:19.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:21:19.637 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:21:19.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:21:19.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:21:19.638 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:21:19.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.626+0000 7fa72b25cb80 20 client.16791 trim_cache size 0 max 16384 2023-10-06T21:21:19.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.627+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 msgr2=0x7fa6fc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.627+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 0x7fa6fc0a3ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005b40 tx=0x7fa70c005ad0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.627+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 msgr2=0x7fa7101108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:19.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.627+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa708007ee0 tx=0x7fa70800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 shutdown_connections 2023-10-06T21:21:19.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa710110430 0x7fa7101108b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa6fc0a15f0 0x7fa6fc0a3ab0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7101010f0 0x7fa710114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 --2- 172.21.15.26:0/1901849713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa710106210 0x7fa7101153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:19.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 >> 172.21.15.26:0/1901849713 conn(0x7fa7100fc990 msgr2=0x7fa710104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:19.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 shutdown_connections 2023-10-06T21:21:19.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:19.628+0000 7fa7179ff700 1 -- 172.21.15.26:0/1901849713 wait complete. 2023-10-06T21:21:19.686 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:21:20.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:20 smithi167 ceph-mon[134494]: pgmap v344: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:20.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:20 smithi026 ceph-mon[98490]: pgmap v344: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:20.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:20 smithi078 ceph-mon[134896]: pgmap v344: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-10-06T21:21:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 640.0 s, 1257.3 tps, lat 79.378 ms stddev 85.073 2023-10-06T21:21:22.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:22 smithi167 ceph-mon[134494]: pgmap v345: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-10-06T21:21:22.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:22 smithi026 ceph-mon[98490]: pgmap v345: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-10-06T21:21:22.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:22 smithi078 ceph-mon[134896]: pgmap v345: 97 pgs: 97 active+clean; 5.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-10-06T21:21:24.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:24 smithi167 ceph-mon[134494]: pgmap v346: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-10-06T21:21:24.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:24 smithi026 ceph-mon[98490]: pgmap v346: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-10-06T21:21:24.623 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:24 smithi078 ceph-mon[134896]: pgmap v346: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-10-06T21:21:26.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:26 smithi167 ceph-mon[134494]: pgmap v347: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-10-06T21:21:26.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:26 smithi026 ceph-mon[98490]: pgmap v347: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-10-06T21:21:26.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:26 smithi078 ceph-mon[134896]: pgmap v347: 97 pgs: 97 active+clean; 5.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-10-06T21:21:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 645.0 s, 1431.3 tps, lat 68.863 ms stddev 71.665 2023-10-06T21:21:28.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:28 smithi167 ceph-mon[134494]: pgmap v348: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-10-06T21:21:28.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:28 smithi026 ceph-mon[98490]: pgmap v348: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-10-06T21:21:28.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:28 smithi078 ceph-mon[134896]: pgmap v348: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-10-06T21:21:30.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:30 smithi167 ceph-mon[134494]: pgmap v349: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 584 op/s 2023-10-06T21:21:30.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:30 smithi026 ceph-mon[98490]: pgmap v349: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 584 op/s 2023-10-06T21:21:30.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:30 smithi078 ceph-mon[134896]: pgmap v349: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.2 MiB/s wr, 584 op/s 2023-10-06T21:21:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.154+0000 7fcf621ff700 1 -- 172.21.15.26:0/1595955765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c10fa90 msgr2=0x7fcf5c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.154+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1595955765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1083f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009990 tx=0x7fcf4c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.158+0000 7fcf621ff700 1 -- 172.21.15.26:0/1595955765 shutdown_connections 2023-10-06T21:21:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.158+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1595955765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c11a690 0x7fcf5c11ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.158+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1595955765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10f620 0x7fcf5c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.158+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1595955765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1083f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.158+0000 7fcf621ff700 1 -- 172.21.15.26:0/1595955765 >> 172.21.15.26:0/1595955765 conn(0x7fcf5c06cfa0 msgr2=0x7fcf5c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.159+0000 7fcf621ff700 1 -- 172.21.15.26:0/1595955765 shutdown_connections 2023-10-06T21:21:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.161+0000 7fcf621ff700 1 -- 172.21.15.26:0/1595955765 wait complete. 2023-10-06T21:21:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.162+0000 7fcf621ff700 1 Processor -- start 2023-10-06T21:21:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 -- start start 2023-10-06T21:21:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c11a690 0x7fcf5c1aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1ab1b0 con 0x7fcf5c11a690 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1ab320 con 0x7fcf5c10fa90 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.164+0000 7fcf621ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1ab4e0 con 0x7fcf5c10f620 2023-10-06T21:21:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.165+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.165+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38146/0 (socket says 172.21.15.26:38146) 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.165+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/1000281802 learned_addr learned my addr 172.21.15.26:0/1000281802 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.165+0000 7fcf5affd700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/1000281802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10fa90 msgr2=0x7fcf5c1a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c11a690 msgr2=0x7fcf5c1aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c11a690 0x7fcf5c1aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/1000281802 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0095f0 con 0x7fcf5c10f620 2023-10-06T21:21:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.166+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b550 tx=0x7fcf5000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.167+0000 7fcf58ff9700 1 -- 172.21.15.26:0/1000281802 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50011830 con 0x7fcf5c10f620 2023-10-06T21:21:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.167+0000 7fcf58ff9700 1 -- 172.21.15.26:0/1000281802 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf50011e50 con 0x7fcf5c10f620 2023-10-06T21:21:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.167+0000 7fcf58ff9700 1 -- 172.21.15.26:0/1000281802 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000f2e0 con 0x7fcf5c10f620 2023-10-06T21:21:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.167+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c1ab7c0 con 0x7fcf5c10f620 2023-10-06T21:21:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.167+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c110050 con 0x7fcf5c10f620 2023-10-06T21:21:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.175+0000 7fcf75bb4b80 1 Processor -- start 2023-10-06T21:21:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 -- start start 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf58ff9700 1 -- 172.21.15.26:0/1000281802 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcf50011990 con 0x7fcf5c10f620 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c801400 con 0x56174c898520 2023-10-06T21:21:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c7f2bc0 con 0x56174c8a33d0 2023-10-06T21:21:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.178+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c7f04e0 con 0x56174c897c00 2023-10-06T21:21:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38148/0 (socket says 172.21.15.26:38148) 2023-10-06T21:21:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 -- 172.21.15.26:0/149350591 learned_addr learned my addr 172.21.15.26:0/149350591 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5affd700 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 -- 172.21.15.26:0/149350591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 msgr2=0x56174c8a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.179+0000 7fcf5bfff700 1 -- 172.21.15.26:0/149350591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 msgr2=0x56174c8a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.181+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.181+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.181+0000 7fcf5bfff700 1 -- 172.21.15.26:0/149350591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c616d80 con 0x56174c897c00 2023-10-06T21:21:31.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.181+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009cd0 tx=0x7fcf5400b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f1f7f6ddd78a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.182+0000 7fcf41ffb700 1 -- 172.21.15.26:0/149350591 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400e070 con 0x56174c897c00 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.182+0000 7fcf41ffb700 1 -- 172.21.15.26:0/149350591 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf5400be20 con 0x56174c897c00 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.182+0000 7fcf41ffb700 1 -- 172.21.15.26:0/149350591 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54011390 con 0x56174c897c00 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.182+0000 7fcf58ff9700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 0x7fcf440a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf58ff9700 1 -- 172.21.15.26:0/1000281802 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf500ccaa0 con 0x7fcf5c10f620 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 0x7fcf440a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/149350591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 msgr2=0x56174c897fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009cd0 tx=0x7fcf5400b050 comp rx=0 tx=0).stop 2023-10-06T21:21:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/149350591 shutdown_connections 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c897fe0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/149350591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.183+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/149350591 >> 172.21.15.26:0/149350591 conn(0x56174c80d200 msgr2=0x56174c894080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.184+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/149350591 shutdown_connections 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.184+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/149350591 wait complete. 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.184+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 0x7fcf440a3c40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf50000c00 tx=0x7fcf500d1040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.184+0000 7fcf75bb4b80 1 -- start start 2023-10-06T21:21:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.185+0000 7fcf75bb4b80 1 Processor -- start 2023-10-06T21:21:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.185+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c7ebdd0 con 0x56174c898520 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c7886b0 con 0x56174c8a33d0 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c788350 con 0x56174c897c00 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf75bb4b80 10 client.0 _ll_register_callbacks cb 0x56174c5e57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:38154/0 (socket says 172.21.15.26:38154) 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3981647554 learned_addr learned my addr 172.21.15.26:0/3981647554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.186+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 msgr2=0x56174c8a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3981647554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 msgr2=0x56174c8a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3981647554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf54009980 con 0x56174c897c00 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.187+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400bfd0 tx=0x7fcf54007150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3652397cc7305027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf3affd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400e040 con 0x56174c897c00 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf540119a0 con 0x56174c897c00 2023-10-06T21:21:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56174c88bd50 con 0x56174c897c00 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf54011b80 con 0x56174c897c00 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf75bb4b80 10 client.26998 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.188+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56174c5ffc00 con 0x56174c897c00 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56174c88bd50 con 0x56174c897c00 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf5401eaa0 con 0x56174c897c00 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 10 client.26998 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 10 client.26998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 20 client.26998 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c8b2dd0 0x56174c8b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 4 client.26998 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56174c7e4db0 con 0x56174c8b2dd0 2023-10-06T21:21:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.189+0000 7fcf5affd700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c8b2dd0 0x56174c8b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.190+0000 7fcf3affd700 10 client.26998 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.191+0000 7fcf5affd700 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c8b2dd0 0x56174c8b51f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1239b1484a65acce server_cookie=b003397a075acd00 in_seq=0 out_seq=0 2023-10-06T21:21:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.191+0000 7fcf3affd700 1 -- 172.21.15.26:0/3981647554 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56174c7e4db0 con 0x56174c8b2dd0 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.191+0000 7fcf3affd700 10 client.26998 handle_command_reply: tid=0 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.191+0000 7fcf75bb4b80 1 client.26998 shutdown 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 msgr2=0x56174c8a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400bfd0 tx=0x7fcf54007150 comp rx=0 tx=0).stop 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 shutdown_connections 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c8b2dd0 0x56174c8b51f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c8a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c897c00 0x56174c8a2e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3981647554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c898520 0x56174c8a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.192+0000 7fcf3affd700 0 client.26998 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.193+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 >> 172.21.15.26:0/3981647554 conn(0x56174c80d200 msgr2=0x56174c893c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.193+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 shutdown_connections 2023-10-06T21:21:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.193+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3981647554 wait complete. 2023-10-06T21:21:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.193+0000 7fcf75bb4b80 20 client.26998 trim_cache size 0 max 16384 2023-10-06T21:21:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.216+0000 7fcf75bb4b80 1 Processor -- start 2023-10-06T21:21:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 -- start start 2023-10-06T21:21:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c8a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c8a33d0 0x56174c89b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c66c8c0 con 0x56174c8a33d0 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c8aa4e0 con 0x56174c897c00 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.217+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c892e60 con 0x56174c898520 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.218+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.218+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41064/0 (socket says 172.21.15.26:41064) 2023-10-06T21:21:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.218+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3903160646 learned_addr learned my addr 172.21.15.26:0/3903160646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3903160646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 msgr2=0x56174c8a2930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c8a2930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3903160646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c8a33d0 msgr2=0x56174c89b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c8a33d0 0x56174c89b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 -- 172.21.15.26:0/3903160646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c63d1f0 con 0x56174c897c00 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.219+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b420 tx=0x7fcf5401c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3030eafbf6639593 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.220+0000 7fcf427fc700 1 -- 172.21.15.26:0/3903160646 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400e030 con 0x56174c897c00 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.220+0000 7fcf427fc700 1 -- 172.21.15.26:0/3903160646 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf54012d50 con 0x56174c897c00 2023-10-06T21:21:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.220+0000 7fcf427fc700 1 -- 172.21.15.26:0/3903160646 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54011a00 con 0x56174c897c00 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3903160646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 msgr2=0x56174c8a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b420 tx=0x7fcf5401c460 comp rx=0 tx=0).stop 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3903160646 shutdown_connections 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c8a33d0 0x56174c89b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c897c00 0x56174c8a9f40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/3903160646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c8a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3903160646 >> 172.21.15.26:0/3903160646 conn(0x56174c80d200 msgr2=0x56174c8ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3903160646 shutdown_connections 2023-10-06T21:21:31.228 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.222+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/3903160646 wait complete. 2023-10-06T21:21:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.223+0000 7fcf75bb4b80 1 -- start start 2023-10-06T21:21:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.223+0000 7fcf75bb4b80 1 Processor -- start 2023-10-06T21:21:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c897c00 0x56174c8a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c89c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c89cdd0 con 0x56174c897c00 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c89cf40 con 0x56174c8a33d0 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.224+0000 7fcf75bb4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c89d100 con 0x56174c898520 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf75bb4b80 10 client.0 _ll_register_callbacks cb 0x7fcf54012740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf5bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41078/0 (socket says 172.21.15.26:41078) 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf5bfff700 1 -- 172.21.15.26:0/2503373658 learned_addr learned my addr 172.21.15.26:0/2503373658 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf5affd700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c89c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.225+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c897c00 0x56174c8a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 -- 172.21.15.26:0/2503373658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 msgr2=0x56174c89c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c89c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 -- 172.21.15.26:0/2503373658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c897c00 msgr2=0x56174c8a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c897c00 0x56174c8a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 -- 172.21.15.26:0/2503373658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174c5ffc00 con 0x56174c8a33d0 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf5bfff700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400bdf0 tx=0x7fcf5402b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd4252c9a5a2acd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf3affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.226+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400e070 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf5400f070 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf5400be20 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5401dd30 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf75bb4b80 10 client.27128 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56174c615250 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.227+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf5404d080 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf5401e720 con 0x56174c8a33d0 2023-10-06T21:21:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 10 client.27128 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 20 client.27128 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c906cd0 0x56174c9090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 4 client.27128 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:31.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.228+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56174c7e4ad0 con 0x56174c906cd0 2023-10-06T21:21:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.229+0000 7fcf5affd700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c906cd0 0x56174c9090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.230+0000 7fcf5affd700 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c906cd0 0x56174c9090f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4525d72c5a0d043e server_cookie=6a0581a776f4f9c in_seq=0 out_seq=0 2023-10-06T21:21:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.233+0000 7fcf3affd700 10 client.27128 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.248+0000 7fcf3affd700 1 -- 172.21.15.26:0/2503373658 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56174c7e4ad0 con 0x56174c906cd0 2023-10-06T21:21:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.248+0000 7fcf3affd700 10 client.27128 handle_command_reply: tid=0 2023-10-06T21:21:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.248+0000 7fcf75bb4b80 1 client.27128 shutdown 2023-10-06T21:21:31.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 msgr2=0x56174c89c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400bdf0 tx=0x7fcf5402b040 comp rx=0 tx=0).stop 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 shutdown_connections 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56174c906cd0 0x56174c9090f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56174c8a33d0 0x56174c89c7f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56174c897c00 0x56174c8a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf3affd700 0 client.27128 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:31.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 --2- 172.21.15.26:0/2503373658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174c898520 0x56174c89c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 >> 172.21.15.26:0/2503373658 conn(0x56174c80d200 msgr2=0x56174c89b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 shutdown_connections 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.249+0000 7fcf75bb4b80 1 -- 172.21.15.26:0/2503373658 wait complete. 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.250+0000 7fcf75bb4b80 20 client.27128 trim_cache size 0 max 16384 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:21:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:21:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.250+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 msgr2=0x7fcf440a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.250+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 0x7fcf440a3c40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf50000c00 tx=0x7fcf500d1040 comp rx=0 tx=0).stop 2023-10-06T21:21:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.250+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 msgr2=0x7fcf5c1a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.250+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b550 tx=0x7fcf5000b910 comp rx=0 tx=0).stop 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 shutdown_connections 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf440a1780 0x7fcf440a3c40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf5c11a690 0x7fcf5c1aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf5c10f620 0x7fcf5c1a5fc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 --2- 172.21.15.26:0/1000281802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf5c10fa90 0x7fcf5c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 >> 172.21.15.26:0/1000281802 conn(0x7fcf5c06cfa0 msgr2=0x7fcf5c118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 shutdown_connections 2023-10-06T21:21:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:31.251+0000 7fcf621ff700 1 -- 172.21.15.26:0/1000281802 wait complete. 2023-10-06T21:21:32.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 650.0 s, 1400.7 tps, lat 71.917 ms stddev 77.170 2023-10-06T21:21:32.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:32 smithi078 ceph-mon[134896]: pgmap v350: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 825 op/s 2023-10-06T21:21:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:32 smithi167 ceph-mon[134494]: pgmap v350: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 825 op/s 2023-10-06T21:21:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:32 smithi026 ceph-mon[98490]: pgmap v350: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 825 op/s 2023-10-06T21:21:34.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:34 smithi167 ceph-mon[134494]: pgmap v351: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 603 op/s 2023-10-06T21:21:34.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:34 smithi026 ceph-mon[98490]: pgmap v351: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 603 op/s 2023-10-06T21:21:34.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:34 smithi078 ceph-mon[134896]: pgmap v351: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 603 op/s 2023-10-06T21:21:36.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:36 smithi167 ceph-mon[134494]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 547 op/s 2023-10-06T21:21:36.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:36 smithi026 ceph-mon[98490]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 547 op/s 2023-10-06T21:21:36.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:36 smithi078 ceph-mon[134896]: pgmap v352: 97 pgs: 97 active+clean; 5.7 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 547 op/s 2023-10-06T21:21:37.130 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 655.0 s, 1234.0 tps, lat 81.968 ms stddev 85.615 2023-10-06T21:21:38.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:38 smithi167 ceph-mon[134494]: pgmap v353: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 790 op/s 2023-10-06T21:21:38.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:38 smithi026 ceph-mon[98490]: pgmap v353: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 790 op/s 2023-10-06T21:21:38.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:38 smithi078 ceph-mon[134896]: pgmap v353: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 790 op/s 2023-10-06T21:21:40.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:40 smithi026 ceph-mon[98490]: pgmap v354: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 538 op/s 2023-10-06T21:21:40.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:40 smithi078 ceph-mon[134896]: pgmap v354: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 538 op/s 2023-10-06T21:21:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:40 smithi167 ceph-mon[134494]: pgmap v354: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 538 op/s 2023-10-06T21:21:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 660.0 s, 1349.0 tps, lat 73.669 ms stddev 86.408 2023-10-06T21:21:42.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:42 smithi026 ceph-mon[98490]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 747 op/s 2023-10-06T21:21:42.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:42 smithi078 ceph-mon[134896]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 747 op/s 2023-10-06T21:21:43.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:42 smithi167 ceph-mon[134494]: pgmap v355: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 10 MiB/s wr, 747 op/s 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/633348055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 msgr2=0x7fcf6410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/633348055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410dd20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009fc0 tx=0x7fcf5c00b330 comp rx=0 tx=0).stop 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/633348055 shutdown_connections 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/633348055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410dd20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/633348055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf64106f60 0x7fcf64107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/633348055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf640682d0 0x7fcf64068730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.482+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/633348055 >> 172.21.15.26:0/633348055 conn(0x7fcf64076b70 msgr2=0x7fcf64076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.483+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/633348055 shutdown_connections 2023-10-06T21:21:44.486 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.483+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/633348055 wait complete. 2023-10-06T21:21:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.485+0000 7fcf6c7ff700 1 Processor -- start 2023-10-06T21:21:44.488 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.485+0000 7fcf6c7ff700 1 -- start start 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 0x7fcf6410bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 0x7fcf6419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6419e4e0 con 0x7fcf640682d0 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6419e650 con 0x7fcf64106f60 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6c7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6419e810 con 0x7fcf64068c70 2023-10-06T21:21:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6a59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 0x7fcf6410bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6a59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 0x7fcf6410bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59212/0 (socket says 172.21.15.26:59212) 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6a59b700 1 -- 172.21.15.26:0/868157431 learned_addr learned my addr 172.21.15.26:0/868157431 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.486+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 0x7fcf6419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/868157431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 msgr2=0x7fcf6410c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 msgr2=0x7fcf6410bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 0x7fcf6410bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/868157431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c009c40 con 0x7fcf64106f60 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.487+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 0x7fcf6419dea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c015000 tx=0x7fcf5c008fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.488+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/868157431 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00e040 con 0x7fcf64106f60 2023-10-06T21:21:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.488+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/868157431 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf5c01d470 con 0x7fcf64106f60 2023-10-06T21:21:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.488+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/868157431 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0052a0 con 0x7fcf64106f60 2023-10-06T21:21:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.488+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6419ea90 con 0x7fcf64106f60 2023-10-06T21:21:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.489+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6419aff0 con 0x7fcf64106f60 2023-10-06T21:21:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.493+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/868157431 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcf5c01d5e0 con 0x7fcf64106f60 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.494+0000 7fcf80195b80 1 Processor -- start 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.494+0000 7fcf80195b80 1 -- start start 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923614520 0x55e92361ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e9236217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92357d400 con 0x55e92361f3d0 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92356ebc0 con 0x55e923614520 2023-10-06T21:21:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf80195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92356c4e0 con 0x55e923613c00 2023-10-06T21:21:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf69d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf69d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39360/0 (socket says 172.21.15.26:39360) 2023-10-06T21:21:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.495+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3570061090 learned_addr learned my addr 172.21.15.26:0/3570061090 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3570061090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923614520 msgr2=0x55e92361ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923614520 0x55e92361ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3570061090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 msgr2=0x55e9236217c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e9236217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3570061090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e923392d80 con 0x55e923613c00 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.496+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcf54011670 tx=0x7fcf54011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf248df39537c64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.497+0000 7fcf58ff9700 1 -- 172.21.15.26:0/3570061090 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54016070 con 0x55e923613c00 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.497+0000 7fcf58ff9700 1 -- 172.21.15.26:0/3570061090 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf540044f0 con 0x55e923613c00 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.497+0000 7fcf58ff9700 1 -- 172.21.15.26:0/3570061090 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5401a660 con 0x55e923613c00 2023-10-06T21:21:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.497+0000 7fcf5b7fe700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 0x7fcf500a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.498+0000 7fcf5b7fe700 1 -- 172.21.15.26:0/868157431 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf5c0ce3c0 con 0x7fcf64106f60 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.498+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 0x7fcf500a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.498+0000 7fcf80195b80 1 -- 172.21.15.26:0/3570061090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 msgr2=0x55e923613fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.498+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcf54011670 tx=0x7fcf54011980 comp rx=0 tx=0).stop 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 -- 172.21.15.26:0/3570061090 shutdown_connections 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e9236217c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3570061090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923614520 0x55e92361ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 -- 172.21.15.26:0/3570061090 >> 172.21.15.26:0/3570061090 conn(0x55e923589200 msgr2=0x55e923610080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 -- 172.21.15.26:0/3570061090 shutdown_connections 2023-10-06T21:21:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 0x7fcf500a3c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009d70 tx=0x7fcf60009910 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.500+0000 7fcf80195b80 1 -- 172.21.15.26:0/3570061090 wait complete. 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.501+0000 7fcf80195b80 1 -- start start 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 Processor -- start 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e92361c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e92361c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e923567dd0 con 0x55e923614520 2023-10-06T21:21:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9235046b0 con 0x55e92361f3d0 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e923504350 con 0x55e923613c00 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.502+0000 7fcf80195b80 10 client.0 _ll_register_callbacks cb 0x55e9233617a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6a59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e92361c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42958/0 (socket says 172.21.15.26:42958) 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/307047700 learned_addr learned my addr 172.21.15.26:0/307047700 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6a59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e92361c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39362/0 (socket says 172.21.15.26:39362) 2023-10-06T21:21:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e92361c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/307047700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 msgr2=0x55e92361c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e92361c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/307047700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 msgr2=0x55e92361c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e92361c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.503+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/307047700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e92337bc00 con 0x55e92361f3d0 2023-10-06T21:21:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.504+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0978c0 tx=0x7fcf5c09b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ffdba7275fa3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.504+0000 7fcf4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.504+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c09b3c0 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.504+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf5c09f510 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.505+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e923607d50 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.506+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5c0a36c0 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.506+0000 7fcf80195b80 10 client.27137 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.506+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e9233b91f0 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.506+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf5c0b7030 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf5c09b700 con 0x55e92361f3d0 2023-10-06T21:21:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 10 client.27137 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 10 client.27137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 20 client.27137 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e92362ee40 0x55e923631260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 4 client.27137 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e923560db0 con 0x55e92362ee40 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.507+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e92362ee40 0x55e923631260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.508+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e92362ee40 0x55e923631260 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f988dc812e32a8 server_cookie=f96b9720277322e0 in_seq=0 out_seq=0 2023-10-06T21:21:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.508+0000 7fcf4dffb700 10 client.27137 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf4dffb700 1 -- 172.21.15.26:0/307047700 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e923560db0 con 0x55e92362ee40 2023-10-06T21:21:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf4dffb700 10 client.27137 handle_command_reply: tid=0 2023-10-06T21:21:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 client.27137 shutdown 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 msgr2=0x55e92361cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0978c0 tx=0x7fcf5c09b260 comp rx=0 tx=0).stop 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 shutdown_connections 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e92362ee40 0x55e923631260 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e92361cc90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e92361c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 --2- 172.21.15.26:0/307047700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e92361c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.510+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 >> 172.21.15.26:0/307047700 conn(0x55e923589200 msgr2=0x55e92360fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.511+0000 7fcf4dffb700 0 client.27137 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.511+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 shutdown_connections 2023-10-06T21:21:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.511+0000 7fcf80195b80 1 -- 172.21.15.26:0/307047700 wait complete. 2023-10-06T21:21:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.511+0000 7fcf80195b80 20 client.27137 trim_cache size 0 max 16384 2023-10-06T21:21:44.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:44 smithi167 ceph-mon[134494]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2023-10-06T21:21:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.533+0000 7fcf80195b80 1 Processor -- start 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.533+0000 7fcf80195b80 1 -- start start 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e923617570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9233e88c0 con 0x55e92361f3d0 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92361f250 con 0x55e923613c00 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf80195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e92362c630 con 0x55e923614520 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf69d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf69d9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39364/0 (socket says 172.21.15.26:39364) 2023-10-06T21:21:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.534+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3290242560 learned_addr learned my addr 172.21.15.26:0/3290242560 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e923617570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3290242560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923613c00 msgr2=0x55e923613fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3290242560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 msgr2=0x55e923617570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e923617570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 -- 172.21.15.26:0/3290242560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e923391250 con 0x55e923614520 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e923617570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.535+0000 7fcf69d9a700 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf54011640 tx=0x7fcf54009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c931f7b2ba3bb93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.536+0000 7fcf597fa700 1 -- 172.21.15.26:0/3290242560 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54016030 con 0x55e923614520 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.536+0000 7fcf597fa700 1 -- 172.21.15.26:0/3290242560 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf54003880 con 0x55e923614520 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.536+0000 7fcf597fa700 1 -- 172.21.15.26:0/3290242560 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54004660 con 0x55e923614520 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- 172.21.15.26:0/3290242560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 msgr2=0x55e923625f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf54011640 tx=0x7fcf54009bf0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- 172.21.15.26:0/3290242560 shutdown_connections 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e92361f3d0 0x55e923617570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e923613c00 0x55e923613fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 --2- 172.21.15.26:0/3290242560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923614520 0x55e923625f40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- 172.21.15.26:0/3290242560 >> 172.21.15.26:0/3290242560 conn(0x55e923589200 msgr2=0x55e92360e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- 172.21.15.26:0/3290242560 shutdown_connections 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- 172.21.15.26:0/3290242560 wait complete. 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.537+0000 7fcf80195b80 1 -- start start 2023-10-06T21:21:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 Processor -- start 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923625990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e923625ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9236245d0 con 0x55e923614520 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e923624740 con 0x55e92361f3d0 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e923624900 con 0x55e923613c00 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf80195b80 10 client.0 _ll_register_callbacks cb 0x7fcf5c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf6ad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf69d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e923625ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf6ad9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42980/0 (socket says 172.21.15.26:42980) 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/723599772 learned_addr learned my addr 172.21.15.26:0/723599772 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.538+0000 7fcf6a59b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923625990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/723599772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 msgr2=0x55e923625990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923625990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/723599772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 msgr2=0x55e923625ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e923625ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 -- 172.21.15.26:0/723599772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e923392d80 con 0x55e92361f3d0 2023-10-06T21:21:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf6ad9c700 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c09b350 tx=0x7fcf5c0a3230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc831ab054917b33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.539+0000 7fcf4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.540+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c09f320 con 0x55e92361f3d0 2023-10-06T21:21:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.540+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcf5c09f510 con 0x55e92361f3d0 2023-10-06T21:21:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.540+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf540044f0 con 0x55e92361f3d0 2023-10-06T21:21:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.541+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5c09af00 con 0x55e92361f3d0 2023-10-06T21:21:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.541+0000 7fcf80195b80 10 client.27140 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.541+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e92345fa20 con 0x55e92361f3d0 2023-10-06T21:21:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf540044f0 con 0x55e92361f3d0 2023-10-06T21:21:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fcf5c0971b0 con 0x55e92361f3d0 2023-10-06T21:21:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 10 client.27140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 20 client.27140 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e923683a80 0x55e923685e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 4 client.27140 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e923560ad0 con 0x55e923683a80 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.542+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e923683a80 0x55e923685e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.543+0000 7fcf4dffb700 10 client.27140 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.543+0000 7fcf6a59b700 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e923683a80 0x55e923685e70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62258dca166b074f server_cookie=3e614133cc7b3851 in_seq=0 out_seq=0 2023-10-06T21:21:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.565+0000 7fcf4dffb700 1 -- 172.21.15.26:0/723599772 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e923560ad0 con 0x55e923683a80 2023-10-06T21:21:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.565+0000 7fcf4dffb700 10 client.27140 handle_command_reply: tid=0 2023-10-06T21:21:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.565+0000 7fcf80195b80 1 client.27140 shutdown 2023-10-06T21:21:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.565+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 msgr2=0x55e923623ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.565+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c09b350 tx=0x7fcf5c0a3230 comp rx=0 tx=0).stop 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 shutdown_connections 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e923683a80 0x55e923685e70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e92361f3d0 0x55e923623ec0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e923613c00 0x55e923625990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 --2- 172.21.15.26:0/723599772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e923614520 0x55e923625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 >> 172.21.15.26:0/723599772 conn(0x55e923589200 msgr2=0x55e923617570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.566+0000 7fcf4dffb700 0 client.27140 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.567+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 shutdown_connections 2023-10-06T21:21:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.567+0000 7fcf80195b80 1 -- 172.21.15.26:0/723599772 wait complete. 2023-10-06T21:21:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.568+0000 7fcf80195b80 20 client.27140 trim_cache size 0 max 16384 2023-10-06T21:21:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:21:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:21:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:21:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:21:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.568+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 msgr2=0x7fcf500a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 0x7fcf500a3c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009d70 tx=0x7fcf60009910 comp rx=0 tx=0).stop 2023-10-06T21:21:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 msgr2=0x7fcf6419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 0x7fcf6419dea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c015000 tx=0x7fcf5c008fd0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 shutdown_connections 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf64106f60 0x7fcf6419dea0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcf500a1740 0x7fcf500a3c00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf640682d0 0x7fcf6410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 --2- 172.21.15.26:0/868157431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf64068c70 0x7fcf6410c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 >> 172.21.15.26:0/868157431 conn(0x7fcf64076b70 msgr2=0x7fcf64100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 shutdown_connections 2023-10-06T21:21:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:44.569+0000 7fcf6c7ff700 1 -- 172.21.15.26:0/868157431 wait complete. 2023-10-06T21:21:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:44 smithi026 ceph-mon[98490]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2023-10-06T21:21:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:44 smithi078 ceph-mon[134896]: pgmap v356: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2023-10-06T21:21:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:46 smithi167 ceph-mon[134494]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2023-10-06T21:21:46.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:46 smithi026 ceph-mon[98490]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2023-10-06T21:21:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:46 smithi078 ceph-mon[134896]: pgmap v357: 97 pgs: 97 active+clean; 5.8 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2023-10-06T21:21:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 665.0 s, 1408.4 tps, lat 70.983 ms stddev 75.198 2023-10-06T21:21:48.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:48 smithi167 ceph-mon[134494]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 746 op/s 2023-10-06T21:21:48.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:48 smithi026 ceph-mon[98490]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 746 op/s 2023-10-06T21:21:48.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:48 smithi078 ceph-mon[134896]: pgmap v358: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.8 MiB/s wr, 746 op/s 2023-10-06T21:21:49.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:21:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 -- 172.21.15.26:0/2792452548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 msgr2=0x7f027810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 --2- 172.21.15.26:0/2792452548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f027810d8d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0270009f90 tx=0x7f027000b330 comp rx=0 tx=0).stop 2023-10-06T21:21:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 -- 172.21.15.26:0/2792452548 shutdown_connections 2023-10-06T21:21:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 --2- 172.21.15.26:0/2792452548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f027810d8d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 --2- 172.21.15.26:0/2792452548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781020b0 0x7f0278102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 --2- 172.21.15.26:0/2792452548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781029d0 0x7f027810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.077+0000 7f027e9ff700 1 -- 172.21.15.26:0/2792452548 >> 172.21.15.26:0/2792452548 conn(0x7f02780fb830 msgr2=0x7f02780fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.078+0000 7f027e9ff700 1 -- 172.21.15.26:0/2792452548 shutdown_connections 2023-10-06T21:21:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.078+0000 7f027e9ff700 1 -- 172.21.15.26:0/2792452548 wait complete. 2023-10-06T21:21:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.079+0000 7f027e9ff700 1 Processor -- start 2023-10-06T21:21:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.079+0000 7f027e9ff700 1 -- start start 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 0x7f02781077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781029d0 0x7f0278107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f02781a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02781a9730 con 0x7f02781029d0 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02781a98a0 con 0x7f02781020b0 2023-10-06T21:21:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027e9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02781a9a60 con 0x7f027810b480 2023-10-06T21:21:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f0277fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 0x7f02781077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f0277fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 0x7f02781077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42988/0 (socket says 172.21.15.26:42988) 2023-10-06T21:21:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f0277fff700 1 -- 172.21.15.26:0/3459648354 learned_addr learned my addr 172.21.15.26:0/3459648354 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.080+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f02781a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 -- 172.21.15.26:0/3459648354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 msgr2=0x7f02781077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 0x7f02781077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 -- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781029d0 msgr2=0x7f0278107cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781029d0 0x7f0278107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 -- 172.21.15.26:0/3459648354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009c40 con 0x7f027810b480 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f02781a91f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f027000b6c0 tx=0x7f0270003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f02757fa700 1 -- 172.21.15.26:0/3459648354 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000e030 con 0x7f027810b480 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f02757fa700 1 -- 172.21.15.26:0/3459648354 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027000be00 con 0x7f027810b480 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f02757fa700 1 -- 172.21.15.26:0/3459648354 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027001a6e0 con 0x7f027810b480 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.081+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02781a9ce0 con 0x7f027810b480 2023-10-06T21:21:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.082+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781aa1a0 con 0x7f027810b480 2023-10-06T21:21:50.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.087+0000 7f02757fa700 1 -- 172.21.15.26:0/3459648354 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0270005600 con 0x7f027810b480 2023-10-06T21:21:50.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.088+0000 7f02923d4b80 1 Processor -- start 2023-10-06T21:21:50.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.088+0000 7f02923d4b80 1 -- start start 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 0x557912d984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912da5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912d01460 con 0x557912da3890 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912cf2c20 con 0x557912d980c0 2023-10-06T21:21:50.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.089+0000 7f02923d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912cf0540 con 0x557912d989e0 2023-10-06T21:21:50.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.093+0000 7f0277fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f02777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 0x557912d984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39402/0 (socket says 172.21.15.26:39402) 2023-10-06T21:21:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 -- 172.21.15.26:0/3446729283 learned_addr learned my addr 172.21.15.26:0/3446729283 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f02777fe700 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912da5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f02757fa700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 0x7f02600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 -- 172.21.15.26:0/3446729283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 msgr2=0x557912d984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 0x557912d984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 -- 172.21.15.26:0/3446729283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 msgr2=0x557912da5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912da5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f0277fff700 1 -- 172.21.15.26:0/3446729283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557912b16de0 con 0x557912d989e0 2023-10-06T21:21:50.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.094+0000 7f02777fe700 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 0x557912d984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f02757fa700 1 -- 172.21.15.26:0/3459648354 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f027000f020 con 0x7f027810b480 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 0x7f02600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f0277fff700 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f026800ae40 tx=0x7f026800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ec8492ac3705ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f027cf9c700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 0x7f02600a3b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02700cb7f0 tx=0x7f0270009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f025e7fc700 1 -- 172.21.15.26:0/3446729283 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800a940 con 0x557912d989e0 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f025e7fc700 1 -- 172.21.15.26:0/3446729283 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026800fec0 con 0x557912d989e0 2023-10-06T21:21:50.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.095+0000 7f025e7fc700 1 -- 172.21.15.26:0/3446729283 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268010a50 con 0x557912d989e0 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 -- 172.21.15.26:0/3446729283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 msgr2=0x557912da3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f026800ae40 tx=0x7f026800f6a0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 -- 172.21.15.26:0/3446729283 shutdown_connections 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912da5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d980c0 0x557912d984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3446729283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da3350 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.096+0000 7f02923d4b80 1 -- 172.21.15.26:0/3446729283 >> 172.21.15.26:0/3446729283 conn(0x557912d0d260 msgr2=0x557912d94450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.097+0000 7f02923d4b80 1 -- 172.21.15.26:0/3446729283 shutdown_connections 2023-10-06T21:21:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.097+0000 7f02923d4b80 1 -- 172.21.15.26:0/3446729283 wait complete. 2023-10-06T21:21:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.097+0000 7f02923d4b80 1 -- start start 2023-10-06T21:21:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.098+0000 7f02923d4b80 1 Processor -- start 2023-10-06T21:21:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912da0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912cebe30 con 0x557912d980c0 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912c88710 con 0x557912da3890 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912c883b0 con 0x557912d989e0 2023-10-06T21:21:50.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02923d4b80 10 client.0 _ll_register_callbacks cb 0x557912ae5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:50.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02777fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59274/0 (socket says 172.21.15.26:59274) 2023-10-06T21:21:50.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02777fe700 1 -- 172.21.15.26:0/1861092941 learned_addr learned my addr 172.21.15.26:0/1861092941 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f0277fff700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.099+0000 7f02777fe700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912da0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 -- 172.21.15.26:0/1861092941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 msgr2=0x557912da0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 -- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 msgr2=0x557912da0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912da0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 -- 172.21.15.26:0/1861092941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026800aaa0 con 0x557912d980c0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02777fe700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f026c014fb0 tx=0x7f026c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aac37427622683 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02577fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c014130 con 0x557912d980c0 2023-10-06T21:21:50.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557912ce4e10 con 0x557912d980c0 2023-10-06T21:21:50.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.100+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f026c014290 con 0x557912d980c0 2023-10-06T21:21:50.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.101+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f026c010c30 con 0x557912d980c0 2023-10-06T21:21:50.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.101+0000 7f02923d4b80 10 client.16818 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:50.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.101+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557912affc60 con 0x557912d980c0 2023-10-06T21:21:50.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.101+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557912ce4e10 con 0x557912d980c0 2023-10-06T21:21:50.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.101+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f026c0051b0 con 0x557912d980c0 2023-10-06T21:21:50.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 10 client.16818 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:50.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:50.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 20 client.16818 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912db3440 0x557912db5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 4 client.16818 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557912ce4b30 con 0x557912db3440 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.102+0000 7f0277fff700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912db3440 0x557912db5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.103+0000 7f0277fff700 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912db3440 0x557912db5860 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e63f7d1f0e0464 server_cookie=a4455617771b9efc in_seq=0 out_seq=0 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.103+0000 7f02577fe700 10 client.16818 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.103+0000 7f02577fe700 1 -- 172.21.15.26:0/1861092941 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557912ce4b30 con 0x557912db3440 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.103+0000 7f02577fe700 10 client.16818 handle_command_reply: tid=0 2023-10-06T21:21:50.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 client.16818 shutdown 2023-10-06T21:21:50.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 msgr2=0x557912da32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f026c014fb0 tx=0x7f026c0099e0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 shutdown_connections 2023-10-06T21:21:50.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912db3440 0x557912db5860 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912da0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912da32a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 --2- 172.21.15.26:0/1861092941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02577fe700 0 client.16818 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.104+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 >> 172.21.15.26:0/1861092941 conn(0x557912d0d260 msgr2=0x557912d93fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.105+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 shutdown_connections 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.105+0000 7f02923d4b80 1 -- 172.21.15.26:0/1861092941 wait complete. 2023-10-06T21:21:50.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.105+0000 7f02923d4b80 20 client.16818 trim_cache size 0 max 16384 2023-10-06T21:21:50.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.156+0000 7f02923d4b80 1 Processor -- start 2023-10-06T21:21:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.157+0000 7f02923d4b80 1 -- start start 2023-10-06T21:21:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.157+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912d9fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.157+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.158+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912d9d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.158+0000 7f02923d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912b6c920 con 0x557912d980c0 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.158+0000 7f02923d4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912d95280 con 0x557912da3890 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.158+0000 7f02923d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912d9ea80 con 0x557912d989e0 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39418/0 (socket says 172.21.15.26:39418) 2023-10-06T21:21:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 -- 172.21.15.26:0/4263028055 learned_addr learned my addr 172.21.15.26:0/4263028055 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 -- 172.21.15.26:0/4263028055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 msgr2=0x557912d9d7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:21:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912d9d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 -- 172.21.15.26:0/4263028055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 msgr2=0x557912d9fce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:21:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912d9fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.159+0000 7f0277fff700 1 -- 172.21.15.26:0/4263028055 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557912b3d250 con 0x557912d989e0 2023-10-06T21:21:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.160+0000 7f0277fff700 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f026801b040 tx=0x7f026800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6e802e0f89ea3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.161+0000 7f025effd700 1 -- 172.21.15.26:0/4263028055 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268028070 con 0x557912d989e0 2023-10-06T21:21:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.161+0000 7f025effd700 1 -- 172.21.15.26:0/4263028055 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026800a920 con 0x557912d989e0 2023-10-06T21:21:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.161+0000 7f025effd700 1 -- 172.21.15.26:0/4263028055 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268029a00 con 0x557912d989e0 2023-10-06T21:21:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.162+0000 7f02923d4b80 1 -- 172.21.15.26:0/4263028055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 msgr2=0x557912da2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.162+0000 7f02923d4b80 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f026801b040 tx=0x7f026800f6a0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 -- 172.21.15.26:0/4263028055 shutdown_connections 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912da3890 0x557912d9d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912d980c0 0x557912d9fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 --2- 172.21.15.26:0/4263028055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d989e0 0x557912da2e40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 -- 172.21.15.26:0/4263028055 >> 172.21.15.26:0/4263028055 conn(0x557912d0d260 msgr2=0x557912d9b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 -- 172.21.15.26:0/4263028055 shutdown_connections 2023-10-06T21:21:50.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.163+0000 7f02923d4b80 1 -- 172.21.15.26:0/4263028055 wait complete. 2023-10-06T21:21:50.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.164+0000 7f02923d4b80 1 -- start start 2023-10-06T21:21:50.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.164+0000 7f02923d4b80 1 Processor -- start 2023-10-06T21:21:50.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d980c0 0x557912daa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912dff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912e03040 con 0x557912da3890 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912dffc60 con 0x557912d989e0 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557912dffdd0 con 0x557912d980c0 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02923d4b80 10 client.0 _ll_register_callbacks cb 0x7f026c014a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f0277fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f0277fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43016/0 (socket says 172.21.15.26:43016) 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f0277fff700 1 -- 172.21.15.26:0/3775114902 learned_addr learned my addr 172.21.15.26:0/3775114902 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:50.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.165+0000 7f02777fe700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d980c0 0x557912daa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.166+0000 7f0277fff700 1 -- 172.21.15.26:0/3775114902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d980c0 msgr2=0x557912daa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.166+0000 7f0277fff700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d980c0 0x557912daa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.166+0000 7f0277fff700 1 -- 172.21.15.26:0/3775114902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 msgr2=0x557912dff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.166+0000 7f0277fff700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912dff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.166+0000 7f0277fff700 1 -- 172.21.15.26:0/3775114902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557912b16de0 con 0x557912d989e0 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.167+0000 7f0277fff700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f026800a320 tx=0x7f0268010590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118f69b0f68a73d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.167+0000 7f02577fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:50.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.167+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268028050 con 0x557912d989e0 2023-10-06T21:21:50.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.167+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f02680056a0 con 0x557912d989e0 2023-10-06T21:21:50.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.167+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557912b37f70 con 0x557912d989e0 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.168+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02680046f0 con 0x557912d989e0 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.168+0000 7f02923d4b80 10 client.27155 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.168+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557912b152b0 con 0x557912d989e0 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.168+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557912b37f70 con 0x557912d989e0 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.168+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0268037020 con 0x557912d989e0 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 10 client.27155 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 20 client.27155 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912e190f0 0x557912e1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 4 client.27155 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:21:50.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557912b6cf60 con 0x557912e190f0 2023-10-06T21:21:50.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.169+0000 7f02777fe700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912e190f0 0x557912e1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:50.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.170+0000 7f02777fe700 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912e190f0 0x557912e1b4e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b381ce375021804 server_cookie=ae68b0d47cc29b04 in_seq=0 out_seq=0 2023-10-06T21:21:50.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.170+0000 7f02577fe700 10 client.27155 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:50.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.170+0000 7f02577fe700 1 -- 172.21.15.26:0/3775114902 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557912b6cf60 con 0x557912e190f0 2023-10-06T21:21:50.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.170+0000 7f02577fe700 10 client.27155 handle_command_reply: tid=0 2023-10-06T21:21:50.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.177+0000 7f02923d4b80 1 client.27155 shutdown 2023-10-06T21:21:50.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.177+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 msgr2=0x557912dff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.177+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f026800a320 tx=0x7f0268010590 comp rx=0 tx=0).stop 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 shutdown_connections 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557912da3890 0x557912dff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557912d980c0 0x557912daa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557912e190f0 0x557912e1b4e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 --2- 172.21.15.26:0/3775114902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557912d989e0 0x557912dff1e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 >> 172.21.15.26:0/3775114902 conn(0x557912d0d260 msgr2=0x557912d93660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.178+0000 7f02577fe700 0 client.27155 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:50.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.179+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 shutdown_connections 2023-10-06T21:21:50.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.179+0000 7f02923d4b80 1 -- 172.21.15.26:0/3775114902 wait complete. 2023-10-06T21:21:50.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.179+0000 7f02923d4b80 20 client.27155 trim_cache size 0 max 16384 2023-10-06T21:21:50.182 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:21:50.183 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 msgr2=0x7f02600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 0x7f02600a3b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02700cb7f0 tx=0x7f0270009920 comp rx=0 tx=0).stop 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 msgr2=0x7f02781a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f02781a91f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f027000b6c0 tx=0x7f0270003c30 comp rx=0 tx=0).stop 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 shutdown_connections 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f027810b480 0x7f02781a91f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02781020b0 0x7f02781077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f02600a1670 0x7f02600a3b30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 --2- 172.21.15.26:0/3459648354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02781029d0 0x7f0278107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:50.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 >> 172.21.15.26:0/3459648354 conn(0x7f02780fb830 msgr2=0x7f027810d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:50.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 shutdown_connections 2023-10-06T21:21:50.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:21:50.180+0000 7f027e9ff700 1 -- 172.21.15.26:0/3459648354 wait complete. 2023-10-06T21:21:50.212 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:21:50.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:50 smithi026 ceph-mon[98490]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-10-06T21:21:50.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:50 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:50.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:50 smithi078 ceph-mon[134896]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-10-06T21:21:50.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:50 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:50 smithi167 ceph-mon[134494]: pgmap v359: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 504 op/s 2023-10-06T21:21:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:50 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:21:51.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:51 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:51.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:51 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:51.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:51 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:21:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:51 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:51 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:51 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:21:52.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:51 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:21:52.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:51 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:21:52.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:51 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:21:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 670.0 s, 1167.6 tps, lat 86.046 ms stddev 93.309 2023-10-06T21:21:52.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:52 smithi078 ceph-mon[134896]: pgmap v360: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 747 op/s 2023-10-06T21:21:53.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:52 smithi167 ceph-mon[134494]: pgmap v360: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 747 op/s 2023-10-06T21:21:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:52 smithi026 ceph-mon[98490]: pgmap v360: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 747 op/s 2023-10-06T21:21:54.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:54 smithi167 ceph-mon[134494]: pgmap v361: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 574 op/s 2023-10-06T21:21:54.576 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:54 smithi026 ceph-mon[98490]: pgmap v361: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 574 op/s 2023-10-06T21:21:54.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:54 smithi078 ceph-mon[134896]: pgmap v361: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 574 op/s 2023-10-06T21:21:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:56 smithi167 ceph-mon[134494]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 547 op/s 2023-10-06T21:21:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:56 smithi026 ceph-mon[98490]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 547 op/s 2023-10-06T21:21:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:56 smithi078 ceph-mon[134896]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 547 op/s 2023-10-06T21:21:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 675.0 s, 1479.6 tps, lat 67.486 ms stddev 71.247 2023-10-06T21:21:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.842+0000 7fbf615ff700 1 -- 172.21.15.26:0/4240338660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0fff70 msgr2=0x7fbf5c1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.842+0000 7fbf615ff700 1 --2- 172.21.15.26:0/4240338660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0fff70 0x7fbf5c1003f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009990 tx=0x7fbf44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.851+0000 7fbf615ff700 1 -- 172.21.15.26:0/4240338660 shutdown_connections 2023-10-06T21:21:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.851+0000 7fbf615ff700 1 --2- 172.21.15.26:0/4240338660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c100930 0x7fbf5c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.851+0000 7fbf615ff700 1 --2- 172.21.15.26:0/4240338660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c105ef0 0x7fbf5c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.851+0000 7fbf615ff700 1 --2- 172.21.15.26:0/4240338660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0fff70 0x7fbf5c1003f0 secure :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009990 tx=0x7fbf44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.851+0000 7fbf615ff700 1 -- 172.21.15.26:0/4240338660 >> 172.21.15.26:0/4240338660 conn(0x7fbf5c076b70 msgr2=0x7fbf5c076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.863+0000 7fbf615ff700 1 -- 172.21.15.26:0/4240338660 shutdown_connections 2023-10-06T21:21:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.873+0000 7fbf615ff700 1 -- 172.21.15.26:0/4240338660 wait complete. 2023-10-06T21:21:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.875+0000 7fbf615ff700 1 Processor -- start 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.875+0000 7fbf615ff700 1 -- start start 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 0x7fbf5c072640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c105ef0 0x7fbf5c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 0x7fbf5c06be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c072ef0 con 0x7fbf5c105ef0 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0701d0 con 0x7fbf5c06b980 2023-10-06T21:21:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.876+0000 7fbf615ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c070390 con 0x7fbf5c100930 2023-10-06T21:21:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.877+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 0x7fbf5c06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.877+0000 7fbf5affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 0x7fbf5c072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.877+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 0x7fbf5c06be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45186/0 (socket says 172.21.15.26:45186) 2023-10-06T21:21:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.877+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/774195515 learned_addr learned my addr 172.21.15.26:0/774195515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.879+0000 7fbf5affd700 1 -- 172.21.15.26:0/774195515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 msgr2=0x7fbf5c06be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.879+0000 7fbf5affd700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 0x7fbf5c06be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.879+0000 7fbf5affd700 1 -- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c105ef0 msgr2=0x7fbf5c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.879+0000 7fbf5affd700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c105ef0 0x7fbf5c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.879+0000 7fbf5affd700 1 -- 172.21.15.26:0/774195515 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440095f0 con 0x7fbf5c100930 2023-10-06T21:21:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.888+0000 7fbf5affd700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 0x7fbf5c072640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00e9f0 tx=0x7fbf4c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.890+0000 7fbf58ff9700 1 -- 172.21.15.26:0/774195515 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00cbd0 con 0x7fbf5c100930 2023-10-06T21:21:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.890+0000 7fbf58ff9700 1 -- 172.21.15.26:0/774195515 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf4c00cd30 con 0x7fbf5c100930 2023-10-06T21:21:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.890+0000 7fbf58ff9700 1 -- 172.21.15.26:0/774195515 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c018680 con 0x7fbf5c100930 2023-10-06T21:21:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.891+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c070610 con 0x7fbf5c100930 2023-10-06T21:21:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.891+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c070ad0 con 0x7fbf5c100930 2023-10-06T21:21:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.897+0000 7fbf74ecbb80 1 Processor -- start 2023-10-06T21:21:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.898+0000 7fbf74ecbb80 1 -- start start 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a67724fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a677386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6768e400 con 0x557a67725520 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6767fbc0 con 0x557a67738be0 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6767d4e0 con 0x557a67724c00 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45196/0 (socket says 172.21.15.26:45196) 2023-10-06T21:21:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.899+0000 7fbf527fc700 1 -- 172.21.15.26:0/408778091 learned_addr learned my addr 172.21.15.26:0/408778091 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 -- 172.21.15.26:0/408778091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 msgr2=0x557a67724fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a67724fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a67724fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 -- 172.21.15.26:0/408778091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 msgr2=0x557a677386a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a677386a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 -- 172.21.15.26:0/408778091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a674a3d80 con 0x557a67738be0 2023-10-06T21:21:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a67724fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.900+0000 7fbf527fc700 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000c00 tx=0x7fbf44005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11abf9921c6c2dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.901+0000 7fbf517fa700 1 -- 172.21.15.26:0/408778091 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4401d070 con 0x557a67738be0 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.901+0000 7fbf517fa700 1 -- 172.21.15.26:0/408778091 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf4400bd10 con 0x557a67738be0 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.901+0000 7fbf517fa700 1 -- 172.21.15.26:0/408778091 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400fc90 con 0x557a67738be0 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/408778091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 msgr2=0x557a6773afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000c00 tx=0x7fbf44005da0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/408778091 shutdown_connections 2023-10-06T21:21:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a6773afd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a67724fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/408778091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a677386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.902+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/408778091 >> 172.21.15.26:0/408778091 conn(0x557a6769a200 msgr2=0x557a67721080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.905+0000 7fbf58ff9700 1 -- 172.21.15.26:0/774195515 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbf4c010430 con 0x7fbf5c100930 2023-10-06T21:21:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.907+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/408778091 shutdown_connections 2023-10-06T21:21:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.908+0000 7fbf58ff9700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 0x7fbf480a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.908+0000 7fbf58ff9700 1 -- 172.21.15.26:0/774195515 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf4c014070 con 0x7fbf5c100930 2023-10-06T21:21:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.909+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/408778091 wait complete. 2023-10-06T21:21:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.909+0000 7fbf74ecbb80 1 -- start start 2023-10-06T21:21:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.909+0000 7fbf527fc700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 0x7fbf480a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 Processor -- start 2023-10-06T21:21:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf527fc700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 0x7fbf480a3b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf440098e0 tx=0x7fbf4401a040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67738610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67735fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67678dd0 con 0x557a67724c00 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a676156b0 con 0x557a67738be0 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67615350 con 0x557a67725520 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.910+0000 7fbf74ecbb80 10 client.0 _ll_register_callbacks cb 0x557a674727a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39270/0 (socket says 172.21.15.26:39270) 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf527fc700 1 -- 172.21.15.26:0/1240925802 learned_addr learned my addr 172.21.15.26:0/1240925802 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67735fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5affd700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67738610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1240925802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 msgr2=0x557a67735890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1240925802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 msgr2=0x557a67738610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67738610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1240925802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c10d560 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf527fc700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.911+0000 7fbf5affd700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67738610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67735fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007ac0 tx=0x7fbf54007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a5643bce1b1d3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf427fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400fa50 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a67718d50 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf54006e40 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf5400fbb0 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf74ecbb80 10 client.27164 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.912+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557a6748cc00 con 0x557a67738be0 2023-10-06T21:21:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a67718d50 con 0x557a67738be0 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf540143f0 con 0x557a67738be0 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 10 client.27164 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 20 client.27164 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a67748730 0x557a6774ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 4 client.27164 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a67671db0 con 0x557a67748730 2023-10-06T21:21:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.913+0000 7fbf5affd700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a67748730 0x557a6774ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.914+0000 7fbf427fc700 10 client.27164 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.914+0000 7fbf5affd700 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a67748730 0x557a6774ab50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69265f7d0827c37 server_cookie=8c1ba754a6a0f72f in_seq=0 out_seq=0 2023-10-06T21:21:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.915+0000 7fbf427fc700 1 -- 172.21.15.26:0/1240925802 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a67671db0 con 0x557a67748730 2023-10-06T21:21:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.915+0000 7fbf427fc700 10 client.27164 handle_command_reply: tid=0 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.915+0000 7fbf74ecbb80 1 client.27164 shutdown 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 msgr2=0x557a67735fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67735fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007ac0 tx=0x7fbf54007dd0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 shutdown_connections 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a67748730 0x557a6774ab50 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67735fd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67738610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1240925802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67725520 0x557a67735890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf427fc700 0 client.27164 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 >> 172.21.15.26:0/1240925802 conn(0x557a6769a200 msgr2=0x557a67720c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 shutdown_connections 2023-10-06T21:21:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1240925802 wait complete. 2023-10-06T21:21:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.916+0000 7fbf74ecbb80 20 client.27164 trim_cache size 0 max 16384 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 Processor -- start 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 -- start start 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67744870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 0x557a677381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a674f98c0 con 0x557a67724c00 2023-10-06T21:21:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6773afd0 con 0x557a67725520 2023-10-06T21:21:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.947+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67744db0 con 0x557a67738be0 2023-10-06T21:21:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 0x557a677381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf527fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 0x557a677381b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45224/0 (socket says 172.21.15.26:45224) 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:39272/0 (socket says 172.21.15.26:39272) 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf527fc700 1 -- 172.21.15.26:0/1083579201 learned_addr learned my addr 172.21.15.26:0/1083579201 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5affd700 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67744870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1083579201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 msgr2=0x557a677381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 0x557a677381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1083579201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 msgr2=0x557a67744870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67744870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.948+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/1083579201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a674ca1f0 con 0x557a67738be0 2023-10-06T21:21:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.949+0000 7fbf5b7fe700 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf54002230 tx=0x7fbf54009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc46ec5bf77c7d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.950+0000 7fbf51ffb700 1 -- 172.21.15.26:0/1083579201 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54006e40 con 0x557a67738be0 2023-10-06T21:21:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.950+0000 7fbf51ffb700 1 -- 172.21.15.26:0/1083579201 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5400fd10 con 0x557a67738be0 2023-10-06T21:21:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.950+0000 7fbf51ffb700 1 -- 172.21.15.26:0/1083579201 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54031690 con 0x557a67738be0 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1083579201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 msgr2=0x557a67730c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbf54002230 tx=0x7fbf54009670 comp rx=0 tx=0).stop 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1083579201 shutdown_connections 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67738be0 0x557a67730c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67724c00 0x557a67744870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/1083579201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67725520 0x557a677381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.951+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1083579201 >> 172.21.15.26:0/1083579201 conn(0x557a6769a200 msgr2=0x557a6774f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.952+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1083579201 shutdown_connections 2023-10-06T21:21:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.952+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/1083579201 wait complete. 2023-10-06T21:21:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.952+0000 7fbf74ecbb80 1 -- start start 2023-10-06T21:21:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 Processor -- start 2023-10-06T21:21:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a6773b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a6773be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67795550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67795a90 con 0x557a67725520 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67795c00 con 0x557a67738be0 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a67795dc0 con 0x557a67724c00 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.953+0000 7fbf74ecbb80 10 client.0 _ll_register_callbacks cb 0x7fbf54007370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:21:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.954+0000 7fbf5affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a6773b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.954+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67795550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.954+0000 7fbf5b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67795550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45226/0 (socket says 172.21.15.26:45226) 2023-10-06T21:21:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.954+0000 7fbf5b7fe700 1 -- 172.21.15.26:0/3783457612 learned_addr learned my addr 172.21.15.26:0/3783457612 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:21:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.954+0000 7fbf527fc700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a6773be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 -- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 msgr2=0x557a67795550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67795550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 -- 172.21.15.26:0/3783457612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 msgr2=0x557a6773be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a6773be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 -- 172.21.15.26:0/3783457612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a6748cc00 con 0x557a67724c00 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf5affd700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a6773b8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0caac0 tx=0x7fbf4c096040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d492d2ea983a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:21:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.955+0000 7fbf427fc700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0af070 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf4c09e4a0 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4400bd10 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf4c0a26e0 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf74ecbb80 10 client.27052 fetch_fsmap learned FSMap version 49 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.956+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557a674a2250 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf4400bd10 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fbf4c0a2250 con 0x557a67724c00 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 10 client.27052 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 20 client.27052 populate_metadata read hostname 'smithi026' 2023-10-06T21:21:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a6779d5d0 0x557a6779f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:21:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 4 client.27052 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:21:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a67671ad0 con 0x557a6779d5d0 2023-10-06T21:21:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.957+0000 7fbf527fc700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a6779d5d0 0x557a6779f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:21:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.958+0000 7fbf427fc700 10 client.27052 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.958+0000 7fbf527fc700 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a6779d5d0 0x557a6779f9f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be4d585296a6df3 server_cookie=a1a5c3d18d21820f in_seq=0 out_seq=0 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf427fc700 1 -- 172.21.15.26:0/3783457612 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a67671ad0 con 0x557a6779d5d0 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf427fc700 10 client.27052 handle_command_reply: tid=0 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 client.27052 shutdown 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 msgr2=0x557a6773b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a6773b8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0caac0 tx=0x7fbf4c096040 comp rx=0 tx=0).stop 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 shutdown_connections 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x557a6779d5d0 0x557a6779f9f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a67738be0 0x557a67795550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a67724c00 0x557a6773b8c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 --2- 172.21.15.26:0/3783457612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557a67725520 0x557a6773be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 >> 172.21.15.26:0/3783457612 conn(0x557a6769a200 msgr2=0x557a67730c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf427fc700 0 client.27052 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:21:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 shutdown_connections 2023-10-06T21:21:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.988+0000 7fbf74ecbb80 1 -- 172.21.15.26:0/3783457612 wait complete. 2023-10-06T21:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.989+0000 7fbf74ecbb80 20 client.27052 trim_cache size 0 max 16384 2023-10-06T21:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:21:57.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.989+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 msgr2=0x7fbf480a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.989+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 0x7fbf480a3b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf440098e0 tx=0x7fbf4401a040 comp rx=0 tx=0).stop 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.989+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 msgr2=0x7fbf5c072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.989+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 0x7fbf5c072640 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00e9f0 tx=0x7fbf4c00edb0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 shutdown_connections 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf5c06b980 0x7fbf5c06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5c100930 0x7fbf5c072640 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbf480a1640 0x7fbf480a3b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 --2- 172.21.15.26:0/774195515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c105ef0 0x7fbf5c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:21:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 >> 172.21.15.26:0/774195515 conn(0x7fbf5c076b70 msgr2=0x7fbf5c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:21:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 shutdown_connections 2023-10-06T21:21:57.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:21:57.990+0000 7fbf615ff700 1 -- 172.21.15.26:0/774195515 wait complete. 2023-10-06T21:21:58.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:21:58 smithi167 ceph-mon[134494]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 767 op/s 2023-10-06T21:21:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:21:58 smithi026 ceph-mon[98490]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 767 op/s 2023-10-06T21:21:58.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:21:58 smithi078 ceph-mon[134896]: pgmap v363: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.3 MiB/s wr, 767 op/s 2023-10-06T21:22:00.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:00 smithi026 ceph-mon[98490]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 525 op/s 2023-10-06T21:22:00.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:00 smithi078 ceph-mon[134896]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 525 op/s 2023-10-06T21:22:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:00 smithi167 ceph-mon[134494]: pgmap v364: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 525 op/s 2023-10-06T21:22:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 680.0 s, 673.6 tps, lat 120.894 ms stddev 134.717 2023-10-06T21:22:02.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:02 smithi026 ceph-mon[98490]: pgmap v365: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 799 op/s 2023-10-06T21:22:02.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:02 smithi078 ceph-mon[134896]: pgmap v365: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 799 op/s 2023-10-06T21:22:03.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:02 smithi167 ceph-mon[134494]: pgmap v365: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 799 op/s 2023-10-06T21:22:04.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:04 smithi026 ceph-mon[98490]: pgmap v366: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-10-06T21:22:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:04 smithi078 ceph-mon[134896]: pgmap v366: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-10-06T21:22:04.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:04 smithi167 ceph-mon[134494]: pgmap v366: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-10-06T21:22:06.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:06 smithi026 ceph-mon[98490]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:22:06.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:06 smithi026 ceph-mon[98490]: pgmap v368: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-10-06T21:22:06.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:06 smithi078 ceph-mon[134896]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:22:06.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:06 smithi078 ceph-mon[134896]: pgmap v368: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-10-06T21:22:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:06 smithi167 ceph-mon[134494]: osdmap e100: 12 total, 12 up, 12 in 2023-10-06T21:22:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:06 smithi167 ceph-mon[134494]: pgmap v368: 97 pgs: 97 active+clean; 5.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-10-06T21:22:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 685.0 s, 780.2 tps, lat 151.378 ms stddev 236.291 2023-10-06T21:22:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:08 smithi026 ceph-mon[98490]: pgmap v369: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:08 smithi078 ceph-mon[134896]: pgmap v369: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:08 smithi167 ceph-mon[134494]: pgmap v369: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:10.874 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:10 smithi078 ceph-mon[134896]: pgmap v370: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:10 smithi167 ceph-mon[134494]: pgmap v370: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:11.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:10 smithi026 ceph-mon[98490]: pgmap v370: 97 pgs: 15 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 57 active+clean; 5.4 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 28 MiB/s wr, 2.04k op/s 2023-10-06T21:22:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 -- 172.21.15.26:0/3707390994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec106f60 msgr2=0x7f75ec107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 --2- 172.21.15.26:0/3707390994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec106f60 0x7f75ec107340 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009990 tx=0x7f75dc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 -- 172.21.15.26:0/3707390994 shutdown_connections 2023-10-06T21:22:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 --2- 172.21.15.26:0/3707390994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec068c90 0x7f75ec10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 --2- 172.21.15.26:0/3707390994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec106f60 0x7f75ec107340 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 --2- 172.21.15.26:0/3707390994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 0x7f75ec068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 -- 172.21.15.26:0/3707390994 >> 172.21.15.26:0/3707390994 conn(0x7f75ec076b70 msgr2=0x7f75ec076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 -- 172.21.15.26:0/3707390994 shutdown_connections 2023-10-06T21:22:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.219+0000 7f75f17ff700 1 -- 172.21.15.26:0/3707390994 wait complete. 2023-10-06T21:22:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.220+0000 7f75f17ff700 1 Processor -- start 2023-10-06T21:22:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 -- start start 2023-10-06T21:22:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 0x7f75ec107c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 0x7f75ec108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 0x7f75ec1a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1089c0 con 0x7f75ec0682d0 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a9c20 con 0x7f75ec068c90 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.221+0000 7f75f17ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a9d60 con 0x7f75ec106f60 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 0x7f75ec108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 0x7f75ec108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58698/0 (socket says 172.21.15.26:58698) 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75ea7fc700 1 -- 172.21.15.26:0/553016848 learned_addr learned my addr 172.21.15.26:0/553016848 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 0x7f75ec1a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75eb7fe700 1 -- 172.21.15.26:0/553016848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 msgr2=0x7f75ec108160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 0x7f75ec108160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75eb7fe700 1 -- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 msgr2=0x7f75ec107c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.222+0000 7f75eaffd700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 0x7f75ec107c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 0x7f75ec107c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75eb7fe700 1 -- 172.21.15.26:0/553016848 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0095f0 con 0x7f75ec106f60 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 0x7f75ec1a96e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f75e400efc0 tx=0x7f75e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75e3fff700 1 -- 172.21.15.26:0/553016848 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4018070 con 0x7f75ec106f60 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75e3fff700 1 -- 172.21.15.26:0/553016848 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e400f040 con 0x7f75ec106f60 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75e3fff700 1 -- 172.21.15.26:0/553016848 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400cd30 con 0x7f75ec106f60 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec1a9f60 con 0x7f75ec106f60 2023-10-06T21:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.223+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1aa3f0 con 0x7f75ec106f60 2023-10-06T21:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.229+0000 7f75e3fff700 1 -- 172.21.15.26:0/553016848 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f75e4003a00 con 0x7f75ec106f60 2023-10-06T21:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.230+0000 7f76051fcb80 1 Processor -- start 2023-10-06T21:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.231+0000 7f76051fcb80 1 -- start start 2023-10-06T21:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.231+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7a9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aaae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f76051fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7a09400 con 0x55b6d7aa0520 2023-10-06T21:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f76051fcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d79fabc0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f76051fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d79f84e0 con 0x55b6d7aab3d0 2023-10-06T21:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.232+0000 7f75e3fff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 0x7f75d80a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75e3fff700 1 -- 172.21.15.26:0/553016848 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f75e40ccd60 con 0x7f75ec106f60 2023-10-06T21:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 0x7f75d80a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aaae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aaae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58210/0 (socket says 172.21.15.26:58210) 2023-10-06T21:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75eaffd700 1 -- 172.21.15.26:0/691420432 learned_addr learned my addr 172.21.15.26:0/691420432 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.233+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7a9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 -- 172.21.15.26:0/691420432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aab3d0 msgr2=0x55b6d7aad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 -- 172.21.15.26:0/691420432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 msgr2=0x55b6d7aaae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aaae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 -- 172.21.15.26:0/691420432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d781ed80 con 0x55b6d7a9fc00 2023-10-06T21:22:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75eb7fe700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 0x7f75d80a3bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75e400ef90 tx=0x7f75e4004cc0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.234+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7a9ffe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75ec109620 tx=0x7f75d400b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a7fe9b1fd90f678 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.235+0000 7f75e17fa700 1 -- 172.21.15.26:0/691420432 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4004030 con 0x55b6d7a9fc00 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.235+0000 7f75e17fa700 1 -- 172.21.15.26:0/691420432 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d4003680 con 0x55b6d7a9fc00 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.235+0000 7f75e17fa700 1 -- 172.21.15.26:0/691420432 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4010630 con 0x55b6d7a9fc00 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 -- 172.21.15.26:0/691420432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 msgr2=0x55b6d7a9ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7a9ffe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75ec109620 tx=0x7f75d400b500 comp rx=0 tx=0).stop 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 -- 172.21.15.26:0/691420432 shutdown_connections 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7a9ffe0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 --2- 172.21.15.26:0/691420432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aaae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 -- 172.21.15.26:0/691420432 >> 172.21.15.26:0/691420432 conn(0x55b6d7a15200 msgr2=0x55b6d7a9c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 -- 172.21.15.26:0/691420432 shutdown_connections 2023-10-06T21:22:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.236+0000 7f76051fcb80 1 -- 172.21.15.26:0/691420432 wait complete. 2023-10-06T21:22:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.237+0000 7f76051fcb80 1 -- start start 2023-10-06T21:22:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.237+0000 7f76051fcb80 1 Processor -- start 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d79f3dd0 con 0x55b6d7aab3d0 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d79906b0 con 0x55b6d7aa0520 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7990350 con 0x55b6d7a9fc00 2023-10-06T21:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.239+0000 7f76051fcb80 10 client.0 _ll_register_callbacks cb 0x55b6d77ed7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52214/0 (socket says 172.21.15.26:52214) 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75ea7fc700 1 -- 172.21.15.26:0/3125576645 learned_addr learned my addr 172.21.15.26:0/3125576645 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 -- 172.21.15.26:0/3125576645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 msgr2=0x55b6d7aa8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 -- 172.21.15.26:0/3125576645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 msgr2=0x55b6d7aa8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.240+0000 7f75eaffd700 1 -- 172.21.15.26:0/3125576645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d401a040 con 0x55b6d7aa0520 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b2e0 tx=0x7f75dc00ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a189abea283b57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f75d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00f870 con 0x55b6d7aa0520 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6d7a93d50 con 0x55b6d7aa0520 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f75dc00f9d0 con 0x55b6d7aa0520 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75dc00be90 con 0x55b6d7aa0520 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f76051fcb80 10 client.27182 fetch_fsmap learned FSMap version 49 2023-10-06T21:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.241+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b6d7807c00 con 0x55b6d7aa0520 2023-10-06T21:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6d7a93d50 con 0x55b6d7aa0520 2023-10-06T21:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f75dc017c50 con 0x55b6d7aa0520 2023-10-06T21:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 10 client.27182 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 10 client.27182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 20 client.27182 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7abb020 0x55b6d7abd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 4 client.27182 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6d79ecdb0 con 0x55b6d7abb020 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.242+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7abb020 0x55b6d7abd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.243+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7abb020 0x55b6d7abd440 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54bfc1211f431411 server_cookie=acd93a1c1f89de95 in_seq=0 out_seq=0 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.243+0000 7f75d27fc700 10 client.27182 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f75d27fc700 1 -- 172.21.15.26:0/3125576645 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7f75d402e730 con 0x55b6d7abb020 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f75d27fc700 10 client.27182 handle_command_reply: tid=0 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 client.27182 shutdown 2023-10-06T21:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 msgr2=0x55b6d7aa8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b2e0 tx=0x7f75dc00ba80 comp rx=0 tx=0).stop 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 shutdown_connections 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7abb020 0x55b6d7abd440 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3125576645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f75d27fc700 0 client.27182 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 >> 172.21.15.26:0/3125576645 conn(0x55b6d7a15200 msgr2=0x55b6d7a9bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.244+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 shutdown_connections 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.245+0000 7f76051fcb80 1 -- 172.21.15.26:0/3125576645 wait complete. 2023-10-06T21:22:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.245+0000 7f76051fcb80 20 client.27182 trim_cache size 0 max 16384 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 Processor -- start 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 -- start start 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d78748c0 con 0x55b6d7aab3d0 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7aa40a0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.268+0000 7f76051fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7a93890 con 0x55b6d7aa0520 2023-10-06T21:22:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52220/0 (socket says 172.21.15.26:52220) 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 -- 172.21.15.26:0/3486826309 learned_addr learned my addr 172.21.15.26:0/3486826309 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 -- 172.21.15.26:0/3486826309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 msgr2=0x55b6d7aa2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 -- 172.21.15.26:0/3486826309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 msgr2=0x55b6d7aa7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 -- 172.21.15.26:0/3486826309 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6d7ac2030 con 0x55b6d7aa0520 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.269+0000 7f75eaffd700 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ce0 tx=0x7f75dc0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ad7f3ab44567e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.270+0000 7f75e1ffb700 1 -- 172.21.15.26:0/3486826309 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc01c070 con 0x55b6d7aa0520 2023-10-06T21:22:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.270+0000 7f75e1ffb700 1 -- 172.21.15.26:0/3486826309 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75dc01f040 con 0x55b6d7aa0520 2023-10-06T21:22:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.270+0000 7f75e1ffb700 1 -- 172.21.15.26:0/3486826309 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc021850 con 0x55b6d7aa0520 2023-10-06T21:22:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.271+0000 7f76051fcb80 1 -- 172.21.15.26:0/3486826309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 msgr2=0x55b6d7aa8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.271+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009ce0 tx=0x7f75dc0058e0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 -- 172.21.15.26:0/3486826309 shutdown_connections 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aab3d0 0x55b6d7aa7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aa2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 --2- 172.21.15.26:0/3486826309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7aa0520 0x55b6d7aa8b00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 -- 172.21.15.26:0/3486826309 >> 172.21.15.26:0/3486826309 conn(0x55b6d7a15200 msgr2=0x55b6d7aa6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 -- 172.21.15.26:0/3486826309 shutdown_connections 2023-10-06T21:22:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.272+0000 7f76051fcb80 1 -- 172.21.15.26:0/3486826309 wait complete. 2023-10-06T21:22:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.273+0000 7f76051fcb80 1 -- start start 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.273+0000 7f76051fcb80 1 Processor -- start 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aafc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aab3d0 0x55b6d7ab01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b0df70 con 0x55b6d7aa0520 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b0e0e0 con 0x55b6d7aab3d0 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6d7b0e2a0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f76051fcb80 10 client.0 _ll_register_callbacks cb 0x7f75e40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52224/0 (socket says 172.21.15.26:52224) 2023-10-06T21:22:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f75ea7fc700 1 -- 172.21.15.26:0/2874202735 learned_addr learned my addr 172.21.15.26:0/2874202735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aab3d0 0x55b6d7ab01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.274+0000 7f75eaffd700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aafc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 -- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aab3d0 msgr2=0x55b6d7ab01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aab3d0 0x55b6d7ab01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 -- 172.21.15.26:0/2874202735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 msgr2=0x55b6d7aafc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aafc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 -- 172.21.15.26:0/2874202735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6d7807c00 con 0x55b6d7a9fc00 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75ea7fc700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b960 tx=0x7f75d400b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e875e7b58929bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4003db0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75d4003680 con 0x55b6d7a9fc00 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f75d4017040 con 0x55b6d7a9fc00 2023-10-06T21:22:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d4005300 con 0x55b6d7a9fc00 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f76051fcb80 10 client.27070 fetch_fsmap learned FSMap version 49 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.275+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b6d78451f0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d4003680 con 0x55b6d7a9fc00 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 49) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f75d4034cd0 con 0x55b6d7a9fc00 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 10 client.27070 fetch_fsmap finished waiting for FSMap version 49 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 10 client.27070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 20 client.27070 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7b0f810 0x55b6d7b11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 4 client.27070 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6d79ecad0 con 0x55b6d7b0f810 2023-10-06T21:22:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.276+0000 7f75eaffd700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7b0f810 0x55b6d7b11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.277+0000 7f75eaffd700 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7b0f810 0x55b6d7b11c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f91185e9f6a6cf7 server_cookie=86aab8aec9dd039e in_seq=0 out_seq=0 2023-10-06T21:22:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.277+0000 7f75d27fc700 10 client.27070 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f75d27fc700 1 -- 172.21.15.26:0/2874202735 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6d79ecad0 con 0x55b6d7b0f810 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f75d27fc700 10 client.27070 handle_command_reply: tid=0 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f76051fcb80 1 client.27070 shutdown 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 msgr2=0x55b6d7aaf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b960 tx=0x7f75d400b500 comp rx=0 tx=0).stop 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 shutdown_connections 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.303+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b6d7b0f810 0x55b6d7b11c30 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b6d7aab3d0 0x55b6d7ab01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6d7a9fc00 0x55b6d7aaf730 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 --2- 172.21.15.26:0/2874202735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b6d7aa0520 0x55b6d7aafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 >> 172.21.15.26:0/2874202735 conn(0x55b6d7a15200 msgr2=0x55b6d7aaa380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f75d27fc700 0 client.27070 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 shutdown_connections 2023-10-06T21:22:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 1 -- 172.21.15.26:0/2874202735 wait complete. 2023-10-06T21:22:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.304+0000 7f76051fcb80 20 client.27070 trim_cache size 0 max 16384 2023-10-06T21:22:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:22:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:22:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:22:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:22:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 msgr2=0x7f75d80a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 0x7f75d80a3bb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75e400ef90 tx=0x7f75e4004cc0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 msgr2=0x7f75ec1a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 0x7f75ec1a96e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f75e400efc0 tx=0x7f75e400c3a0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 shutdown_connections 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ec106f60 0x7f75ec1a96e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f75d80a16f0 0x7f75d80a3bb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec0682d0 0x7f75ec107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 --2- 172.21.15.26:0/553016848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75ec068c90 0x7f75ec108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 >> 172.21.15.26:0/553016848 conn(0x7f75ec076b70 msgr2=0x7f75ec10bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 shutdown_connections 2023-10-06T21:22:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:11.306+0000 7f75f17ff700 1 -- 172.21.15.26:0/553016848 wait complete. 2023-10-06T21:22:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 690.0 s, 1087.4 tps, lat 92.689 ms stddev 105.085 2023-10-06T21:22:12.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:12 smithi078 ceph-mon[134896]: pgmap v371: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 29 MiB/s wr, 1.96k op/s 2023-10-06T21:22:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:12 smithi167 ceph-mon[134494]: pgmap v371: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 29 MiB/s wr, 1.96k op/s 2023-10-06T21:22:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:12 smithi026 ceph-mon[98490]: pgmap v371: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 29 MiB/s wr, 1.96k op/s 2023-10-06T21:22:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:14 smithi167 ceph-mon[134494]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:22:14.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:14 smithi167 ceph-mon[134494]: pgmap v373: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-06T21:22:14.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:14 smithi026 ceph-mon[98490]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:22:14.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:14 smithi026 ceph-mon[98490]: pgmap v373: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-06T21:22:14.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:14 smithi078 ceph-mon[134896]: osdmap e101: 12 total, 12 up, 12 in 2023-10-06T21:22:14.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:14 smithi078 ceph-mon[134896]: pgmap v373: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 29 MiB/s wr, 2.02k op/s 2023-10-06T21:22:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:16 smithi026 ceph-mon[98490]: pgmap v374: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2023-10-06T21:22:16.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:16 smithi078 ceph-mon[134896]: pgmap v374: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2023-10-06T21:22:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:16 smithi167 ceph-mon[134494]: pgmap v374: 97 pgs: 97 active+clean; 5.3 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2023-10-06T21:22:17.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 695.0 s, 1230.0 tps, lat 79.812 ms stddev 82.252 2023-10-06T21:22:18.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:18 smithi026 ceph-mon[98490]: pgmap v375: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:18.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:18 smithi078 ceph-mon[134896]: pgmap v375: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:18 smithi167 ceph-mon[134494]: pgmap v375: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:20.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 -- 172.21.15.26:0/2677500875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 msgr2=0x7fb89811ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 --2- 172.21.15.26:0/2677500875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb89811ca80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb890009f90 tx=0x7fb89000b330 comp rx=0 tx=0).stop 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 -- 172.21.15.26:0/2677500875 shutdown_connections 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 --2- 172.21.15.26:0/2677500875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb89811ca80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 --2- 172.21.15.26:0/2677500875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb89810f640 0x7fb898107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 --2- 172.21.15.26:0/2677500875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8981083f0 0x7fb898108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.568+0000 7fb89ffff700 1 -- 172.21.15.26:0/2677500875 >> 172.21.15.26:0/2677500875 conn(0x7fb89806d1a0 msgr2=0x7fb89806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.569+0000 7fb89ffff700 1 -- 172.21.15.26:0/2677500875 shutdown_connections 2023-10-06T21:22:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.569+0000 7fb89ffff700 1 -- 172.21.15.26:0/2677500875 wait complete. 2023-10-06T21:22:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.570+0000 7fb89ffff700 1 Processor -- start 2023-10-06T21:22:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 -- start start 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8981083f0 0x7fb8981a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 0x7fb8981a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb8981aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981aae10 con 0x7fb8981083f0 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981aaf80 con 0x7fb89810f640 2023-10-06T21:22:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981ab140 con 0x7fb89811a690 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.571+0000 7fb89e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb8981aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.572+0000 7fb89d59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 0x7fb8981a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.572+0000 7fb89e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb8981aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52246/0 (socket says 172.21.15.26:52246) 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.572+0000 7fb89e59c700 1 -- 172.21.15.26:0/1666028918 learned_addr learned my addr 172.21.15.26:0/1666028918 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.572+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8981083f0 0x7fb8981a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 -- 172.21.15.26:0/1666028918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 msgr2=0x7fb8981aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb8981aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 -- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8981083f0 msgr2=0x7fb8981a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8981083f0 0x7fb8981a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 -- 172.21.15.26:0/1666028918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890009c40 con 0x7fb89810f640 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.573+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 0x7fb8981a6370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d950 tx=0x7fb88800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.574+0000 7fb88effd700 1 -- 172.21.15.26:0/1666028918 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888009930 con 0x7fb89810f640 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.574+0000 7fb88effd700 1 -- 172.21.15.26:0/1666028918 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb888010460 con 0x7fb89810f640 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.574+0000 7fb88effd700 1 -- 172.21.15.26:0/1666028918 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88800b5d0 con 0x7fb89810f640 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.574+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8981ab420 con 0x7fb89810f640 2023-10-06T21:22:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.574+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8981b2c60 con 0x7fb89810f640 2023-10-06T21:22:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.580+0000 7fb88effd700 1 -- 172.21.15.26:0/1666028918 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb888010840 con 0x7fb89810f640 2023-10-06T21:22:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.583+0000 7fb8b3964b80 1 Processor -- start 2023-10-06T21:22:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 -- start start 2023-10-06T21:22:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c64584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 0x55c0c6463350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 0x55c0c6465c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c63c1460 con 0x55c0c64580c0 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c63b2c20 con 0x55c0c64589e0 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.584+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c63b0540 con 0x55c0c6463890 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.585+0000 7fb89d59a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 0x55c0c6465c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.585+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 0x55c0c6463350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.585+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 0x55c0c6463350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:58772/0 (socket says 172.21.15.26:58772) 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.585+0000 7fb89dd9b700 1 -- 172.21.15.26:0/3138175921 learned_addr learned my addr 172.21.15.26:0/3138175921 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.585+0000 7fb89e59c700 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c64584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb89d59a700 1 -- 172.21.15.26:0/3138175921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 msgr2=0x55c0c6463350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 0x55c0c6463350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb89d59a700 1 -- 172.21.15.26:0/3138175921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 msgr2=0x55c0c64584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c64584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb89d59a700 1 -- 172.21.15.26:0/3138175921 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c61d6de0 con 0x55c0c6463890 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.586+0000 7fb88effd700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 0x7fb8840a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.587+0000 7fb88effd700 1 -- 172.21.15.26:0/1666028918 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb8880ccc80 con 0x7fb89810f640 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.587+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 0x55c0c6465c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb8880ca360 tx=0x7fb8880d0c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b80639ffe17d8df4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.587+0000 7fb883fff700 1 -- 172.21.15.26:0/3138175921 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888095f30 con 0x55c0c6463890 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.587+0000 7fb89e59c700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 0x7fb8840a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.588+0000 7fb883fff700 1 -- 172.21.15.26:0/3138175921 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8880d0890 con 0x55c0c6463890 2023-10-06T21:22:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.588+0000 7fb883fff700 1 -- 172.21.15.26:0/3138175921 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888099bf0 con 0x55c0c6463890 2023-10-06T21:22:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.589+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3138175921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 msgr2=0x55c0c6465c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.589+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 0x55c0c6465c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb8880ca360 tx=0x7fb8880d0c20 comp rx=0 tx=0).stop 2023-10-06T21:22:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3138175921 shutdown_connections 2023-10-06T21:22:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c6463890 0x55c0c6465c80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c64584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3138175921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c64589e0 0x55c0c6463350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3138175921 >> 172.21.15.26:0/3138175921 conn(0x55c0c63cd260 msgr2=0x55c0c6454450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3138175921 shutdown_connections 2023-10-06T21:22:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb89e59c700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 0x7fb8840a3b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb890009550 tx=0x7fb890006000 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.591+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3138175921 wait complete. 2023-10-06T21:22:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.592+0000 7fb8b3964b80 1 -- start start 2023-10-06T21:22:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.592+0000 7fb8b3964b80 1 Processor -- start 2023-10-06T21:22:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c6460b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c6461070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c63abe30 con 0x55c0c64589e0 2023-10-06T21:22:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c6348710 con 0x55c0c6463890 2023-10-06T21:22:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c63483b0 con 0x55c0c64580c0 2023-10-06T21:22:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.593+0000 7fb8b3964b80 10 client.0 _ll_register_callbacks cb 0x55c0c61a5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.594+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.594+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52260/0 (socket says 172.21.15.26:52260) 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.594+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1246059343 learned_addr learned my addr 172.21.15.26:0/1246059343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.594+0000 7fb89e59c700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c6461070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 msgr2=0x55c0c6461070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c6461070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1246059343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 msgr2=0x55c0c6460b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c6460b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c6460b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1246059343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8981b2c60 con 0x55c0c64580c0 2023-10-06T21:22:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c6460b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:22:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.595+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb8940104d0 tx=0x7fb894010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee3589c2410c89c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8817fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:22:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894014640 con 0x55c0c64580c0 2023-10-06T21:22:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb8940147a0 con 0x55c0c64580c0 2023-10-06T21:22:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0c63a4e10 con 0x55c0c64580c0 2023-10-06T21:22:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb894014960 con 0x55c0c64580c0 2023-10-06T21:22:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8b3964b80 10 client.27082 fetch_fsmap learned FSMap version 50 2023-10-06T21:22:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.596+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c0c61bfc60 con 0x55c0c64580c0 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0c63a4e10 con 0x55c0c64580c0 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 50) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb894013b00 con 0x55c0c64580c0 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 10 client.27082 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 10 client.27082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 20 client.27082 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64843e0 0x55c0c6486800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 4 client.27082 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.597+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0c63a4b30 con 0x55c0c64843e0 2023-10-06T21:22:20.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.598+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64843e0 0x55c0c6486800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.599+0000 7fb89d59a700 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64843e0 0x55c0c6486800 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2b73b220c48a1e server_cookie=7cd6f55615b38df in_seq=0 out_seq=0 2023-10-06T21:22:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.599+0000 7fb8817fa700 10 client.27082 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.601+0000 7fb8817fa700 1 -- 172.21.15.26:0/1246059343 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c0c63a4b30 con 0x55c0c64843e0 2023-10-06T21:22:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.601+0000 7fb8817fa700 10 client.27082 handle_command_reply: tid=0 2023-10-06T21:22:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.601+0000 7fb8b3964b80 1 client.27082 shutdown 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 msgr2=0x55c0c64605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb8940104d0 tx=0x7fb894010890 comp rx=0 tx=0).stop 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 shutdown_connections 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64843e0 0x55c0c6486800 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c6461070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64605f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1246059343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c6460b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8817fa700 0 client.27082 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.602+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 >> 172.21.15.26:0/1246059343 conn(0x55c0c63cd260 msgr2=0x55c0c6454040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.603+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 shutdown_connections 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.603+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1246059343 wait complete. 2023-10-06T21:22:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.603+0000 7fb8b3964b80 20 client.27082 trim_cache size 0 max 16384 2023-10-06T21:22:20.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.641+0000 7fb8b3964b80 1 Processor -- start 2023-10-06T21:22:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 -- start start 2023-10-06T21:22:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64589e0 0x55c0c645b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c645be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c622c920 con 0x55c0c64580c0 2023-10-06T21:22:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c6461ba0 con 0x55c0c6463890 2023-10-06T21:22:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.642+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c647b930 con 0x55c0c64589e0 2023-10-06T21:22:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.643+0000 7fb89e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c645be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.645+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.645+0000 7fb89dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58292/0 (socket says 172.21.15.26:58292) 2023-10-06T21:22:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1735197901 learned_addr learned my addr 172.21.15.26:0/1735197901 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1735197901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64589e0 msgr2=0x55c0c645b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64589e0 0x55c0c645b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1735197901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 msgr2=0x55c0c645be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c645be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.646+0000 7fb89dd9b700 1 -- 172.21.15.26:0/1735197901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c61fd250 con 0x55c0c64580c0 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.647+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb894013e90 tx=0x7fb894007140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80dbc4a1b560b8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.647+0000 7fb88cff9700 1 -- 172.21.15.26:0/1735197901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89402c070 con 0x55c0c64580c0 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.647+0000 7fb88cff9700 1 -- 172.21.15.26:0/1735197901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb894021070 con 0x55c0c64580c0 2023-10-06T21:22:20.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.647+0000 7fb88cff9700 1 -- 172.21.15.26:0/1735197901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89401c9d0 con 0x55c0c64580c0 2023-10-06T21:22:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1735197901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 msgr2=0x55c0c647b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb894013e90 tx=0x7fb894007140 comp rx=0 tx=0).stop 2023-10-06T21:22:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1735197901 shutdown_connections 2023-10-06T21:22:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c645be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64580c0 0x55c0c647b3f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/1735197901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64589e0 0x55c0c645b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.649+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1735197901 >> 172.21.15.26:0/1735197901 conn(0x55c0c63cd260 msgr2=0x55c0c647fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.652+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1735197901 shutdown_connections 2023-10-06T21:22:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.652+0000 7fb8b3964b80 1 -- 172.21.15.26:0/1735197901 wait complete. 2023-10-06T21:22:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.653+0000 7fb8b3964b80 1 -- start start 2023-10-06T21:22:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.653+0000 7fb8b3964b80 1 Processor -- start 2023-10-06T21:22:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c64cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c64cc780 con 0x55c0c64589e0 2023-10-06T21:22:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c64cc8f0 con 0x55c0c6463890 2023-10-06T21:22:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c64ccab0 con 0x55c0c64580c0 2023-10-06T21:22:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.654+0000 7fb8b3964b80 10 client.0 _ll_register_callbacks cb 0x7fb894007a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.656+0000 7fb89d59a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.656+0000 7fb89d59a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58302/0 (socket says 172.21.15.26:58302) 2023-10-06T21:22:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.656+0000 7fb89d59a700 1 -- 172.21.15.26:0/3320095036 learned_addr learned my addr 172.21.15.26:0/3320095036 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 -- 172.21.15.26:0/3320095036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 msgr2=0x55c0c64d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 -- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 msgr2=0x55c0c64cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c64cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 -- 172.21.15.26:0/3320095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c61bfc60 con 0x55c0c64589e0 2023-10-06T21:22:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb89d59a700 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb888005db0 tx=0x7fb88800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c1a401a8ea723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.657+0000 7fb8817fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:22:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.658+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8880d0220 con 0x55c0c64589e0 2023-10-06T21:22:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.658+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb88809a2f0 con 0x55c0c64589e0 2023-10-06T21:22:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.658+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0c644bf10 con 0x55c0c64589e0 2023-10-06T21:22:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.659+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb888099730 con 0x55c0c64589e0 2023-10-06T21:22:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.659+0000 7fb8b3964b80 10 client.16857 fetch_fsmap learned FSMap version 50 2023-10-06T21:22:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.659+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c0c61d52b0 con 0x55c0c64589e0 2023-10-06T21:22:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8880b6070 con 0x55c0c64589e0 2023-10-06T21:22:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 50) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb888096300 con 0x55c0c64589e0 2023-10-06T21:22:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 10 client.16857 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:22:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 20 client.16857 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64d9320 0x55c0c64db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 4 client.16857 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:22:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.660+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0c61f7f70 con 0x55c0c64d9320 2023-10-06T21:22:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.662+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64d9320 0x55c0c64db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.663+0000 7fb89dd9b700 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64d9320 0x55c0c64db740 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a55d39ce819d53b server_cookie=40506da2bf5b210a in_seq=0 out_seq=0 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.663+0000 7fb8817fa700 10 client.16857 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.663+0000 7fb8817fa700 1 -- 172.21.15.26:0/3320095036 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c0c61f7f70 con 0x55c0c64d9320 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.663+0000 7fb8817fa700 10 client.16857 handle_command_reply: tid=0 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.663+0000 7fb8b3964b80 1 client.16857 shutdown 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 msgr2=0x55c0c64d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb888005db0 tx=0x7fb88800d8d0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 shutdown_connections 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c0c64d9320 0x55c0c64db740 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0c6463890 0x55c0c64cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0c64580c0 0x55c0c64d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8817fa700 0 client.16857 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 --2- 172.21.15.26:0/3320095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c0c64589e0 0x55c0c64d1d70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.664+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 >> 172.21.15.26:0/3320095036 conn(0x55c0c63cd260 msgr2=0x55c0c645b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.666+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 shutdown_connections 2023-10-06T21:22:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.666+0000 7fb8b3964b80 1 -- 172.21.15.26:0/3320095036 wait complete. 2023-10-06T21:22:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.666+0000 7fb8b3964b80 20 client.16857 trim_cache size 0 max 16384 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:22:20.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:22:20.672 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:22:20.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:22:20.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:22:20.672 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:22:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 msgr2=0x7fb8840a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 0x7fb8840a3b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb890009550 tx=0x7fb890006000 comp rx=0 tx=0).stop 2023-10-06T21:22:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 msgr2=0x7fb8981a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 0x7fb8981a6370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb88800d950 tx=0x7fb88800dc60 comp rx=0 tx=0).stop 2023-10-06T21:22:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 shutdown_connections 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb89811a690 0x7fb8981aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8840a16b0 0x7fb8840a3b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8981083f0 0x7fb8981a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 --2- 172.21.15.26:0/1666028918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb89810f640 0x7fb8981a6370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 >> 172.21.15.26:0/1666028918 conn(0x7fb89806d1a0 msgr2=0x7fb898118680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.669+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 shutdown_connections 2023-10-06T21:22:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:20.670+0000 7fb89ffff700 1 -- 172.21.15.26:0/1666028918 wait complete. 2023-10-06T21:22:20.698 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:22:20.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:20 smithi078 ceph-mon[134896]: pgmap v376: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:20.884 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:20 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:20 smithi167 ceph-mon[134494]: pgmap v376: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:21.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:20 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:20 smithi026 ceph-mon[98490]: pgmap v376: 97 pgs: 97 active+clean; 5.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 649 op/s 2023-10-06T21:22:21.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:20 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 700.0 s, 1312.2 tps, lat 77.340 ms stddev 98.472 2023-10-06T21:22:22.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:22 smithi078 ceph-mon[134896]: pgmap v377: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 669 op/s 2023-10-06T21:22:23.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:22 smithi167 ceph-mon[134494]: pgmap v377: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 669 op/s 2023-10-06T21:22:23.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:22 smithi026 ceph-mon[98490]: pgmap v377: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 669 op/s 2023-10-06T21:22:24.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:24 smithi167 ceph-mon[134494]: pgmap v378: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 689 op/s 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.557+0000 7f73a8bff700 1 -- 172.21.15.26:0/1001997535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 msgr2=0x7f73a410d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.557+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1001997535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a410d980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7394009920 tx=0x7f7394009c30 comp rx=0 tx=0).stop 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 -- 172.21.15.26:0/1001997535 shutdown_connections 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1001997535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a4107ea0 0x7f73a4114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1001997535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a410d980 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1001997535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a41074e0 0x7f73a4107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 -- 172.21.15.26:0/1001997535 >> 172.21.15.26:0/1001997535 conn(0x7f73a4076cd0 msgr2=0x7f73a4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 -- 172.21.15.26:0/1001997535 shutdown_connections 2023-10-06T21:22:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.558+0000 7f73a8bff700 1 -- 172.21.15.26:0/1001997535 wait complete. 2023-10-06T21:22:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.559+0000 7f73a8bff700 1 Processor -- start 2023-10-06T21:22:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.559+0000 7f73a8bff700 1 -- start start 2023-10-06T21:22:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a4107ea0 0x7f73a41a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419da70 con 0x7f73a410d5a0 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419dbe0 con 0x7f73a41074e0 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a8bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419dda0 con 0x7f73a4107ea0 2023-10-06T21:22:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:50962/0 (socket says 172.21.15.26:50962) 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.560+0000 7f73a259c700 1 -- 172.21.15.26:0/1538137025 learned_addr learned my addr 172.21.15.26:0/1538137025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 -- 172.21.15.26:0/1538137025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a4107ea0 msgr2=0x7f73a41a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a4107ea0 0x7f73a41a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 -- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 msgr2=0x7f73a419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 -- 172.21.15.26:0/1538137025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009520 con 0x7f73a41074e0 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.561+0000 7f73a259c700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7394009230 tx=0x7f7394005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.562+0000 7f73937fe700 1 -- 172.21.15.26:0/1538137025 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739401c070 con 0x7f73a41074e0 2023-10-06T21:22:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.562+0000 7f73937fe700 1 -- 172.21.15.26:0/1538137025 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7394010ef0 con 0x7f73a41074e0 2023-10-06T21:22:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.562+0000 7f73937fe700 1 -- 172.21.15.26:0/1538137025 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394004b70 con 0x7f73a41074e0 2023-10-06T21:22:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.562+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a419e020 con 0x7f73a41074e0 2023-10-06T21:22:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.562+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41aa590 con 0x7f73a41074e0 2023-10-06T21:22:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.566+0000 7f73937fe700 1 -- 172.21.15.26:0/1538137025 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7394020030 con 0x7f73a41074e0 2023-10-06T21:22:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.567+0000 7f73bc51fb80 1 Processor -- start 2023-10-06T21:22:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 -- start start 2023-10-06T21:22:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8388fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8393e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae82f2400 con 0x55eae8389520 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae82e3bc0 con 0x55eae83943d0 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.568+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae82e14e0 con 0x55eae8388c00 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.569+0000 7f73a2d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8388fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.569+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.569+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:50978/0 (socket says 172.21.15.26:50978) 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.569+0000 7f73a1d9b700 1 -- 172.21.15.26:0/642014830 learned_addr learned my addr 172.21.15.26:0/642014830 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.570+0000 7f73a259c700 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8393e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.570+0000 7f73937fe700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 0x7f738c0a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73937fe700 1 -- 172.21.15.26:0/1538137025 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f73940ce460 con 0x7f73a41074e0 2023-10-06T21:22:24.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:24 smithi026 ceph-mon[98490]: pgmap v378: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 689 op/s 2023-10-06T21:22:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 -- 172.21.15.26:0/642014830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 msgr2=0x55eae8388fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8388fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 -- 172.21.15.26:0/642014830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 msgr2=0x55eae83967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 -- 172.21.15.26:0/642014830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eae8107d80 con 0x55eae8389520 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.571+0000 7f73a259c700 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8393e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73940cbba0 tx=0x7f7394009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5b9fee55550e3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.572+0000 7f7390ff9700 1 -- 172.21.15.26:0/642014830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739409b000 con 0x55eae8389520 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.572+0000 7f7390ff9700 1 -- 172.21.15.26:0/642014830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f739409ae20 con 0x55eae8389520 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.572+0000 7f7390ff9700 1 -- 172.21.15.26:0/642014830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940a37c0 con 0x55eae8389520 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.572+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8388fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.572+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 0x7f738c0a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.573+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 0x7f738c0a3c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73a41a42a0 tx=0x7f739c004ad0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 -- 172.21.15.26:0/642014830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 msgr2=0x55eae8393e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8393e90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73940cbba0 tx=0x7f7394009260 comp rx=0 tx=0).stop 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 -- 172.21.15.26:0/642014830 shutdown_connections 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8388fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/642014830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8393e90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 -- 172.21.15.26:0/642014830 >> 172.21.15.26:0/642014830 conn(0x55eae82fe200 msgr2=0x55eae8385080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.575+0000 7f73bc51fb80 1 -- 172.21.15.26:0/642014830 shutdown_connections 2023-10-06T21:22:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.576+0000 7f73bc51fb80 1 -- 172.21.15.26:0/642014830 wait complete. 2023-10-06T21:22:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.576+0000 7f73bc51fb80 1 -- start start 2023-10-06T21:22:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.577+0000 7f73bc51fb80 1 Processor -- start 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8393c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae82dcdd0 con 0x55eae8389520 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae82796b0 con 0x55eae83943d0 2023-10-06T21:22:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae8279350 con 0x55eae8388c00 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.578+0000 7f73bc51fb80 10 client.0 _ll_register_callbacks cb 0x55eae80d67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.579+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.579+0000 7f73a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.579+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55590/0 (socket says 172.21.15.26:55590) 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.579+0000 7f73a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83913f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:50990/0 (socket says 172.21.15.26:50990) 2023-10-06T21:22:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.579+0000 7f73a1d9b700 1 -- 172.21.15.26:0/3550370936 learned_addr learned my addr 172.21.15.26:0/3550370936 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 -- 172.21.15.26:0/3550370936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 msgr2=0x55eae8393c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8393c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 -- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 msgr2=0x55eae83913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 -- 172.21.15.26:0/3550370936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c009960 con 0x55eae8389520 2023-10-06T21:22:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.580+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f739800e480 tx=0x7f739800e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb01fce032fd1979 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.581+0000 7f7389ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:22:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.581+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398013470 con 0x55eae8389520 2023-10-06T21:22:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.581+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f73980135d0 con 0x55eae8389520 2023-10-06T21:22:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.581+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eae837cd50 con 0x55eae8389520 2023-10-06T21:22:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.582+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7398013790 con 0x55eae8389520 2023-10-06T21:22:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.582+0000 7f73bc51fb80 10 client.16869 fetch_fsmap learned FSMap version 50 2023-10-06T21:22:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.582+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f73a41aa590 con 0x55eae8389520 2023-10-06T21:22:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eae837cd50 con 0x55eae8389520 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 50) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7398011070 con 0x55eae8389520 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 10 client.16869 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 20 client.16869 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae83c5c90 0x55eae83c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 4 client.16869 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eae82d5db0 con 0x55eae83c5c90 2023-10-06T21:22:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.583+0000 7f73a259c700 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae83c5c90 0x55eae83c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.584+0000 7f73a259c700 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae83c5c90 0x55eae83c80b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5eaa2484713e2c server_cookie=a2ead1d4c143193a in_seq=0 out_seq=0 2023-10-06T21:22:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.584+0000 7f7389ffb700 10 client.16869 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.585+0000 7f7389ffb700 1 -- 172.21.15.26:0/3550370936 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eae82d5db0 con 0x55eae83c5c90 2023-10-06T21:22:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.585+0000 7f7389ffb700 10 client.16869 handle_command_reply: tid=0 2023-10-06T21:22:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.585+0000 7f73bc51fb80 1 client.16869 shutdown 2023-10-06T21:22:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 msgr2=0x55eae8390eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f739800e480 tx=0x7f739800e840 comp rx=0 tx=0).stop 2023-10-06T21:22:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 shutdown_connections 2023-10-06T21:22:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae83c5c90 0x55eae83c80b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae83913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae8393c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/3550370936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae8390eb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 >> 172.21.15.26:0/3550370936 conn(0x55eae82fe200 msgr2=0x55eae8384c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.586+0000 7f7389ffb700 0 client.16869 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.587+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 shutdown_connections 2023-10-06T21:22:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.587+0000 7f73bc51fb80 1 -- 172.21.15.26:0/3550370936 wait complete. 2023-10-06T21:22:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.589+0000 7f73bc51fb80 20 client.16869 trim_cache size 0 max 16384 2023-10-06T21:22:24.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:24 smithi078 ceph-mon[134896]: pgmap v378: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 689 op/s 2023-10-06T21:22:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.618+0000 7f73bc51fb80 1 Processor -- start 2023-10-06T21:22:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 -- start start 2023-10-06T21:22:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae83c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae83943d0 0x55eae8384250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae815d8c0 con 0x55eae83943d0 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae8391e30 con 0x55eae8389520 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.619+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae8385dc0 con 0x55eae8388c00 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51004/0 (socket says 172.21.15.26:51004) 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 -- 172.21.15.26:0/2078701059 learned_addr learned my addr 172.21.15.26:0/2078701059 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 -- 172.21.15.26:0/2078701059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 msgr2=0x55eae83c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae83c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 -- 172.21.15.26:0/2078701059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae83943d0 msgr2=0x55eae8384250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae83943d0 0x55eae8384250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 -- 172.21.15.26:0/2078701059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eae80f0c00 con 0x55eae8389520 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.620+0000 7f73a1d9b700 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7398006d20 tx=0x7f7398006f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0b9e347b50d123a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.621+0000 7f73917fa700 1 -- 172.21.15.26:0/2078701059 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398005d80 con 0x55eae8389520 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.621+0000 7f73917fa700 1 -- 172.21.15.26:0/2078701059 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f739801c900 con 0x55eae8389520 2023-10-06T21:22:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.621+0000 7f73917fa700 1 -- 172.21.15.26:0/2078701059 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398012d70 con 0x55eae8389520 2023-10-06T21:22:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 -- 172.21.15.26:0/2078701059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 msgr2=0x55eae83abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7398006d20 tx=0x7f7398006f50 comp rx=0 tx=0).stop 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 -- 172.21.15.26:0/2078701059 shutdown_connections 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae83943d0 0x55eae8384250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae83c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/2078701059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae8389520 0x55eae83abd40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.622+0000 7f73bc51fb80 1 -- 172.21.15.26:0/2078701059 >> 172.21.15.26:0/2078701059 conn(0x55eae82fe200 msgr2=0x55eae83c1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.623+0000 7f73bc51fb80 1 -- 172.21.15.26:0/2078701059 shutdown_connections 2023-10-06T21:22:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.623+0000 7f73bc51fb80 1 -- 172.21.15.26:0/2078701059 wait complete. 2023-10-06T21:22:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.623+0000 7f73bc51fb80 1 -- start start 2023-10-06T21:22:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.624+0000 7f73bc51fb80 1 Processor -- start 2023-10-06T21:22:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae840dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae840e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae8413060 con 0x55eae8389520 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae84131d0 con 0x55eae83943d0 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae8413390 con 0x55eae8388c00 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73bc51fb80 10 client.0 _ll_register_callbacks cb 0x7f73980099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73a2d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51010/0 (socket says 172.21.15.26:51010) 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.625+0000 7f73a2d9d700 1 -- 172.21.15.26:0/212111138 learned_addr learned my addr 172.21.15.26:0/212111138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a259c700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae840dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 -- 172.21.15.26:0/212111138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 msgr2=0x55eae840dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae840dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 -- 172.21.15.26:0/212111138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 msgr2=0x55eae840e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae840e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 -- 172.21.15.26:0/212111138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41aa590 con 0x55eae83943d0 2023-10-06T21:22:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.626+0000 7f73a2d9d700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f739c017c70 tx=0x7f739c0093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b100cdbf93873a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.631+0000 7f7389ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:22:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.631+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c016d80 con 0x55eae83943d0 2023-10-06T21:22:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.631+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f739c023420 con 0x55eae83943d0 2023-10-06T21:22:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.631+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f739409ae20 con 0x55eae83943d0 2023-10-06T21:22:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.632+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f739c009d90 con 0x55eae83943d0 2023-10-06T21:22:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.632+0000 7f73bc51fb80 10 client.27218 fetch_fsmap learned FSMap version 50 2023-10-06T21:22:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.632+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55eae812e1f0 con 0x55eae83943d0 2023-10-06T21:22:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.632+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f739c030030 con 0x55eae83943d0 2023-10-06T21:22:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 50) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f739c02a060 con 0x55eae83943d0 2023-10-06T21:22:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 10 client.27218 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T21:22:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 10 client.27218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 20 client.27218 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae841ad40 0x55eae841d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 4 client.27218 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eae82d5ad0 con 0x55eae841ad40 2023-10-06T21:22:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.633+0000 7f73a259c700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae841ad40 0x55eae841d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.634+0000 7f73a259c700 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae841ad40 0x55eae841d160 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a8e21d84871dee server_cookie=8ae6bb9b38b4e5d3 in_seq=0 out_seq=0 2023-10-06T21:22:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.634+0000 7f7389ffb700 10 client.27218 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.658+0000 7f7389ffb700 1 -- 172.21.15.26:0/212111138 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eae82d5ad0 con 0x55eae841ad40 2023-10-06T21:22:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.658+0000 7f7389ffb700 10 client.27218 handle_command_reply: tid=0 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.660+0000 7f73bc51fb80 1 client.27218 shutdown 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 msgr2=0x55eae8412af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f739c017c70 tx=0x7f739c0093a0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 shutdown_connections 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55eae841ad40 0x55eae841d160 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eae83943d0 0x55eae8412af0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae8388c00 0x55eae840dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 --2- 172.21.15.26:0/212111138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eae8389520 0x55eae840e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 >> 172.21.15.26:0/212111138 conn(0x55eae82fe200 msgr2=0x55eae8384250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f7389ffb700 0 client.27218 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 shutdown_connections 2023-10-06T21:22:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.661+0000 7f73bc51fb80 1 -- 172.21.15.26:0/212111138 wait complete. 2023-10-06T21:22:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.662+0000 7f73bc51fb80 20 client.27218 trim_cache size 0 max 16384 2023-10-06T21:22:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:22:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:22:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:22:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:22:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.666+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 msgr2=0x7f738c0a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.666+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 0x7f738c0a3c40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73a41a42a0 tx=0x7f739c004ad0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.666+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 msgr2=0x7f73a41a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.666+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7394009230 tx=0x7f7394005320 comp rx=0 tx=0).stop 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 shutdown_connections 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f738c0a1780 0x7f738c0a3c40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73a410d5a0 0x7f73a419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73a41074e0 0x7f73a41a35a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 --2- 172.21.15.26:0/1538137025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73a4107ea0 0x7f73a41a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 >> 172.21.15.26:0/1538137025 conn(0x7f73a4076cd0 msgr2=0x7f73a410b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 shutdown_connections 2023-10-06T21:22:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:24.667+0000 7f73a8bff700 1 -- 172.21.15.26:0/1538137025 wait complete. 2023-10-06T21:22:26.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:26 smithi078 ceph-mon[134896]: pgmap v379: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-06T21:22:27.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:26 smithi167 ceph-mon[134494]: pgmap v379: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-06T21:22:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:26 smithi026 ceph-mon[98490]: pgmap v379: 97 pgs: 97 active+clean; 5.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 586 op/s 2023-10-06T21:22:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 705.0 s, 1287.4 tps, lat 77.409 ms stddev 86.920 2023-10-06T21:22:28.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:28 smithi078 ceph-mon[134896]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 870 op/s 2023-10-06T21:22:29.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:28 smithi167 ceph-mon[134494]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 870 op/s 2023-10-06T21:22:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:28 smithi026 ceph-mon[98490]: pgmap v380: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 870 op/s 2023-10-06T21:22:30.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:30 smithi078 ceph-mon[134896]: pgmap v381: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-06T21:22:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:30 smithi167 ceph-mon[134494]: pgmap v381: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-06T21:22:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:30 smithi026 ceph-mon[98490]: pgmap v381: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-06T21:22:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 710.0 s, 945.8 tps, lat 105.195 ms stddev 145.864 2023-10-06T21:22:33.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:32 smithi167 ceph-mon[134494]: pgmap v382: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-10-06T21:22:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:32 smithi026 ceph-mon[98490]: pgmap v382: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-10-06T21:22:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:32 smithi078 ceph-mon[134896]: pgmap v382: 97 pgs: 97 active+clean; 5.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-10-06T21:22:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:34 smithi167 ceph-mon[134494]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-10-06T21:22:35.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:34 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:35.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:34 smithi026 ceph-mon[98490]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-10-06T21:22:35.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:34 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:35.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:34 smithi078 ceph-mon[134896]: pgmap v383: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-10-06T21:22:35.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:34 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:37.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:36 smithi167 ceph-mon[134494]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:22:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:36 smithi026 ceph-mon[98490]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:22:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 715.0 s, 1180.7 tps, lat 85.402 ms stddev 94.209 2023-10-06T21:22:37.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:36 smithi078 ceph-mon[134896]: pgmap v384: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-06T21:22:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.984+0000 7f7aa9bff700 1 -- 172.21.15.26:0/2577728283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa410fa90 msgr2=0x7f7aa41083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.984+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/2577728283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa410fa90 0x7f7aa41083f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a940098f0 tx=0x7f7a94009c00 comp rx=0 tx=0).stop 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.989+0000 7f7aa9bff700 1 -- 172.21.15.26:0/2577728283 shutdown_connections 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.989+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/2577728283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa411a690 0x7f7aa411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.989+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/2577728283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa410f620 0x7f7aa4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.989+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/2577728283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa410fa90 0x7f7aa41083f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.989+0000 7f7aa9bff700 1 -- 172.21.15.26:0/2577728283 >> 172.21.15.26:0/2577728283 conn(0x7f7aa406cfa0 msgr2=0x7f7aa406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.990+0000 7f7aa9bff700 1 -- 172.21.15.26:0/2577728283 shutdown_connections 2023-10-06T21:22:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.990+0000 7f7aa9bff700 1 -- 172.21.15.26:0/2577728283 wait complete. 2023-10-06T21:22:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.991+0000 7f7aa9bff700 1 Processor -- start 2023-10-06T21:22:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.991+0000 7f7aa9bff700 1 -- start start 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa410fa90 0x7f7aa4114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa411a690 0x7f7aa4079110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4079750 con 0x7f7aa410f620 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40798c0 con 0x7f7aa411a690 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.992+0000 7f7aa9bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4079a80 con 0x7f7aa410fa90 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.993+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.993+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55686/0 (socket says 172.21.15.26:55686) 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.993+0000 7f7aa37fe700 1 -- 172.21.15.26:0/588051587 learned_addr learned my addr 172.21.15.26:0/588051587 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.994+0000 7f7aa37fe700 1 -- 172.21.15.26:0/588051587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa410fa90 msgr2=0x7f7aa4114da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.994+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa410fa90 0x7f7aa4114da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.994+0000 7f7aa37fe700 1 -- 172.21.15.26:0/588051587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa411a690 msgr2=0x7f7aa4079110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.994+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa411a690 0x7f7aa4079110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.994+0000 7f7aa37fe700 1 -- 172.21.15.26:0/588051587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009520 con 0x7f7aa410f620 2023-10-06T21:22:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.995+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c007fa0 tx=0x7f7a9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.996+0000 7f7aa0ff9700 1 -- 172.21.15.26:0/588051587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00de60 con 0x7f7aa410f620 2023-10-06T21:22:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.996+0000 7f7aa0ff9700 1 -- 172.21.15.26:0/588051587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c00f040 con 0x7f7aa410f620 2023-10-06T21:22:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.996+0000 7f7aa0ff9700 1 -- 172.21.15.26:0/588051587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c014720 con 0x7f7aa410f620 2023-10-06T21:22:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.996+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4079d60 con 0x7f7aa410f620 2023-10-06T21:22:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:37.996+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa407a220 con 0x7f7aa410f620 2023-10-06T21:22:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.005+0000 7f7aa0ff9700 1 -- 172.21.15.26:0/588051587 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7a9c014880 con 0x7f7aa410f620 2023-10-06T21:22:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.006+0000 7f7abd5a7b80 1 Processor -- start 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.007+0000 7f7abd5a7b80 1 -- start start 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.007+0000 7f7aa0ff9700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 0x7f7a8c0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.007+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.007+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.007+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2df3d0 0x55a73f2e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f23d400 con 0x55a73f2d3c00 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f22ebc0 con 0x55a73f2d4520 2023-10-06T21:22:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f22c4e0 con 0x55a73f2df3d0 2023-10-06T21:22:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa0ff9700 1 -- 172.21.15.26:0/588051587 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7a9c0cc760 con 0x7f7aa410f620 2023-10-06T21:22:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55700/0 (socket says 172.21.15.26:55700) 2023-10-06T21:22:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa37fe700 1 -- 172.21.15.26:0/850826932 learned_addr learned my addr 172.21.15.26:0/850826932 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 0x7f7a8c0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.008+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2df3d0 0x55a73f2e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 -- 172.21.15.26:0/850826932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2df3d0 msgr2=0x55a73f2e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2df3d0 0x55a73f2e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 -- 172.21.15.26:0/850826932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 msgr2=0x55a73f2dee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 -- 172.21.15.26:0/850826932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73f052d80 con 0x55a73f2d3c00 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c095bd0 tx=0x7f7a9c0990b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d4a9f876761516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 0x7f7a8c0a3cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005e10 tx=0x7f7a98005d20 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.009+0000 7f7a89ffb700 1 -- 172.21.15.26:0/850826932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0abcb0 con 0x55a73f2d3c00 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7a89ffb700 1 -- 172.21.15.26:0/850826932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c0abad0 con 0x55a73f2d3c00 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7a89ffb700 1 -- 172.21.15.26:0/850826932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0a1a60 con 0x55a73f2d3c00 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/850826932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 msgr2=0x55a73f2d3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c095bd0 tx=0x7f7a9c0990b0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/850826932 shutdown_connections 2023-10-06T21:22:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2df3d0 0x55a73f2e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.010+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2d3fe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.011+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/850826932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.011+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/850826932 >> 172.21.15.26:0/850826932 conn(0x55a73f249200 msgr2=0x55a73f2d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.011+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/850826932 shutdown_connections 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.011+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/850826932 wait complete. 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.011+0000 7f7abd5a7b80 1 -- start start 2023-10-06T21:22:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.012+0000 7f7abd5a7b80 1 Processor -- start 2023-10-06T21:22:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2df3d0 0x55a73f2dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f227dd0 con 0x55a73f2df3d0 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f1c46b0 con 0x55a73f2d4520 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f1c4350 con 0x55a73f2d3c00 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7abd5a7b80 10 client.0 _ll_register_callbacks cb 0x55a73f0217a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43544/0 (socket says 172.21.15.26:43544) 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.013+0000 7f7aa37fe700 1 -- 172.21.15.26:0/2760855943 learned_addr learned my addr 172.21.15.26:0/2760855943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2df3d0 0x55a73f2dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 -- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 msgr2=0x55a73f2dc020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dc020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 -- 172.21.15.26:0/2760855943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2df3d0 msgr2=0x55a73f2dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2df3d0 0x55a73f2dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 -- 172.21.15.26:0/2760855943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa407a220 con 0x55a73f2d3c00 2023-10-06T21:22:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.014+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a1530 tx=0x7f7a9c0996d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7da4dc0bae438898 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.015+0000 7f7a82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.015+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0ad860 con 0x55a73f2d3c00 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.015+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7a9c0a1a60 con 0x55a73f2d3c00 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.015+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a73f2c7d50 con 0x55a73f2d3c00 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.015+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0ad110 con 0x55a73f2d3c00 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.016+0000 7f7abd5a7b80 10 client.27094 fetch_fsmap learned FSMap version 51 2023-10-06T21:22:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.016+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a73f03bc00 con 0x55a73f2d3c00 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.016+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a73f2c7d50 con 0x55a73f2d3c00 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.016+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 51) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7a9c099930 con 0x55a73f2d3c00 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 10 client.27094 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 10 client.27094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 20 client.27094 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f2eec10 0x55a73f2f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 4 client.27094 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.017+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a73f220db0 con 0x55a73f2eec10 2023-10-06T21:22:38.019 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.018+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f2eec10 0x55a73f2f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.019+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f2eec10 0x55a73f2f1030 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf3a220c0662c0d server_cookie=bae5657e6941def7 in_seq=0 out_seq=0 2023-10-06T21:22:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.019+0000 7f7a82ffd700 10 client.27094 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7a82ffd700 1 -- 172.21.15.26:0/2760855943 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a73f220db0 con 0x55a73f2eec10 2023-10-06T21:22:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7a82ffd700 10 client.27094 handle_command_reply: tid=0 2023-10-06T21:22:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 client.27094 shutdown 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 msgr2=0x55a73f2deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a1530 tx=0x7f7a9c0996d0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 shutdown_connections 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2df3d0 0x55a73f2dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d3c00 0x55a73f2deda0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f2eec10 0x55a73f2f1030 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/2760855943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2d4520 0x55a73f2dc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 >> 172.21.15.26:0/2760855943 conn(0x55a73f249200 msgr2=0x55a73f2cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.020+0000 7f7a82ffd700 0 client.27094 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.021+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 shutdown_connections 2023-10-06T21:22:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.021+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/2760855943 wait complete. 2023-10-06T21:22:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.022+0000 7f7abd5a7b80 20 client.27094 trim_cache size 0 max 16384 2023-10-06T21:22:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.045+0000 7f7abd5a7b80 1 Processor -- start 2023-10-06T21:22:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.045+0000 7f7abd5a7b80 1 -- start start 2023-10-06T21:22:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f2db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f0a88c0 con 0x55a73f2d3c00 2023-10-06T21:22:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f2cecf0 con 0x55a73f2df3d0 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f2d0dc0 con 0x55a73f2d4520 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2f1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55706/0 (socket says 172.21.15.26:55706) 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7aa37fe700 1 -- 172.21.15.26:0/1645214673 learned_addr learned my addr 172.21.15.26:0/1645214673 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f2db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.046+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 -- 172.21.15.26:0/1645214673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 msgr2=0x55a73f2ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 -- 172.21.15.26:0/1645214673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 msgr2=0x55a73f2f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 -- 172.21.15.26:0/1645214673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73f0791f0 con 0x55a73f2df3d0 2023-10-06T21:22:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.047+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f2db820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005950 tx=0x7f7a98016070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d7f773fe0f32cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.049+0000 7f7a8a7fc700 1 -- 172.21.15.26:0/1645214673 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98009bf0 con 0x55a73f2df3d0 2023-10-06T21:22:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.049+0000 7f7a8a7fc700 1 -- 172.21.15.26:0/1645214673 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a98004cf0 con 0x55a73f2df3d0 2023-10-06T21:22:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.049+0000 7f7a8a7fc700 1 -- 172.21.15.26:0/1645214673 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98005570 con 0x55a73f2df3d0 2023-10-06T21:22:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/1645214673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 msgr2=0x55a73f2db820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f2db820 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005950 tx=0x7f7a98016070 comp rx=0 tx=0).stop 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/1645214673 shutdown_connections 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f2db820 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/1645214673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/1645214673 >> 172.21.15.26:0/1645214673 conn(0x55a73f249200 msgr2=0x55a73f2f5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.050+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/1645214673 shutdown_connections 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.051+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/1645214673 wait complete. 2023-10-06T21:22:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.051+0000 7f7abd5a7b80 1 -- start start 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.051+0000 7f7abd5a7b80 1 Processor -- start 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f341a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f341f80 con 0x55a73f2d3c00 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f3420c0 con 0x55a73f2df3d0 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73f342200 con 0x55a73f2d4520 2023-10-06T21:22:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.052+0000 7f7abd5a7b80 10 client.0 _ll_register_callbacks cb 0x7f7a98009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:43572/0 (socket says 172.21.15.26:43572) 2023-10-06T21:22:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 -- 172.21.15.26:0/3828800094 learned_addr learned my addr 172.21.15.26:0/3828800094 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa3fff700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f341a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 -- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 msgr2=0x55a73f341a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f341a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 -- 172.21.15.26:0/3828800094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 msgr2=0x55a73f2e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 -- 172.21.15.26:0/3828800094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a98019040 con 0x55a73f2d4520 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7aa2ffd700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4116080 tx=0x7f7a94022540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211d9e18ba76e1d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.053+0000 7f7a82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9401d040 con 0x55a73f2d4520 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a9c0abad0 con 0x55a73f2d4520 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7a9400fa10 con 0x55a73f2d4520 2023-10-06T21:22:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a94022b00 con 0x55a73f2d4520 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 10 client.27100 fetch_fsmap learned FSMap version 51 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a73f051250 con 0x55a73f2d4520 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c0abad0 con 0x55a73f2d4520 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 51) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7a94015020 con 0x55a73f2d4520 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 10 client.27100 fetch_fsmap finished waiting for FSMap version 51 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 10 client.27100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.054+0000 7f7abd5a7b80 20 client.27100 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.055+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f343960 0x55a73f345d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.055+0000 7f7abd5a7b80 4 client.27100 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.055+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a73f220ad0 con 0x55a73f343960 2023-10-06T21:22:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.055+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f343960 0x55a73f345d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.056+0000 7f7aa37fe700 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f343960 0x55a73f345d80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca93a8221dc72bf server_cookie=1a287618c4214ab6 in_seq=0 out_seq=0 2023-10-06T21:22:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.056+0000 7f7a82ffd700 10 client.27100 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.094+0000 7f7a82ffd700 1 -- 172.21.15.26:0/3828800094 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a73f220ad0 con 0x55a73f343960 2023-10-06T21:22:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.094+0000 7f7a82ffd700 10 client.27100 handle_command_reply: tid=0 2023-10-06T21:22:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.096+0000 7f7abd5a7b80 1 client.27100 shutdown 2023-10-06T21:22:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 msgr2=0x55a73f2e4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4116080 tx=0x7f7a94022540 comp rx=0 tx=0).stop 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 shutdown_connections 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a73f343960 0x55a73f345d80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a73f2df3d0 0x55a73f341a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a73f2d3c00 0x55a73f2e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 --2- 172.21.15.26:0/3828800094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a73f2d4520 0x55a73f2e4630 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 >> 172.21.15.26:0/3828800094 conn(0x55a73f249200 msgr2=0x55a73f2d7390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7a82ffd700 0 client.27100 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.097+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 shutdown_connections 2023-10-06T21:22:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.098+0000 7f7abd5a7b80 1 -- 172.21.15.26:0/3828800094 wait complete. 2023-10-06T21:22:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.098+0000 7f7abd5a7b80 20 client.27100 trim_cache size 0 max 16384 2023-10-06T21:22:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:22:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:22:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:22:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:22:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 msgr2=0x7f7a8c0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 0x7f7a8c0a3cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005e10 tx=0x7f7a98005d20 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 msgr2=0x7f7aa4114860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c007fa0 tx=0x7f7a9c00d3b0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 shutdown_connections 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa411a690 0x7f7aa4079110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a8c0a1800 0x7f7a8c0a3cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa410f620 0x7f7aa4114860 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 --2- 172.21.15.26:0/588051587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa410fa90 0x7f7aa4114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 >> 172.21.15.26:0/588051587 conn(0x7f7aa406cfa0 msgr2=0x7f7aa411bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 shutdown_connections 2023-10-06T21:22:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:38.099+0000 7f7aa9bff700 1 -- 172.21.15.26:0/588051587 wait complete. 2023-10-06T21:22:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:38 smithi167 ceph-mon[134494]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 22 MiB/s wr, 1.76k op/s 2023-10-06T21:22:39.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:38 smithi026 ceph-mon[98490]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 22 MiB/s wr, 1.76k op/s 2023-10-06T21:22:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:38 smithi078 ceph-mon[134896]: pgmap v385: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 22 MiB/s wr, 1.76k op/s 2023-10-06T21:22:40.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:39 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:40.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:39 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:40.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:39 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:40 smithi078 ceph-mon[134896]: pgmap v386: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:22:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:40 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:41.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:40 smithi167 ceph-mon[134494]: pgmap v386: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:22:41.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:40 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:41.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:40 smithi026 ceph-mon[98490]: pgmap v386: 97 pgs: 97 active+clean; 5.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:22:41.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:40 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:22:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 720.0 s, 1239.7 tps, lat 80.886 ms stddev 80.832 2023-10-06T21:22:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:42 smithi078 ceph-mon[134896]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-10-06T21:22:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:42 smithi167 ceph-mon[134494]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-10-06T21:22:43.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:42 smithi026 ceph-mon[98490]: pgmap v387: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2023-10-06T21:22:44.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:44 smithi167 ceph-mon[134494]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-10-06T21:22:44.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:44 smithi026 ceph-mon[98490]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-10-06T21:22:44.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:44 smithi078 ceph-mon[134896]: pgmap v388: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-10-06T21:22:46.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:46 smithi026 ceph-mon[98490]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-10-06T21:22:46.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:46 smithi078 ceph-mon[134896]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-10-06T21:22:47.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:46 smithi167 ceph-mon[134494]: pgmap v389: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-10-06T21:22:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 725.0 s, 1271.2 tps, lat 77.633 ms stddev 79.507 2023-10-06T21:22:48.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:48 smithi026 ceph-mon[98490]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-10-06T21:22:48.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:48 smithi078 ceph-mon[134896]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-10-06T21:22:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:48 smithi167 ceph-mon[134494]: pgmap v390: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-10-06T21:22:50.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:22:50.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:50 smithi078 ceph-mon[134896]: pgmap v391: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.5 MiB/s wr, 523 op/s 2023-10-06T21:22:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:50 smithi167 ceph-mon[134494]: pgmap v391: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.5 MiB/s wr, 523 op/s 2023-10-06T21:22:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.060+0000 7f002b3ff700 1 -- 172.21.15.26:0/2574190038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 msgr2=0x7f00240ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.060+0000 7f002b3ff700 1 --2- 172.21.15.26:0/2574190038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f00240ff360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f00200098f0 tx=0x7f0020009c00 comp rx=0 tx=0).stop 2023-10-06T21:22:51.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.061+0000 7f002b3ff700 1 -- 172.21.15.26:0/2574190038 shutdown_connections 2023-10-06T21:22:51.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.061+0000 7f002b3ff700 1 --2- 172.21.15.26:0/2574190038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f002410faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.061+0000 7f002b3ff700 1 --2- 172.21.15.26:0/2574190038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f00240ff360 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.061+0000 7f002b3ff700 1 --2- 172.21.15.26:0/2574190038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 0x7f00240ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.061+0000 7f002b3ff700 1 -- 172.21.15.26:0/2574190038 >> 172.21.15.26:0/2574190038 conn(0x7f0024077330 msgr2=0x7f00240fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.062+0000 7f002b3ff700 1 -- 172.21.15.26:0/2574190038 shutdown_connections 2023-10-06T21:22:51.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.062+0000 7f002b3ff700 1 -- 172.21.15.26:0/2574190038 wait complete. 2023-10-06T21:22:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.064+0000 7f002b3ff700 1 Processor -- start 2023-10-06T21:22:51.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.064+0000 7f002b3ff700 1 -- start start 2023-10-06T21:22:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f0024102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f0024102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 0x7f00241031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002410e2b0 con 0x7f00240ffec0 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002410e420 con 0x7f0024101d80 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002b3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002410e5e0 con 0x7f0024102160 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f0024102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002919b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f0024102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42364/0 (socket says 172.21.15.26:42364) 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002919b700 1 -- 172.21.15.26:0/1166135554 learned_addr learned my addr 172.21.15.26:0/1166135554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002999c700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 0x7f00241031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.065+0000 7f002899a700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f0024102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 -- 172.21.15.26:0/1166135554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 msgr2=0x7f00241031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 0x7f00241031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 -- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 msgr2=0x7f0024102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f0024102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 -- 172.21.15.26:0/1166135554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009d00 con 0x7f0024101d80 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002899a700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f0024102c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f001a7fc700 1 -- 172.21.15.26:0/1166135554 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001400c9c0 con 0x7f0024101d80 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f001a7fc700 1 -- 172.21.15.26:0/1166135554 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0014010430 con 0x7f0024101d80 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.066+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0020009520 con 0x7f0024101d80 2023-10-06T21:22:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.067+0000 7f001a7fc700 1 -- 172.21.15.26:0/1166135554 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014018680 con 0x7f0024101d80 2023-10-06T21:22:51.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.067+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002410ec90 con 0x7f0024101d80 2023-10-06T21:22:51.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:50 smithi026 ceph-mon[98490]: pgmap v391: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.5 MiB/s wr, 523 op/s 2023-10-06T21:22:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.072+0000 7f001a7fc700 1 -- 172.21.15.26:0/1166135554 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0014010860 con 0x7f0024101d80 2023-10-06T21:22:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 Processor -- start 2023-10-06T21:22:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 -- start start 2023-10-06T21:22:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f4fd460 con 0x560e0f5949e0 2023-10-06T21:22:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f4eec20 con 0x560e0f5940c0 2023-10-06T21:22:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.074+0000 7f003ed96b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f4ec540 con 0x560e0f59f890 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53968/0 (socket says 172.21.15.26:53968) 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 -- 172.21.15.26:0/3788007106 learned_addr learned my addr 172.21.15.26:0/3788007106 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002919b700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002899a700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 -- 172.21.15.26:0/3788007106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 msgr2=0x560e0f5a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 -- 172.21.15.26:0/3788007106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 msgr2=0x560e0f59f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002999c700 1 -- 172.21.15.26:0/3788007106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0f312de0 con 0x560e0f5940c0 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002919b700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.075+0000 7f002899a700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f002999c700 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f001c009530 tx=0x7f001c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a63923734c10bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f001a7fc700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 0x7f00100a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f001a7fc700 1 -- 172.21.15.26:0/1166135554 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0014014070 con 0x7f0024101d80 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f000f7fe700 1 -- 172.21.15.26:0/3788007106 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00e070 con 0x560e0f5940c0 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f000f7fe700 1 -- 172.21.15.26:0/3788007106 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x560e0f5940c0 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f000f7fe700 1 -- 172.21.15.26:0/3788007106 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c004290 con 0x560e0f5940c0 2023-10-06T21:22:51.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.076+0000 7f002919b700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 0x7f00100a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 -- 172.21.15.26:0/3788007106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 msgr2=0x560e0f5944a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f001c009530 tx=0x7f001c0099e0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f002919b700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 0x7f00100a3b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f002000c010 tx=0x7f0020019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 -- 172.21.15.26:0/3788007106 shutdown_connections 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5944a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 --2- 172.21.15.26:0/3788007106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 -- 172.21.15.26:0/3788007106 >> 172.21.15.26:0/3788007106 conn(0x560e0f509260 msgr2=0x560e0f590450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 -- 172.21.15.26:0/3788007106 shutdown_connections 2023-10-06T21:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.077+0000 7f003ed96b80 1 -- 172.21.15.26:0/3788007106 wait complete. 2023-10-06T21:22:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.078+0000 7f003ed96b80 1 -- start start 2023-10-06T21:22:51.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.078+0000 7f003ed96b80 1 Processor -- start 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 0x560e0f59cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f4e7e30 con 0x560e0f5949e0 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f484710 con 0x560e0f59f890 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f4843b0 con 0x560e0f5940c0 2023-10-06T21:22:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.079+0000 7f003ed96b80 10 client.0 _ll_register_callbacks cb 0x560e0f2e1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41496/0 (socket says 172.21.15.26:41496) 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 -- 172.21.15.26:0/1013881898 learned_addr learned my addr 172.21.15.26:0/1013881898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002999c700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 0x560e0f59cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002899a700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 -- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 msgr2=0x560e0f59cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 0x560e0f59cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 -- 172.21.15.26:0/1013881898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 msgr2=0x560e0f59c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002919b700 1 -- 172.21.15.26:0/1013881898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c0091e0 con 0x560e0f5940c0 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002999c700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 0x560e0f59cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.080+0000 7f002899a700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.081+0000 7f002919b700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f002000bc40 tx=0x7f0020009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307eae0b57aee6de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.083+0000 7f000cff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.083+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002002a070 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.083+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f002000f580 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.083+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e0f587f10 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.084+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f002000f940 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.084+0000 7f003ed96b80 10 client.27109 fetch_fsmap learned FSMap version 53 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.084+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x560e0f2fbc60 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f002003b080 con 0x560e0f5940c0 2023-10-06T21:22:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f00200235d0 con 0x560e0f5940c0 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 10 client.27109 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 20 client.27109 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f5af300 0x560e0f5b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 4 client.27109 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.085+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e0f4e0e10 con 0x560e0f5af300 2023-10-06T21:22:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.086+0000 7f002899a700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f5af300 0x560e0f5b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.086+0000 7f000cff9700 10 client.27109 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.086+0000 7f002899a700 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f5af300 0x560e0f5b1720 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b10d34d26fdbbf26 server_cookie=aba39f56d7d85166 in_seq=0 out_seq=0 2023-10-06T21:22:51.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.087+0000 7f000cff9700 1 -- 172.21.15.26:0/1013881898 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e0f4e0e10 con 0x560e0f5af300 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.087+0000 7f000cff9700 10 client.27109 handle_command_reply: tid=0 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.087+0000 7f003ed96b80 1 client.27109 shutdown 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 msgr2=0x560e0f59f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f002000bc40 tx=0x7f0020009c00 comp rx=0 tx=0).stop 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 shutdown_connections 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f5af300 0x560e0f5b1720 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f59f890 0x560e0f59cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f59f2c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 --2- 172.21.15.26:0/1013881898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f59c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 >> 172.21.15.26:0/1013881898 conn(0x560e0f509260 msgr2=0x560e0f58ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f000cff9700 0 client.27109 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 shutdown_connections 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 1 -- 172.21.15.26:0/1013881898 wait complete. 2023-10-06T21:22:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.088+0000 7f003ed96b80 20 client.27109 trim_cache size 0 max 16384 2023-10-06T21:22:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.118+0000 7f003ed96b80 1 Processor -- start 2023-10-06T21:22:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.118+0000 7f003ed96b80 1 -- start start 2023-10-06T21:22:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f5a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5949e0 0x560e0f597850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f368920 con 0x560e0f59f890 2023-10-06T21:22:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f594800 con 0x560e0f5949e0 2023-10-06T21:22:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.119+0000 7f003ed96b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f591280 con 0x560e0f5940c0 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42390/0 (socket says 172.21.15.26:42390) 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 -- 172.21.15.26:0/967219103 learned_addr learned my addr 172.21.15.26:0/967219103 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002899a700 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5949e0 0x560e0f597850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 -- 172.21.15.26:0/967219103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 msgr2=0x560e0f5a63e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f5a63e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 -- 172.21.15.26:0/967219103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5949e0 msgr2=0x560e0f597850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5949e0 0x560e0f597850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.120+0000 7f002999c700 1 -- 172.21.15.26:0/967219103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0f58c140 con 0x560e0f59f890 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.121+0000 7f002999c700 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f001c007c30 tx=0x7f001c004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f348a836845419 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.122+0000 7f000ffff700 1 -- 172.21.15.26:0/967219103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0178f0 con 0x560e0f59f890 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.122+0000 7f000ffff700 1 -- 172.21.15.26:0/967219103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c017a50 con 0x560e0f59f890 2023-10-06T21:22:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.122+0000 7f000ffff700 1 -- 172.21.15.26:0/967219103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c020690 con 0x560e0f59f890 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 -- 172.21.15.26:0/967219103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 msgr2=0x560e0f597d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f001c007c30 tx=0x7f001c004f70 comp rx=0 tx=0).stop 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 -- 172.21.15.26:0/967219103 shutdown_connections 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f59f890 0x560e0f597d90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f5940c0 0x560e0f5a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 --2- 172.21.15.26:0/967219103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5949e0 0x560e0f597850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 -- 172.21.15.26:0/967219103 >> 172.21.15.26:0/967219103 conn(0x560e0f509260 msgr2=0x560e0f5b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 -- 172.21.15.26:0/967219103 shutdown_connections 2023-10-06T21:22:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.123+0000 7f003ed96b80 1 -- 172.21.15.26:0/967219103 wait complete. 2023-10-06T21:22:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.124+0000 7f003ed96b80 1 -- start start 2023-10-06T21:22:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.124+0000 7f003ed96b80 1 Processor -- start 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f5a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f5fc660 con 0x560e0f5949e0 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f5fc7a0 con 0x560e0f5940c0 2023-10-06T21:22:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0f5fc960 con 0x560e0f59f890 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f003ed96b80 10 client.0 _ll_register_callbacks cb 0x560e0f2e1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f002919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f002919b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53992/0 (socket says 172.21.15.26:53992) 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.125+0000 7f002919b700 1 -- 172.21.15.26:0/2576767156 learned_addr learned my addr 172.21.15.26:0/2576767156 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002899a700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f5a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002999c700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 -- 172.21.15.26:0/2576767156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 msgr2=0x560e0f5fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 -- 172.21.15.26:0/2576767156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 msgr2=0x560e0f5a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f5a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 -- 172.21.15.26:0/2576767156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e0f312de0 con 0x560e0f5940c0 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002999c700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002899a700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f5a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f002919b700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0020009440 tx=0x7f0020009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c017eda44588f2ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.126+0000 7f000cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002002a070 con 0x560e0f5940c0 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f001c00f040 con 0x560e0f5940c0 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0020023ca0 con 0x560e0f5940c0 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0020023b20 con 0x560e0f5940c0 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f003ed96b80 10 client.27248 fetch_fsmap learned FSMap version 53 2023-10-06T21:22:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x560e0f339250 con 0x560e0f5940c0 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x560e0f5940c0 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.127+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0020041030 con 0x560e0f5940c0 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 10 client.27248 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 20 client.27248 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f603ef0 0x560e0f606310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 4 client.27248 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:22:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e0f4e0b30 con 0x560e0f603ef0 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f002899a700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f603ef0 0x560e0f606310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f002899a700 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f603ef0 0x560e0f606310 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1b4f5d6f837fa3 server_cookie=b8813b5776bbb83d in_seq=0 out_seq=0 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.128+0000 7f000cff9700 10 client.27248 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f000cff9700 1 -- 172.21.15.26:0/2576767156 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560e0f4e0b30 con 0x560e0f603ef0 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f000cff9700 10 client.27248 handle_command_reply: tid=0 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 client.27248 shutdown 2023-10-06T21:22:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 msgr2=0x560e0f5a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0020009440 tx=0x7f0020009230 comp rx=0 tx=0).stop 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 shutdown_connections 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e0f59f890 0x560e0f5fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e0f603ef0 0x560e0f606310 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e0f5940c0 0x560e0f5a4280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 --2- 172.21.15.26:0/2576767156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e0f5949e0 0x560e0f5a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 >> 172.21.15.26:0/2576767156 conn(0x560e0f509260 msgr2=0x560e0f597850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f000cff9700 0 client.27248 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.129+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 shutdown_connections 2023-10-06T21:22:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.130+0000 7f003ed96b80 1 -- 172.21.15.26:0/2576767156 wait complete. 2023-10-06T21:22:51.142 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:22:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:22:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:22:51.144 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.130+0000 7f003ed96b80 20 client.27248 trim_cache size 0 max 16384 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.131+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 msgr2=0x7f00100a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.131+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 0x7f00100a3b70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f002000c010 tx=0x7f0020019040 comp rx=0 tx=0).stop 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.131+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 msgr2=0x7f0024102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.131+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f0024102c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400ed00 comp rx=0 tx=0).stop 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 shutdown_connections 2023-10-06T21:22:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0024102160 0x7f00241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f00100a16b0 0x7f00100a3b70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00240ffec0 0x7f0024102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 --2- 172.21.15.26:0/1166135554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0024101d80 0x7f0024102c90 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400ed00 comp rx=0 tx=0).stop 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.132+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 >> 172.21.15.26:0/1166135554 conn(0x7f0024077330 msgr2=0x7f00240fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.133+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 shutdown_connections 2023-10-06T21:22:51.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:22:51.133+0000 7f002b3ff700 1 -- 172.21.15.26:0/1166135554 wait complete. 2023-10-06T21:22:51.172 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:22:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.346+0000 7fc36edff700 1 -- 172.21.15.26:0/2720969333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3680ffab0 msgr2=0x7fc368108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.346+0000 7fc36edff700 1 --2- 172.21.15.26:0/2720969333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3680ffab0 0x7fc368108f00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc354009990 tx=0x7fc354009ca0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.372+0000 7fc36edff700 1 -- 172.21.15.26:0/2720969333 shutdown_connections 2023-10-06T21:22:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.372+0000 7fc36edff700 1 --2- 172.21.15.26:0/2720969333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc368109630 0x7fc36810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.372+0000 7fc36edff700 1 --2- 172.21.15.26:0/2720969333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3680ff190 0x7fc3680ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.372+0000 7fc36edff700 1 --2- 172.21.15.26:0/2720969333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3680ffab0 0x7fc368108f00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.372+0000 7fc36edff700 1 -- 172.21.15.26:0/2720969333 >> 172.21.15.26:0/2720969333 conn(0x7fc368076e70 msgr2=0x7fc3680fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.383+0000 7fc36edff700 1 -- 172.21.15.26:0/2720969333 shutdown_connections 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.385+0000 7fc36edff700 1 -- 172.21.15.26:0/2720969333 wait complete. 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.386+0000 7fc36edff700 1 Processor -- start 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.386+0000 7fc36edff700 1 -- start start 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3680ff190 0x7fc36810edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3680ffab0 0x7fc36810f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681151f0 con 0x7fc368109630 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368115360 con 0x7fc3680ffab0 2023-10-06T21:22:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.387+0000 7fc36edff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc368115520 con 0x7fc3680ff190 2023-10-06T21:22:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.389+0000 7fc36d39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.389+0000 7fc36d39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42432/0 (socket says 172.21.15.26:42432) 2023-10-06T21:22:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.389+0000 7fc36d39c700 1 -- 172.21.15.26:0/4072213261 learned_addr learned my addr 172.21.15.26:0/4072213261 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.391+0000 7fc36d39c700 1 -- 172.21.15.26:0/4072213261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3680ff190 msgr2=0x7fc36810edb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:22:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.391+0000 7fc36d39c700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3680ff190 0x7fc36810edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.391+0000 7fc36d39c700 1 -- 172.21.15.26:0/4072213261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3680ffab0 msgr2=0x7fc36810f2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:22:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.391+0000 7fc36d39c700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3680ffab0 0x7fc36810f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.391+0000 7fc36d39c700 1 -- 172.21.15.26:0/4072213261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540095f0 con 0x7fc368109630 2023-10-06T21:22:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.392+0000 7fc36d39c700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc360007f10 tx=0x7fc36000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.399+0000 7fc365ffb700 1 -- 172.21.15.26:0/4072213261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360017070 con 0x7fc368109630 2023-10-06T21:22:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.399+0000 7fc365ffb700 1 -- 172.21.15.26:0/4072213261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36000f040 con 0x7fc368109630 2023-10-06T21:22:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.399+0000 7fc365ffb700 1 -- 172.21.15.26:0/4072213261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360005740 con 0x7fc368109630 2023-10-06T21:22:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.400+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368115800 con 0x7fc368109630 2023-10-06T21:22:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.400+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36811d1a0 con 0x7fc368109630 2023-10-06T21:22:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.407+0000 7fc38267fb80 1 Processor -- start 2023-10-06T21:22:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.418+0000 7fc365ffb700 1 -- 172.21.15.26:0/4072213261 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc3600058a0 con 0x7fc368109630 2023-10-06T21:22:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.418+0000 7fc38267fb80 1 -- start start 2023-10-06T21:22:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567df9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567dfa520 0x558567e04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567e077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567d63400 con 0x558567e053d0 2023-10-06T21:22:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567d54bc0 con 0x558567df9c00 2023-10-06T21:22:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.419+0000 7fc38267fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567d524e0 con 0x558567dfa520 2023-10-06T21:22:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.422+0000 7fc367fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567df9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.422+0000 7fc367fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567df9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54016/0 (socket says 172.21.15.26:54016) 2023-10-06T21:22:51.424 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.422+0000 7fc367fff700 1 -- 172.21.15.26:0/3096806796 learned_addr learned my addr 172.21.15.26:0/3096806796 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.422+0000 7fc365ffb700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 0x7fc3580a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.422+0000 7fc365ffb700 1 -- 172.21.15.26:0/4072213261 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc3600cc750 con 0x7fc368109630 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.423+0000 7fc367fff700 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567e077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567dfa520 0x558567e04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc367fff700 1 -- 172.21.15.26:0/3096806796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567dfa520 msgr2=0x558567e04e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc367fff700 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567dfa520 0x558567e04e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc367fff700 1 -- 172.21.15.26:0/3096806796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 msgr2=0x558567df9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc367fff700 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567df9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.424+0000 7fc367fff700 1 -- 172.21.15.26:0/3096806796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558567b78d80 con 0x558567e053d0 2023-10-06T21:22:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.425+0000 7fc367fff700 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567e077c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc354000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c49027e6962c8ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.425+0000 7fc36d39c700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 0x7fc3580a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.428+0000 7fc352ffd700 1 -- 172.21.15.26:0/3096806796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400b9d0 con 0x558567e053d0 2023-10-06T21:22:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.428+0000 7fc352ffd700 1 -- 172.21.15.26:0/3096806796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35400be80 con 0x558567e053d0 2023-10-06T21:22:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.428+0000 7fc352ffd700 1 -- 172.21.15.26:0/3096806796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400f460 con 0x558567e053d0 2023-10-06T21:22:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.429+0000 7fc38267fb80 1 -- 172.21.15.26:0/3096806796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 msgr2=0x558567e077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.429+0000 7fc38267fb80 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567e077c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc354000bc0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.431+0000 7fc38267fb80 1 -- 172.21.15.26:0/3096806796 shutdown_connections 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.431+0000 7fc38267fb80 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567e077c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.431+0000 7fc38267fb80 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567df9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.431+0000 7fc38267fb80 1 --2- 172.21.15.26:0/3096806796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567dfa520 0x558567e04e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.431+0000 7fc38267fb80 1 -- 172.21.15.26:0/3096806796 >> 172.21.15.26:0/3096806796 conn(0x558567d6f200 msgr2=0x558567df6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.432+0000 7fc38267fb80 1 -- 172.21.15.26:0/3096806796 shutdown_connections 2023-10-06T21:22:51.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.433+0000 7fc38267fb80 1 -- 172.21.15.26:0/3096806796 wait complete. 2023-10-06T21:22:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.434+0000 7fc38267fb80 1 -- start start 2023-10-06T21:22:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.434+0000 7fc38267fb80 1 Processor -- start 2023-10-06T21:22:51.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.434+0000 7fc36d39c700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 0x7fc3580a3c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3600995b0 tx=0x7fc36000b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567e04e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 0x558567e020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567d4ddd0 con 0x558567dfa520 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567cea6b0 con 0x558567df9c00 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567cea350 con 0x558567e053d0 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.435+0000 7fc38267fb80 10 client.0 _ll_register_callbacks cb 0x558567b477a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.436+0000 7fc367fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567e04e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.436+0000 7fc36cb9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 0x558567e020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.436+0000 7fc367fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567e04e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54030/0 (socket says 172.21.15.26:54030) 2023-10-06T21:22:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.436+0000 7fc367fff700 1 -- 172.21.15.26:0/1145185825 learned_addr learned my addr 172.21.15.26:0/1145185825 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 -- 172.21.15.26:0/1145185825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 msgr2=0x558567e027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 -- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 msgr2=0x558567e04e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567e04e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 -- 172.21.15.26:0/1145185825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681107f0 con 0x558567dfa520 2023-10-06T21:22:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 0x558567e020b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009f70 tx=0x7fc35c00e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f862420017e90e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.437+0000 7fc347fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:22:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004a30 con 0x558567dfa520 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc35c004b90 con 0x558567dfa520 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558567d46db0 con 0x558567dfa520 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c004d50 con 0x558567dfa520 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc38267fb80 10 client.16917 fetch_fsmap learned FSMap version 53 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.438+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x558567b61c00 con 0x558567dfa520 2023-10-06T21:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558567d46db0 con 0x558567dfa520 2023-10-06T21:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc35c00c070 con 0x558567dfa520 2023-10-06T21:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 10 client.16917 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 20 client.16917 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e15050 0x558567e17470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 4 client.16917 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558567d46ad0 con 0x558567e15050 2023-10-06T21:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.439+0000 7fc367fff700 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e15050 0x558567e17470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.441+0000 7fc367fff700 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e15050 0x558567e17470 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b112e313d9b1ca3 server_cookie=c33ef8f8299e3873 in_seq=0 out_seq=0 2023-10-06T21:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.441+0000 7fc347fff700 10 client.16917 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.441+0000 7fc347fff700 1 -- 172.21.15.26:0/1145185825 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558567d46ad0 con 0x558567e15050 2023-10-06T21:22:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.441+0000 7fc347fff700 10 client.16917 handle_command_reply: tid=0 2023-10-06T21:22:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 client.16917 shutdown 2023-10-06T21:22:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 msgr2=0x558567e020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 0x558567e020b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009f70 tx=0x7fc35c00e680 comp rx=0 tx=0).stop 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 shutdown_connections 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.442+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e15050 0x558567e17470 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567df9c00 0x558567e04e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 1 --2- 172.21.15.26:0/1145185825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567dfa520 0x558567e020b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc347fff700 0 client.16917 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 >> 172.21.15.26:0/1145185825 conn(0x558567d6f200 msgr2=0x558567df5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 shutdown_connections 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 1 -- 172.21.15.26:0/1145185825 wait complete. 2023-10-06T21:22:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.443+0000 7fc38267fb80 20 client.16917 trim_cache size 0 max 16384 2023-10-06T21:22:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.466+0000 7fc38267fb80 1 Processor -- start 2023-10-06T21:22:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.466+0000 7fc38267fb80 1 -- start start 2023-10-06T21:22:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.466+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567df9c00 0x558567e0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.466+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567dfc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc38267fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567bce8c0 con 0x558567e053d0 2023-10-06T21:22:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc38267fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567dfb3f0 con 0x558567dfa520 2023-10-06T21:22:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc38267fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567df2860 con 0x558567df9c00 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc36cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc36cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54038/0 (socket says 172.21.15.26:54038) 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc36cb9b700 1 -- 172.21.15.26:0/2111132511 learned_addr learned my addr 172.21.15.26:0/2111132511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc367fff700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567df9c00 0x558567e0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.467+0000 7fc367fff700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567dfc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 -- 172.21.15.26:0/2111132511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567df9c00 msgr2=0x558567e0bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567df9c00 0x558567e0bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 -- 172.21.15.26:0/2111132511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 msgr2=0x558567dfc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567dfc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 -- 172.21.15.26:0/2111132511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558567b9f1f0 con 0x558567dfa520 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc367fff700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567dfc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009f40 tx=0x7fc35c0093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5535a44916697f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc3537fe700 1 -- 172.21.15.26:0/2111132511 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c011470 con 0x558567dfa520 2023-10-06T21:22:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc3537fe700 1 -- 172.21.15.26:0/2111132511 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c0115d0 con 0x558567dfa520 2023-10-06T21:22:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.468+0000 7fc3537fe700 1 -- 172.21.15.26:0/2111132511 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c003b40 con 0x558567dfa520 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.469+0000 7fc38267fb80 1 -- 172.21.15.26:0/2111132511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 msgr2=0x558567dfc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.469+0000 7fc38267fb80 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009f40 tx=0x7fc35c0093f0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.469+0000 7fc38267fb80 1 -- 172.21.15.26:0/2111132511 shutdown_connections 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567e053d0 0x558567dfc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567df9c00 0x558567e0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 --2- 172.21.15.26:0/2111132511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567dfc1c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 -- 172.21.15.26:0/2111132511 >> 172.21.15.26:0/2111132511 conn(0x558567d6f200 msgr2=0x558567dfbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 -- 172.21.15.26:0/2111132511 shutdown_connections 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 -- 172.21.15.26:0/2111132511 wait complete. 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.470+0000 7fc38267fb80 1 -- start start 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 Processor -- start 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567df9c00 0x558567e0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567e08830 con 0x558567df9c00 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567e089a0 con 0x558567dfa520 2023-10-06T21:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.471+0000 7fc38267fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558567e08b60 con 0x558567e053d0 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc38267fb80 10 client.0 _ll_register_callbacks cb 0x7fc35c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc367fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567df9c00 0x558567e0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54050/0 (socket says 172.21.15.26:54050) 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 -- 172.21.15.26:0/4088268912 learned_addr learned my addr 172.21.15.26:0/4088268912 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc367fff700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 -- 172.21.15.26:0/4088268912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 msgr2=0x558567e080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 -- 172.21.15.26:0/4088268912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567df9c00 msgr2=0x558567e0b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567df9c00 0x558567e0b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.472+0000 7fc36cb9b700 1 -- 172.21.15.26:0/4088268912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558567b61c00 con 0x558567dfa520 2023-10-06T21:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.473+0000 7fc36cb9b700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02a040 tx=0x7fc35c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b983a001faf0120f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.473+0000 7fc347fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.473+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0050d0 con 0x558567dfa520 2023-10-06T21:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.473+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc35c000e40 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558567dedd50 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c005230 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc38267fb80 10 client.27263 fetch_fsmap learned FSMap version 53 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x558567b77250 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558567dedd50 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.474+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fc35c038070 con 0x558567dfa520 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 10 client.27263 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 10 client.27263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 20 client.27263 populate_metadata read hostname 'smithi026' 2023-10-06T21:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e69d60 0x558567e6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 4 client.27263 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558567b99f10 con 0x558567e69d60 2023-10-06T21:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.475+0000 7fc367fff700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e69d60 0x558567e6c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.476+0000 7fc347fff700 10 client.27263 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.476+0000 7fc367fff700 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e69d60 0x558567e6c180 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1eb7580cb5c73b server_cookie=800d70df64a70c5a in_seq=0 out_seq=0 2023-10-06T21:22:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc347fff700 1 -- 172.21.15.26:0/4088268912 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558567b99f10 con 0x558567e69d60 2023-10-06T21:22:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc347fff700 10 client.27263 handle_command_reply: tid=0 2023-10-06T21:22:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 client.27263 shutdown 2023-10-06T21:22:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 msgr2=0x558567e0bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc35c02a040 tx=0x7fc35c005bf0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 shutdown_connections 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558567e053d0 0x558567e080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558567df9c00 0x558567e0b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x558567e69d60 0x558567e6c180 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 --2- 172.21.15.26:0/4088268912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558567dfa520 0x558567e0bf10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 >> 172.21.15.26:0/4088268912 conn(0x558567d6f200 msgr2=0x558567dfc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.489+0000 7fc347fff700 0 client.27263 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.490+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 shutdown_connections 2023-10-06T21:22:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.490+0000 7fc38267fb80 1 -- 172.21.15.26:0/4088268912 wait complete. 2023-10-06T21:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.490+0000 7fc38267fb80 20 client.27263 trim_cache size 0 max 16384 2023-10-06T21:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:22:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 msgr2=0x7fc3580a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 0x7fc3580a3c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc3600995b0 tx=0x7fc36000b040 comp rx=0 tx=0).stop 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 msgr2=0x7fc368114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc360007f10 tx=0x7fc36000d3b0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 shutdown_connections 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc3580a17c0 0x7fc3580a3c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc368109630 0x7fc368114cb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3680ff190 0x7fc36810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 --2- 172.21.15.26:0/4072213261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3680ffab0 0x7fc36810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:22:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 >> 172.21.15.26:0/4072213261 conn(0x7fc368076e70 msgr2=0x7fc3680fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:22:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 shutdown_connections 2023-10-06T21:22:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:22:51.491+0000 7fc36edff700 1 -- 172.21.15.26:0/4072213261 wait complete. 2023-10-06T21:22:52.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:51 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:52.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:51 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:52.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:51 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:22:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 730.0 s, 1433.8 tps, lat 70.369 ms stddev 83.310 2023-10-06T21:22:53.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: pgmap v392: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:52 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.078 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: pgmap v392: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:52 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: pgmap v392: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 764 op/s 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:52 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:54.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:54 smithi026 ceph-mon[98490]: pgmap v393: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.2 MiB/s wr, 649 op/s 2023-10-06T21:22:54.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:54 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:54.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:54 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:54.574 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:54 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:54.624 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:54 smithi078 ceph-mon[134896]: pgmap v393: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.2 MiB/s wr, 649 op/s 2023-10-06T21:22:54.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:54 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:54.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:54 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:54.625 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:54 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:54 smithi167 ceph-mon[134494]: pgmap v393: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.2 MiB/s wr, 649 op/s 2023-10-06T21:22:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:54 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:22:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:54 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:22:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:54 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:22:56.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:56 smithi026 ceph-mon[98490]: pgmap v394: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 570 op/s 2023-10-06T21:22:56.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:56 smithi078 ceph-mon[134896]: pgmap v394: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 570 op/s 2023-10-06T21:22:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:56 smithi167 ceph-mon[134494]: pgmap v394: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.2 MiB/s wr, 570 op/s 2023-10-06T21:22:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 735.0 s, 1191.8 tps, lat 82.995 ms stddev 87.277 2023-10-06T21:22:58.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:22:58 smithi026 ceph-mon[98490]: pgmap v395: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 822 op/s 2023-10-06T21:22:58.881 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:22:58 smithi078 ceph-mon[134896]: pgmap v395: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 822 op/s 2023-10-06T21:22:59.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:22:58 smithi167 ceph-mon[134494]: pgmap v395: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 822 op/s 2023-10-06T21:23:00.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:00 smithi026 ceph-mon[98490]: pgmap v396: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 574 op/s 2023-10-06T21:23:00.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:00 smithi078 ceph-mon[134896]: pgmap v396: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 574 op/s 2023-10-06T21:23:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:00 smithi167 ceph-mon[134494]: pgmap v396: 97 pgs: 97 active+clean; 5.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 574 op/s 2023-10-06T21:23:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 740.0 s, 924.6 tps, lat 109.944 ms stddev 155.241 2023-10-06T21:23:02.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:02 smithi078 ceph-mon[134896]: pgmap v397: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 771 op/s 2023-10-06T21:23:03.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:02 smithi167 ceph-mon[134494]: pgmap v397: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 771 op/s 2023-10-06T21:23:03.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:02 smithi026 ceph-mon[98490]: pgmap v397: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 771 op/s 2023-10-06T21:23:04.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:04 smithi078 ceph-mon[134896]: pgmap v398: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 930 op/s 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 -- 172.21.15.26:0/2729148000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072411a690 msgr2=0x7f072411ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 --2- 172.21.15.26:0/2729148000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072411a690 0x7f072411ca80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f071c00b0a0 tx=0x7f071c00b3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 -- 172.21.15.26:0/2729148000 shutdown_connections 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 --2- 172.21.15.26:0/2729148000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072411a690 0x7f072411ca80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 --2- 172.21.15.26:0/2729148000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072410f620 0x7f0724107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 --2- 172.21.15.26:0/2729148000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f07241083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.777+0000 7f0729fff700 1 -- 172.21.15.26:0/2729148000 >> 172.21.15.26:0/2729148000 conn(0x7f072406cfa0 msgr2=0x7f072406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.778+0000 7f0729fff700 1 -- 172.21.15.26:0/2729148000 shutdown_connections 2023-10-06T21:23:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.778+0000 7f0729fff700 1 -- 172.21.15.26:0/2729148000 wait complete. 2023-10-06T21:23:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.780+0000 7f0729fff700 1 Processor -- start 2023-10-06T21:23:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:04 smithi167 ceph-mon[134494]: pgmap v398: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 930 op/s 2023-10-06T21:23:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.780+0000 7f0729fff700 1 -- start start 2023-10-06T21:23:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072410f620 0x7f07241189c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 0x7f07241117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0724111fb0 con 0x7f072411a690 2023-10-06T21:23:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0724112120 con 0x7f072410f620 2023-10-06T21:23:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0729fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07241122e0 con 0x7f072410fa90 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0722ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0722ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49024/0 (socket says 172.21.15.26:49024) 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0722ffd700 1 -- 172.21.15.26:0/1545277999 learned_addr learned my addr 172.21.15.26:0/1545277999 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.781+0000 7f0723fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 0x7f07241117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f07237fe700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072410f620 0x7f07241189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 -- 172.21.15.26:0/1545277999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072410f620 msgr2=0x7f07241189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072410f620 0x7f07241189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 -- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 msgr2=0x7f07241117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 0x7f07241117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 -- 172.21.15.26:0/1545277999 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c009d00 con 0x7f072410fa90 2023-10-06T21:23:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0723fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 0x7f07241117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.782+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f071800b550 tx=0x7f071800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.783+0000 7f0720ff9700 1 -- 172.21.15.26:0/1545277999 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718011830 con 0x7f072410fa90 2023-10-06T21:23:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.783+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07241125c0 con 0x7f072410fa90 2023-10-06T21:23:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.783+0000 7f0720ff9700 1 -- 172.21.15.26:0/1545277999 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0718011e50 con 0x7f072410fa90 2023-10-06T21:23:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.783+0000 7f0720ff9700 1 -- 172.21.15.26:0/1545277999 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800f2e0 con 0x7f072410fa90 2023-10-06T21:23:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.783+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07241168c0 con 0x7f072410fa90 2023-10-06T21:23:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.790+0000 7f0720ff9700 1 -- 172.21.15.26:0/1545277999 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0718011990 con 0x7f072410fa90 2023-10-06T21:23:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.793+0000 7f0720ff9700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 0x7f070c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.793+0000 7f073da0cb80 1 Processor -- start 2023-10-06T21:23:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.793+0000 7f0720ff9700 1 -- 172.21.15.26:0/1545277999 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f07180ccaa0 con 0x7f072410fa90 2023-10-06T21:23:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.794+0000 7f07237fe700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 0x7f070c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.794+0000 7f073da0cb80 1 -- start start 2023-10-06T21:23:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 0x55cc086f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc087007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0865c400 con 0x55cc086f3520 2023-10-06T21:23:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0864dbc0 con 0x55cc086f2c00 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f073da0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0864b4e0 con 0x55cc086fe3d0 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f0723fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc087007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f07237fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 0x55cc086f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f0723fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc087007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49034/0 (socket says 172.21.15.26:49034) 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f0723fff700 1 -- 172.21.15.26:0/65146264 learned_addr learned my addr 172.21.15.26:0/65146264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.795+0000 7f07237fe700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 0x7f070c0a3bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0714005950 tx=0x7f071400b410 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 -- 172.21.15.26:0/65146264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 msgr2=0x55cc087007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc087007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 -- 172.21.15.26:0/65146264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 msgr2=0x55cc086fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 -- 172.21.15.26:0/65146264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08471d80 con 0x55cc086f2c00 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f07237fe700 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 0x55cc086f2fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0714010de0 tx=0x7f0714013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b0020469eb49f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f0709ffb700 1 -- 172.21.15.26:0/65146264 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714011bb0 con 0x55cc086f2c00 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f0709ffb700 1 -- 172.21.15.26:0/65146264 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0714015070 con 0x55cc086f2c00 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.796+0000 7f0709ffb700 1 -- 172.21.15.26:0/65146264 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140055d0 con 0x55cc086f2c00 2023-10-06T21:23:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 -- 172.21.15.26:0/65146264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 msgr2=0x55cc086f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 0x55cc086f2fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0714010de0 tx=0x7f0714013000 comp rx=0 tx=0).stop 2023-10-06T21:23:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 -- 172.21.15.26:0/65146264 shutdown_connections 2023-10-06T21:23:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc087007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f2c00 0x55cc086f2fe0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 --2- 172.21.15.26:0/65146264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.797+0000 7f073da0cb80 1 -- 172.21.15.26:0/65146264 >> 172.21.15.26:0/65146264 conn(0x55cc08668200 msgr2=0x55cc086ef080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.798+0000 7f073da0cb80 1 -- 172.21.15.26:0/65146264 shutdown_connections 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.798+0000 7f073da0cb80 1 -- 172.21.15.26:0/65146264 wait complete. 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.798+0000 7f073da0cb80 1 -- start start 2023-10-06T21:23:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 Processor -- start 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc086fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086fe3d0 0x55cc086fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08646dd0 con 0x55cc086fe3d0 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc085e36b0 con 0x55cc086f3520 2023-10-06T21:23:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc085e3350 con 0x55cc086f2c00 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.799+0000 7f073da0cb80 10 client.0 _ll_register_callbacks cb 0x55cc084407a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49046/0 (socket says 172.21.15.26:49046) 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 -- 172.21.15.26:0/2097347446 learned_addr learned my addr 172.21.15.26:0/2097347446 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f0722ffd700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc086fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f0723fff700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086fe3d0 0x55cc086fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 -- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 msgr2=0x55cc086fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc086fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 -- 172.21.15.26:0/2097347446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086fe3d0 msgr2=0x55cc086fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086fe3d0 0x55cc086fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.800+0000 7f07237fe700 1 -- 172.21.15.26:0/2097347446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071400be70 con 0x55cc086f2c00 2023-10-06T21:23:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.801+0000 7f07237fe700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0714005170 tx=0x7f0714012ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d5fa042cccfb1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.801+0000 7f0702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:23:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.802+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071402c070 con 0x55cc086f2c00 2023-10-06T21:23:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.802+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0714017070 con 0x55cc086f2c00 2023-10-06T21:23:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.807+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc086e6d50 con 0x55cc086f2c00 2023-10-06T21:23:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.807+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0714005560 con 0x55cc086f2c00 2023-10-06T21:23:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.807+0000 7f073da0cb80 10 client.27130 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.807+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55cc0845ac00 con 0x55cc086f2c00 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc086e6d50 con 0x55cc086f2c00 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0714039070 con 0x55cc086f2c00 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 10 client.27130 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 10 client.27130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 20 client.27130 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc0870de20 0x55cc08710240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 4 client.27130 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc0863fdb0 con 0x55cc0870de20 2023-10-06T21:23:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.808+0000 7f0722ffd700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc0870de20 0x55cc08710240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.809+0000 7f0702ffd700 10 client.27130 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.809+0000 7f0722ffd700 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc0870de20 0x55cc08710240 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7a0589401e6af1 server_cookie=d5e832c5271ce7d9 in_seq=0 out_seq=0 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f0702ffd700 1 -- 172.21.15.26:0/2097347446 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc0863fdb0 con 0x55cc0870de20 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f0702ffd700 10 client.27130 handle_command_reply: tid=0 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 client.27130 shutdown 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 msgr2=0x55cc086fb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0714005170 tx=0x7f0714012ea0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 shutdown_connections 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086fe3d0 0x55cc086fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc086fb1d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc0870de20 0x55cc08710240 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2097347446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc086fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 >> 172.21.15.26:0/2097347446 conn(0x55cc08668200 msgr2=0x55cc086eec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.810+0000 7f0702ffd700 0 client.27130 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.811+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 shutdown_connections 2023-10-06T21:23:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.811+0000 7f073da0cb80 1 -- 172.21.15.26:0/2097347446 wait complete. 2023-10-06T21:23:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.811+0000 7f073da0cb80 20 client.27130 trim_cache size 0 max 16384 2023-10-06T21:23:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:04 smithi026 ceph-mon[98490]: pgmap v398: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 930 op/s 2023-10-06T21:23:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.835+0000 7f073da0cb80 1 Processor -- start 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.836+0000 7f073da0cb80 1 -- start start 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc08704f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc084c78c0 con 0x55cc086f3520 2023-10-06T21:23:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc086f5b30 con 0x55cc086fe3d0 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.837+0000 7f073da0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc086efdc0 con 0x55cc086f2c00 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.838+0000 7f0723fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.838+0000 7f0723fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45034/0 (socket says 172.21.15.26:45034) 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.838+0000 7f0723fff700 1 -- 172.21.15.26:0/2663196356 learned_addr learned my addr 172.21.15.26:0/2663196356 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.839+0000 7f0723fff700 1 -- 172.21.15.26:0/2663196356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 msgr2=0x55cc08704f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.839+0000 7f0723fff700 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc08704f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.839+0000 7f0723fff700 1 -- 172.21.15.26:0/2663196356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 msgr2=0x55cc086f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.839+0000 7f0723fff700 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.839+0000 7f0723fff700 1 -- 172.21.15.26:0/2663196356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc084981f0 con 0x55cc086fe3d0 2023-10-06T21:23:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.840+0000 7f0723fff700 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f071c0092f0 tx=0x7f071c009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e59b1dbebf86bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.841+0000 7f070a7fc700 1 -- 172.21.15.26:0/2663196356 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00e040 con 0x55cc086fe3d0 2023-10-06T21:23:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.841+0000 7f070a7fc700 1 -- 172.21.15.26:0/2663196356 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f071c020070 con 0x55cc086fe3d0 2023-10-06T21:23:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.841+0000 7f070a7fc700 1 -- 172.21.15.26:0/2663196356 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c005500 con 0x55cc086fe3d0 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 -- 172.21.15.26:0/2663196356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 msgr2=0x55cc086f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f071c0092f0 tx=0x7f071c009320 comp rx=0 tx=0).stop 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 -- 172.21.15.26:0/2663196356 shutdown_connections 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086fe3d0 0x55cc086f6a10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086f2c00 0x55cc08704f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 --2- 172.21.15.26:0/2663196356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f3520 0x55cc086f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 -- 172.21.15.26:0/2663196356 >> 172.21.15.26:0/2663196356 conn(0x55cc08668200 msgr2=0x55cc08709670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 -- 172.21.15.26:0/2663196356 shutdown_connections 2023-10-06T21:23:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.842+0000 7f073da0cb80 1 -- 172.21.15.26:0/2663196356 wait complete. 2023-10-06T21:23:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.843+0000 7f073da0cb80 1 -- start start 2023-10-06T21:23:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.843+0000 7f073da0cb80 1 Processor -- start 2023-10-06T21:23:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f2c00 0x55cc08755c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc08756680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08760d20 con 0x55cc086f2c00 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08760e60 con 0x55cc086f3520 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08761020 con 0x55cc086fe3d0 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f073da0cb80 10 client.0 _ll_register_callbacks cb 0x7f071c00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.844+0000 7f0722ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45042/0 (socket says 172.21.15.26:45042) 2023-10-06T21:23:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 -- 172.21.15.26:0/1286720316 learned_addr learned my addr 172.21.15.26:0/1286720316 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0723fff700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc08756680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f07237fe700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f2c00 0x55cc08755c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 -- 172.21.15.26:0/1286720316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 msgr2=0x55cc08756680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc08756680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 -- 172.21.15.26:0/1286720316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f2c00 msgr2=0x55cc08755c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f2c00 0x55cc08755c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 -- 172.21.15.26:0/1286720316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072411d3b0 con 0x55cc086f3520 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0722ffd700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0718099ab0 tx=0x7f07180994d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601d4b84461e0afd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0702ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.845+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718099b20 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f071809dc70 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0714015070 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07180a1eb0 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f073da0cb80 10 client.27281 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55cc08470250 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.846+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0714015070 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0718099d80 con 0x55cc086f3520 2023-10-06T21:23:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 10 client.27281 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 10 client.27281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 20 client.27281 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc087627f0 0x55cc08764c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 4 client.27281 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc0863fad0 con 0x55cc087627f0 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f07237fe700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc087627f0 0x55cc08764c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f07237fe700 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc087627f0 0x55cc08764c10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0921c96eee09d05 server_cookie=c3f9de1f82e7ad1e in_seq=0 out_seq=0 2023-10-06T21:23:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.847+0000 7f0702ffd700 10 client.27281 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.861+0000 7f0702ffd700 1 -- 172.21.15.26:0/1286720316 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc0863fad0 con 0x55cc087627f0 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.861+0000 7f0702ffd700 10 client.27281 handle_command_reply: tid=0 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.861+0000 7f073da0cb80 1 client.27281 shutdown 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 msgr2=0x55cc08756140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0718099ab0 tx=0x7f07180994d0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 shutdown_connections 2023-10-06T21:23:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc086fe3d0 0x55cc08756680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cc086f2c00 0x55cc08755c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55cc087627f0 0x55cc08764c10 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f0702ffd700 0 client.27281 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 --2- 172.21.15.26:0/1286720316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc086f3520 0x55cc08756140 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 >> 172.21.15.26:0/1286720316 conn(0x55cc08668200 msgr2=0x55cc086f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 shutdown_connections 2023-10-06T21:23:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.862+0000 7f073da0cb80 1 -- 172.21.15.26:0/1286720316 wait complete. 2023-10-06T21:23:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.863+0000 7f073da0cb80 20 client.27281 trim_cache size 0 max 16384 2023-10-06T21:23:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:23:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:23:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:23:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.863+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 msgr2=0x7f070c0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.863+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 0x7f070c0a3bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0714005950 tx=0x7f071400b410 comp rx=0 tx=0).stop 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.863+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 msgr2=0x7f0724118f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.863+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f071800b550 tx=0x7f071800b860 comp rx=0 tx=0).stop 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 shutdown_connections 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f070c0a1730 0x7f070c0a3bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f072411a690 0x7f07241117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f072410f620 0x7f07241189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 --2- 172.21.15.26:0/1545277999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f072410fa90 0x7f0724118f40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 >> 172.21.15.26:0/1545277999 conn(0x7f072406cfa0 msgr2=0x7f072406f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 shutdown_connections 2023-10-06T21:23:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:04.864+0000 7f0729fff700 1 -- 172.21.15.26:0/1545277999 wait complete. 2023-10-06T21:23:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:06 smithi167 ceph-mon[134494]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:23:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:06 smithi167 ceph-mon[134494]: pgmap v400: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-10-06T21:23:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:06 smithi026 ceph-mon[98490]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:23:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:06 smithi026 ceph-mon[98490]: pgmap v400: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-10-06T21:23:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:06 smithi078 ceph-mon[134896]: osdmap e102: 12 total, 12 up, 12 in 2023-10-06T21:23:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:06 smithi078 ceph-mon[134896]: pgmap v400: 97 pgs: 97 active+clean; 5.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-10-06T21:23:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 745.0 s, 887.0 tps, lat 112.440 ms stddev 144.546 2023-10-06T21:23:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:08 smithi167 ceph-mon[134494]: pgmap v401: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:08 smithi026 ceph-mon[98490]: pgmap v401: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:08 smithi078 ceph-mon[134896]: pgmap v401: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:10 smithi167 ceph-mon[134494]: pgmap v402: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:10 smithi026 ceph-mon[98490]: pgmap v402: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:10 smithi078 ceph-mon[134896]: pgmap v402: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.2 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:23:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 750.0 s, 1333.2 tps, lat 74.866 ms stddev 78.590 2023-10-06T21:23:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:12 smithi167 ceph-mon[134494]: pgmap v403: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-10-06T21:23:13.078 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:12 smithi026 ceph-mon[98490]: pgmap v403: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-10-06T21:23:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:12 smithi078 ceph-mon[134896]: pgmap v403: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-10-06T21:23:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:14 smithi078 ceph-mon[134896]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:23:14.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:14 smithi078 ceph-mon[134896]: pgmap v405: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:23:14.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:14 smithi167 ceph-mon[134494]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:23:14.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:14 smithi167 ceph-mon[134494]: pgmap v405: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:23:14.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:14 smithi026 ceph-mon[98490]: osdmap e103: 12 total, 12 up, 12 in 2023-10-06T21:23:14.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:14 smithi026 ceph-mon[98490]: pgmap v405: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:23:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:16 smithi078 ceph-mon[134896]: pgmap v406: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-10-06T21:23:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:16 smithi167 ceph-mon[134494]: pgmap v406: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-10-06T21:23:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:16 smithi026 ceph-mon[98490]: pgmap v406: 97 pgs: 97 active+clean; 5.1 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-10-06T21:23:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 755.0 s, 1408.2 tps, lat 70.913 ms stddev 82.627 2023-10-06T21:23:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.114+0000 7f1a517ff700 1 -- 172.21.15.26:0/3009871634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c105150 msgr2=0x7f1a4c10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.114+0000 7f1a517ff700 1 --2- 172.21.15.26:0/3009871634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c105150 0x7f1a4c10cc90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400b0a0 tx=0x7f1a4400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 -- 172.21.15.26:0/3009871634 shutdown_connections 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 --2- 172.21.15.26:0/3009871634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c105150 0x7f1a4c10cc90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 --2- 172.21.15.26:0/3009871634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c100a60 0x7f1a4c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 --2- 172.21.15.26:0/3009871634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c101410 0x7f1a4c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 -- 172.21.15.26:0/3009871634 >> 172.21.15.26:0/3009871634 conn(0x7f1a4c0762e0 msgr2=0x7f1a4c0766f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 -- 172.21.15.26:0/3009871634 shutdown_connections 2023-10-06T21:23:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.115+0000 7f1a517ff700 1 -- 172.21.15.26:0/3009871634 wait complete. 2023-10-06T21:23:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.117+0000 7f1a517ff700 1 Processor -- start 2023-10-06T21:23:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.117+0000 7f1a517ff700 1 -- start start 2023-10-06T21:23:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c100a60 0x7f1a4c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c1958d0 0x7f1a4c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1aa000 con 0x7f1a4c100a60 2023-10-06T21:23:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1aa140 con 0x7f1a4c1958d0 2023-10-06T21:23:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a517ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1aa300 con 0x7f1a4c105150 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a4a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a4a7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:34198/0 (socket says 172.21.15.26:34198) 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/468597915 learned_addr learned my addr 172.21.15.26:0/468597915 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c1958d0 0x7f1a4c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.118+0000 7f1a4affd700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c100a60 0x7f1a4c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/468597915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c1958d0 msgr2=0x7f1a4c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c1958d0 0x7f1a4c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c100a60 msgr2=0x7f1a4c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c100a60 0x7f1a4c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/468597915 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44009d00 con 0x7f1a4c105150 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e970 tx=0x7f1a4000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a33fff700 1 -- 172.21.15.26:0/468597915 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009790 con 0x7f1a4c105150 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a33fff700 1 -- 172.21.15.26:0/468597915 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4000c7e0 con 0x7f1a4c105150 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.119+0000 7f1a33fff700 1 -- 172.21.15.26:0/468597915 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400054e0 con 0x7f1a4c105150 2023-10-06T21:23:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.120+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c1aa580 con 0x7f1a4c105150 2023-10-06T21:23:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.120+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c1aaa40 con 0x7f1a4c105150 2023-10-06T21:23:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.126+0000 7f1a65213b80 1 Processor -- start 2023-10-06T21:23:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 -- start start 2023-10-06T21:23:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfbbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfbe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edf1a400 con 0x5606edfbc3d0 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edf0bbc0 con 0x5606edfb0c00 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.127+0000 7f1a65213b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edf094e0 con 0x5606edfb1520 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60764/0 (socket says 172.21.15.26:60764) 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/2332392754 learned_addr learned my addr 172.21.15.26:0/2332392754 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4affd700 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfbbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/2332392754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 msgr2=0x5606edfbbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfbbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/2332392754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 msgr2=0x5606edfbe7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfbe7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/2332392754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606edd2fd80 con 0x5606edfb0c00 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.128+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a44015000 tx=0x7f1a44009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23745c9acb402ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.129+0000 7f1a317fa700 1 -- 172.21.15.26:0/2332392754 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400e040 con 0x5606edfb0c00 2023-10-06T21:23:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.129+0000 7f1a317fa700 1 -- 172.21.15.26:0/2332392754 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4400bcb0 con 0x5606edfb0c00 2023-10-06T21:23:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.129+0000 7f1a317fa700 1 -- 172.21.15.26:0/2332392754 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4401c6a0 con 0x5606edfb0c00 2023-10-06T21:23:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.130+0000 7f1a33fff700 1 -- 172.21.15.26:0/468597915 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f1a4000c950 con 0x7f1a4c105150 2023-10-06T21:23:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 -- 172.21.15.26:0/2332392754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 msgr2=0x5606edfb0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a44015000 tx=0x7f1a44009c20 comp rx=0 tx=0).stop 2023-10-06T21:23:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 -- 172.21.15.26:0/2332392754 shutdown_connections 2023-10-06T21:23:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfbe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb0fe0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 --2- 172.21.15.26:0/2332392754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfbbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.131+0000 7f1a65213b80 1 -- 172.21.15.26:0/2332392754 >> 172.21.15.26:0/2332392754 conn(0x5606edf26200 msgr2=0x5606edfad080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.132+0000 7f1a65213b80 1 -- 172.21.15.26:0/2332392754 shutdown_connections 2023-10-06T21:23:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.132+0000 7f1a65213b80 1 -- 172.21.15.26:0/2332392754 wait complete. 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.133+0000 7f1a33fff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 0x7f1a340a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.133+0000 7f1a65213b80 1 -- start start 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.133+0000 7f1a33fff700 1 -- 172.21.15.26:0/468597915 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1a40014070 con 0x7f1a4c105150 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.133+0000 7f1a65213b80 1 Processor -- start 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.133+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 0x7f1a340a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb0c00 0x5606edfbbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edf04dd0 con 0x5606edfbc3d0 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edea16b0 con 0x5606edfb1520 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edea1350 con 0x5606edfb0c00 2023-10-06T21:23:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a65213b80 10 client.0 _ll_register_callbacks cb 0x5606edcfe7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb0c00 0x5606edfbbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4a7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60766/0 (socket says 172.21.15.26:60766) 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/979011115 learned_addr learned my addr 172.21.15.26:0/979011115 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 0x7f1a340a3ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a440076a0 tx=0x7f1a44009c90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.134+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/979011115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb0c00 msgr2=0x5606edfbbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb0c00 0x5606edfbbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/979011115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 msgr2=0x5606edfb9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 -- 172.21.15.26:0/979011115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c10d4d0 con 0x5606edfb1520 2023-10-06T21:23:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000efd0 tx=0x7f1a4000cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293e06ea29786b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.135+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4009e170 con 0x5606edfb1520 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1a4009e390 con 0x5606edfb1520 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606edfa4d50 con 0x5606edfb1520 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a400a25d0 con 0x5606edfb1520 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a65213b80 10 client.27293 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5606edd18c00 con 0x5606edfb1520 2023-10-06T21:23:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606edfa4d50 con 0x5606edfb1520 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.136+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1a40099730 con 0x5606edfb1520 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 10 client.27293 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 10 client.27293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 20 client.27293 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606edfcbf80 0x5606edfce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 4 client.27293 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.137+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606edefddb0 con 0x5606edfcbf80 2023-10-06T21:23:18.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.138+0000 7f1a4affd700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606edfcbf80 0x5606edfce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.140+0000 7f1a4affd700 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606edfcbf80 0x5606edfce3a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471ee14edc399bbd server_cookie=d039f165d74a5f90 in_seq=0 out_seq=0 2023-10-06T21:23:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.140+0000 7f1a2eff5700 10 client.27293 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.141+0000 7f1a2eff5700 1 -- 172.21.15.26:0/979011115 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5606edefddb0 con 0x5606edfcbf80 2023-10-06T21:23:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.141+0000 7f1a2eff5700 10 client.27293 handle_command_reply: tid=0 2023-10-06T21:23:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.141+0000 7f1a65213b80 1 client.27293 shutdown 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.141+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 msgr2=0x5606edfb9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.141+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000efd0 tx=0x7f1a4000cd40 comp rx=0 tx=0).stop 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 shutdown_connections 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb0c00 0x5606edfbbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606edfcbf80 0x5606edfce3a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 --2- 172.21.15.26:0/979011115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb1520 0x5606edfb9090 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 >> 172.21.15.26:0/979011115 conn(0x5606edf26200 msgr2=0x5606edfacc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a2eff5700 0 client.27293 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 shutdown_connections 2023-10-06T21:23:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.142+0000 7f1a65213b80 1 -- 172.21.15.26:0/979011115 wait complete. 2023-10-06T21:23:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.143+0000 7f1a65213b80 20 client.27293 trim_cache size 0 max 16384 2023-10-06T21:23:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.167+0000 7f1a65213b80 1 Processor -- start 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.167+0000 7f1a65213b80 1 -- start start 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfb43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edd858c0 con 0x5606edfbc3d0 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edefddb0 con 0x5606edfb0c00 2023-10-06T21:23:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a65213b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edfc9500 con 0x5606edfb1520 2023-10-06T21:23:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60774/0 (socket says 172.21.15.26:60774) 2023-10-06T21:23:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.168+0000 7f1a4affd700 1 -- 172.21.15.26:0/914915023 learned_addr learned my addr 172.21.15.26:0/914915023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 -- 172.21.15.26:0/914915023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 msgr2=0x5606edfb43d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfb43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 -- 172.21.15.26:0/914915023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 msgr2=0x5606edfb4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 -- 172.21.15.26:0/914915023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606edd561f0 con 0x5606edfb0c00 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.169+0000 7f1a4affd700 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c90 tx=0x7f1a3c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4174242dda170763 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.170+0000 7f1a31ffb700 1 -- 172.21.15.26:0/914915023 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c016070 con 0x5606edfb0c00 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.170+0000 7f1a31ffb700 1 -- 172.21.15.26:0/914915023 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a3c017040 con 0x5606edfb0c00 2023-10-06T21:23:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.170+0000 7f1a31ffb700 1 -- 172.21.15.26:0/914915023 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c005020 con 0x5606edfb0c00 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 -- 172.21.15.26:0/914915023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 msgr2=0x5606edfb3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c90 tx=0x7f1a3c009260 comp rx=0 tx=0).stop 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 -- 172.21.15.26:0/914915023 shutdown_connections 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfbc3d0 0x5606edfb4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfb3aa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 --2- 172.21.15.26:0/914915023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfb1520 0x5606edfb43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 -- 172.21.15.26:0/914915023 >> 172.21.15.26:0/914915023 conn(0x5606edf26200 msgr2=0x5606edfc7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.171+0000 7f1a65213b80 1 -- 172.21.15.26:0/914915023 shutdown_connections 2023-10-06T21:23:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.172+0000 7f1a65213b80 1 -- 172.21.15.26:0/914915023 wait complete. 2023-10-06T21:23:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.172+0000 7f1a65213b80 1 -- start start 2023-10-06T21:23:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.172+0000 7f1a65213b80 1 Processor -- start 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfc2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfb1520 0x5606edfc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 0x5606edfc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edfc1a70 con 0x5606edfb1520 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edfc1be0 con 0x5606edfb0c00 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606edfc1da0 con 0x5606edfbc3d0 2023-10-06T21:23:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.173+0000 7f1a65213b80 10 client.0 _ll_register_callbacks cb 0x7f1a40099c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfc2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 0x5606edfc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfc2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60786/0 (socket says 172.21.15.26:60786) 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4affd700 1 -- 172.21.15.26:0/4202549541 learned_addr learned my addr 172.21.15.26:0/4202549541 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4a7fc700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfb1520 0x5606edfc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 msgr2=0x5606edfc2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfc2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/4202549541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfb1520 msgr2=0x5606edfc0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfb1520 0x5606edfc0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.174+0000 7f1a4b7fe700 1 -- 172.21.15.26:0/4202549541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c009eb0 con 0x5606edfbc3d0 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.175+0000 7f1a2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.175+0000 7f1a4b7fe700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 0x5606edfc1530 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1a440062a0 tx=0x7f1a4400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cf72c989b7b89a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.175+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400e040 con 0x5606edfbc3d0 2023-10-06T21:23:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.176+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1a4401d650 con 0x5606edfbc3d0 2023-10-06T21:23:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.177+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a4400bcb0 con 0x5606edfbc3d0 2023-10-06T21:23:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.178+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4401d810 con 0x5606edfbc3d0 2023-10-06T21:23:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.178+0000 7f1a65213b80 10 client.27148 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.178+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5606edd2e250 con 0x5606edfbc3d0 2023-10-06T21:23:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4403e030 con 0x5606edfbc3d0 2023-10-06T21:23:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1a440055d0 con 0x5606edfbc3d0 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 10 client.27148 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 10 client.27148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 20 client.27148 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606ee01ad50 0x5606ee01b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 4 client.27148 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5606edefdad0 con 0x5606ee01ad50 2023-10-06T21:23:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.179+0000 7f1a4affd700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606ee01ad50 0x5606ee01b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.180+0000 7f1a2eff5700 10 client.27148 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.180+0000 7f1a4affd700 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606ee01ad50 0x5606ee01b160 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fea1516a2acf63 server_cookie=24c8ca5d61aa30 in_seq=0 out_seq=0 2023-10-06T21:23:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a2eff5700 1 -- 172.21.15.26:0/4202549541 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5606edefdad0 con 0x5606ee01ad50 2023-10-06T21:23:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a2eff5700 10 client.27148 handle_command_reply: tid=0 2023-10-06T21:23:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 client.27148 shutdown 2023-10-06T21:23:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 msgr2=0x5606edfc1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 0x5606edfc1530 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1a440062a0 tx=0x7f1a4400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 shutdown_connections 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606edfbc3d0 0x5606edfc1530 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5606ee01ad50 0x5606ee01b160 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606edfb0c00 0x5606edfc2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 --2- 172.21.15.26:0/4202549541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606edfb1520 0x5606edfc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a2eff5700 0 client.27148 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 >> 172.21.15.26:0/4202549541 conn(0x5606edf26200 msgr2=0x5606edfb43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 shutdown_connections 2023-10-06T21:23:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.248+0000 7f1a65213b80 1 -- 172.21.15.26:0/4202549541 wait complete. 2023-10-06T21:23:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:23:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:23:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:23:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:23:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.250+0000 7f1a65213b80 20 client.27148 trim_cache size 0 max 16384 2023-10-06T21:23:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 msgr2=0x7f1a340a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 0x7f1a340a3ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a440076a0 tx=0x7f1a44009c90 comp rx=0 tx=0).stop 2023-10-06T21:23:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 msgr2=0x7f1a4c195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e970 tx=0x7f1a4000ec80 comp rx=0 tx=0).stop 2023-10-06T21:23:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 shutdown_connections 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a4c1958d0 0x7f1a4c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f1a340a1600 0x7f1a340a3ac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a4c100a60 0x7f1a4c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 --2- 172.21.15.26:0/468597915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4c105150 0x7f1a4c195390 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.251+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 >> 172.21.15.26:0/468597915 conn(0x7f1a4c0762e0 msgr2=0x7f1a4c102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.252+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 shutdown_connections 2023-10-06T21:23:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:18.252+0000 7f1a517ff700 1 -- 172.21.15.26:0/468597915 wait complete. 2023-10-06T21:23:19.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:18 smithi167 ceph-mon[134494]: pgmap v407: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:18 smithi026 ceph-mon[98490]: pgmap v407: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:18 smithi078 ceph-mon[134896]: pgmap v407: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:20 smithi167 ceph-mon[134494]: pgmap v408: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:21.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:20 smithi026 ceph-mon[98490]: pgmap v408: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:21.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:20 smithi078 ceph-mon[134896]: pgmap v408: 97 pgs: 97 active+clean; 5.1 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 604 op/s 2023-10-06T21:23:21.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:23:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 -- 172.21.15.26:0/281001324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64000ff9a0 msgr2=0x7f6400103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 --2- 172.21.15.26:0/281001324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64000ff9a0 0x7f6400103ce0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009990 tx=0x7f63e8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 -- 172.21.15.26:0/281001324 shutdown_connections 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 --2- 172.21.15.26:0/281001324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6400104220 0x7f64001046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 --2- 172.21.15.26:0/281001324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 0x7f64000ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 --2- 172.21.15.26:0/281001324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64000ff9a0 0x7f6400103ce0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 -- 172.21.15.26:0/281001324 >> 172.21.15.26:0/281001324 conn(0x7f6400076ed0 msgr2=0x7f64000fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 -- 172.21.15.26:0/281001324 shutdown_connections 2023-10-06T21:23:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.508+0000 7f64065ff700 1 -- 172.21.15.26:0/281001324 wait complete. 2023-10-06T21:23:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.510+0000 7f64065ff700 1 Processor -- start 2023-10-06T21:23:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.510+0000 7f64065ff700 1 -- start start 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 0x7f64001a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64000ff9a0 0x7f64001a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640010a360 con 0x7f64000ff080 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640010a4d0 con 0x7f6400104220 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f64065ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640010a690 con 0x7f64000ff9a0 2023-10-06T21:23:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64000ff9a0 0x7f64001a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59800/0 (socket says 172.21.15.26:59800) 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f63fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 0x7f64001a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 -- 172.21.15.26:0/863712996 learned_addr learned my addr 172.21.15.26:0/863712996 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 -- 172.21.15.26:0/863712996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64000ff9a0 msgr2=0x7f64001a2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64000ff9a0 0x7f64001a2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 -- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 msgr2=0x7f64001a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 0x7f64001a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.511+0000 7f6404b9c700 1 -- 172.21.15.26:0/863712996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0009520 con 0x7f6400104220 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f6404b9c700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c370 tx=0x7f63f800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f63fdffb700 1 -- 172.21.15.26:0/863712996 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017070 con 0x7f6400104220 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f63fdffb700 1 -- 172.21.15.26:0/863712996 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f800f040 con 0x7f6400104220 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f8009c40 con 0x7f6400104220 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f63fdffb700 1 -- 172.21.15.26:0/863712996 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f800e050 con 0x7f6400104220 2023-10-06T21:23:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.512+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e80095f0 con 0x7f6400104220 2023-10-06T21:23:21.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.517+0000 7f63fdffb700 1 -- 172.21.15.26:0/863712996 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63f800cb60 con 0x7f6400104220 2023-10-06T21:23:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.519+0000 7f63fdffb700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 0x7f63e00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.520+0000 7f6419e94b80 1 Processor -- start 2023-10-06T21:23:21.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.520+0000 7f63fdffb700 1 -- 172.21.15.26:0/863712996 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63f80cc9e0 con 0x7f6400104220 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.521+0000 7f6419e94b80 1 -- start start 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4cad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 0x5577f4cafc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4c0b460 con 0x5577f4ca20c0 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4bfcc20 con 0x5577f4cad890 2023-10-06T21:23:21.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.522+0000 7f6419e94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4bfa540 con 0x5577f4ca29e0 2023-10-06T21:23:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.523+0000 7f63fffff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 0x7f63e00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.523+0000 7f63fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.523+0000 7f63fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36880/0 (socket says 172.21.15.26:36880) 2023-10-06T21:23:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.523+0000 7f63fffff700 1 -- 172.21.15.26:0/3944328813 learned_addr learned my addr 172.21.15.26:0/3944328813 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:21.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.523+0000 7f6404b9c700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 0x5577f4cafc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4cad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 -- 172.21.15.26:0/3944328813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 msgr2=0x5577f4cafc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 0x5577f4cafc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 -- 172.21.15.26:0/3944328813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 msgr2=0x5577f4ca24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63f7fff700 1 -- 172.21.15.26:0/3944328813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577f4a20de0 con 0x5577f4ca29e0 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f6404b9c700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 0x5577f4cafc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.524+0000 7f63fffff700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.525+0000 7f63fffff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 0x7f63e00a3b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63f0011bf0 tx=0x7f63f0009e30 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.525+0000 7f63f7fff700 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4cad350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b2e0 tx=0x7f63e800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e2c23ad4f1c10f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.525+0000 7f63f67fc700 1 -- 172.21.15.26:0/3944328813 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e801d070 con 0x5577f4ca29e0 2023-10-06T21:23:21.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.525+0000 7f63f67fc700 1 -- 172.21.15.26:0/3944328813 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e801e040 con 0x5577f4ca29e0 2023-10-06T21:23:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.525+0000 7f63f67fc700 1 -- 172.21.15.26:0/3944328813 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80047f0 con 0x5577f4ca29e0 2023-10-06T21:23:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 -- 172.21.15.26:0/3944328813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 msgr2=0x5577f4cad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4cad350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b2e0 tx=0x7f63e800b610 comp rx=0 tx=0).stop 2023-10-06T21:23:21.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 -- 172.21.15.26:0/3944328813 shutdown_connections 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4cad890 0x5577f4cafc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca20c0 0x5577f4ca24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3944328813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4cad350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.526+0000 7f6419e94b80 1 -- 172.21.15.26:0/3944328813 >> 172.21.15.26:0/3944328813 conn(0x5577f4c17260 msgr2=0x5577f4c9e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.527+0000 7f6419e94b80 1 -- 172.21.15.26:0/3944328813 shutdown_connections 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.527+0000 7f6419e94b80 1 -- 172.21.15.26:0/3944328813 wait complete. 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.527+0000 7f6419e94b80 1 -- start start 2023-10-06T21:23:21.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.528+0000 7f6419e94b80 1 Processor -- start 2023-10-06T21:23:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.528+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4cad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.528+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4caa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.528+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6419e94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4bf5e30 con 0x5577f4cad890 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6419e94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4b92710 con 0x5577f4ca20c0 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6419e94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4b923b0 con 0x5577f4ca29e0 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6419e94b80 10 client.0 _ll_register_callbacks cb 0x5577f49ef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36894/0 (socket says 172.21.15.26:36894) 2023-10-06T21:23:21.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.529+0000 7f6404b9c700 1 -- 172.21.15.26:0/1074456538 learned_addr learned my addr 172.21.15.26:0/1074456538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.530+0000 7f6404b9c700 1 -- 172.21.15.26:0/1074456538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 msgr2=0x5577f4caa530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:23:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.530+0000 7f6404b9c700 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4caa530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.530+0000 7f6404b9c700 1 -- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 msgr2=0x5577f4cad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.530+0000 7f6404b9c700 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4cad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.530+0000 7f6404b9c700 1 -- 172.21.15.26:0/1074456538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f0019040 con 0x5577f4cad890 2023-10-06T21:23:21.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.531+0000 7f6404b9c700 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f63f80ca1d0 tx=0x7f63f80993c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135a66995aa28a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.531+0000 7f63ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:23:21.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.537+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80999b0 con 0x5577f4cad890 2023-10-06T21:23:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.538+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63f809db00 con 0x5577f4cad890 2023-10-06T21:23:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.538+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577f4c95f10 con 0x5577f4cad890 2023-10-06T21:23:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.538+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f80b2240 con 0x5577f4cad890 2023-10-06T21:23:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.539+0000 7f6419e94b80 10 client.16959 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:21.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.539+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f63e80095f0 con 0x5577f4cad890 2023-10-06T21:23:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.539+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577f4c95f10 con 0x5577f4cad890 2023-10-06T21:23:21.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.539+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63f80a18b0 con 0x5577f4cad890 2023-10-06T21:23:21.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:21.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:21.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 20 client.16959 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:21.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4cce2a0 0x5577f4cd06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 4 client.16959 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:21.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.540+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577f4beee10 con 0x5577f4cce2a0 2023-10-06T21:23:21.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.551+0000 7f63fffff700 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4cce2a0 0x5577f4cd06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.557+0000 7f63fffff700 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4cce2a0 0x5577f4cd06e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93066f04caea4c3 server_cookie=570cd14738cd750f in_seq=0 out_seq=0 2023-10-06T21:23:21.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.558+0000 7f63ef7fe700 10 client.16959 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.570+0000 7f63ef7fe700 1 -- 172.21.15.26:0/1074456538 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577f4beee10 con 0x5577f4cce2a0 2023-10-06T21:23:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.570+0000 7f63ef7fe700 10 client.16959 handle_command_reply: tid=0 2023-10-06T21:23:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.570+0000 7f6419e94b80 1 client.16959 shutdown 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 msgr2=0x5577f4caac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f63f80ca1d0 tx=0x7f63f80993c0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 shutdown_connections 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4caac70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4cce2a0 0x5577f4cd06e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4cad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 --2- 172.21.15.26:0/1074456538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca29e0 0x5577f4caa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 >> 172.21.15.26:0/1074456538 conn(0x5577f4c17260 msgr2=0x5577f4c9dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f63ef7fe700 0 client.16959 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 shutdown_connections 2023-10-06T21:23:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 1 -- 172.21.15.26:0/1074456538 wait complete. 2023-10-06T21:23:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.571+0000 7f6419e94b80 20 client.16959 trim_cache size 0 max 16384 2023-10-06T21:23:21.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.600+0000 7f6419e94b80 1 Processor -- start 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.600+0000 7f6419e94b80 1 -- start start 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca29e0 0x5577f4ca4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4cad890 0x5577f4cace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4a76920 con 0x5577f4ca29e0 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4ccac60 con 0x5577f4ca20c0 2023-10-06T21:23:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6419e94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4c9fd40 con 0x5577f4cad890 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca29e0 0x5577f4ca4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59826/0 (socket says 172.21.15.26:59826) 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 -- 172.21.15.26:0/3383258 learned_addr learned my addr 172.21.15.26:0/3383258 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f6404b9c700 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4cad890 0x5577f4cace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 -- 172.21.15.26:0/3383258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4cad890 msgr2=0x5577f4cace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4cad890 0x5577f4cace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.601+0000 7f63fffff700 1 -- 172.21.15.26:0/3383258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca29e0 msgr2=0x5577f4ca4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63fffff700 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca29e0 0x5577f4ca4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63fffff700 1 -- 172.21.15.26:0/3383258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640010d510 con 0x5577f4ca20c0 2023-10-06T21:23:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63fffff700 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5577f4cd1000 tx=0x7f63f0021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dd3747e33f41c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63f6ffd700 1 -- 172.21.15.26:0/3383258 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0016040 con 0x5577f4ca20c0 2023-10-06T21:23:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63f6ffd700 1 -- 172.21.15.26:0/3383258 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f0011320 con 0x5577f4ca20c0 2023-10-06T21:23:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.602+0000 7f63f6ffd700 1 -- 172.21.15.26:0/3383258 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000fba0 con 0x5577f4ca20c0 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 -- 172.21.15.26:0/3383258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 msgr2=0x5577f4caa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5577f4cd1000 tx=0x7f63f0021000 comp rx=0 tx=0).stop 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 -- 172.21.15.26:0/3383258 shutdown_connections 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4cad890 0x5577f4cace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca20c0 0x5577f4caa520 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 --2- 172.21.15.26:0/3383258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4ca29e0 0x5577f4ca4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.603+0000 7f6419e94b80 1 -- 172.21.15.26:0/3383258 >> 172.21.15.26:0/3383258 conn(0x5577f4c17260 msgr2=0x5577f4ca9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.604+0000 7f6419e94b80 1 -- 172.21.15.26:0/3383258 shutdown_connections 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.605+0000 7f6419e94b80 1 -- 172.21.15.26:0/3383258 wait complete. 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.605+0000 7f6419e94b80 1 -- start start 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.606+0000 7f6419e94b80 1 Processor -- start 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.606+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 0x5577f4cc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.606+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca29e0 0x5577f4cc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.607+0000 7f6419e94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4cc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.607+0000 7f6419e94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4cc45d0 con 0x5577f4cad890 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.607+0000 7f6419e94b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4d22140 con 0x5577f4ca29e0 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.607+0000 7f6419e94b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f4d22280 con 0x5577f4ca20c0 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.607+0000 7f6419e94b80 10 client.0 _ll_register_callbacks cb 0x7f63f8099850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.608+0000 7f63f7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca29e0 0x5577f4cc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.608+0000 7f63fffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 0x5577f4cc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.608+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4cc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.608+0000 7f6404b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4cc3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36908/0 (socket says 172.21.15.26:36908) 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.608+0000 7f6404b9c700 1 -- 172.21.15.26:0/2662768668 learned_addr learned my addr 172.21.15.26:0/2662768668 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 -- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca29e0 msgr2=0x5577f4cc37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca29e0 0x5577f4cc37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 -- 172.21.15.26:0/2662768668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 msgr2=0x5577f4cc3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4cc3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 -- 172.21.15.26:0/2662768668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f00115b0 con 0x5577f4ca20c0 2023-10-06T21:23:21.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63fffff700 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 0x5577f4cc54f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009d70 tx=0x7f63f0011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3534c749e33d3344 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.609+0000 7f63ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0016050 con 0x5577f4ca20c0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63f0004020 con 0x5577f4ca20c0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e801e040 con 0x5577f4ca20c0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f00041e0 con 0x5577f4ca20c0 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f6419e94b80 10 client.27163 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:21.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.610+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5577f4a09c60 con 0x5577f4ca20c0 2023-10-06T21:23:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e801e040 con 0x5577f4ca20c0 2023-10-06T21:23:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63f0030020 con 0x5577f4ca20c0 2023-10-06T21:23:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 10 client.27163 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 10 client.27163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:21.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 20 client.27163 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4d23860 0x5577f4d25c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 4 client.27163 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577f4beeb30 con 0x5577f4d23860 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.611+0000 7f63f7fff700 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4d23860 0x5577f4d25c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.613+0000 7f63ef7fe700 10 client.27163 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:21.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.613+0000 7f63f7fff700 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4d23860 0x5577f4d25c80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60675c16c201cd22 server_cookie=d3cbd01e846d2e0e in_seq=0 out_seq=0 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f63ef7fe700 1 -- 172.21.15.26:0/2662768668 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5577f4beeb30 con 0x5577f4d23860 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f63ef7fe700 10 client.27163 handle_command_reply: tid=0 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 client.27163 shutdown 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 msgr2=0x5577f4cc54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 0x5577f4cc54f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009d70 tx=0x7f63f0011a10 comp rx=0 tx=0).stop 2023-10-06T21:23:21.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 shutdown_connections 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577f4cad890 0x5577f4cc3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577f4ca20c0 0x5577f4cc54f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5577f4d23860 0x5577f4d25c80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 --2- 172.21.15.26:0/2662768668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5577f4ca29e0 0x5577f4cc37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 >> 172.21.15.26:0/2662768668 conn(0x5577f4c17260 msgr2=0x5577f4c9def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f63ef7fe700 0 client.27163 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:21.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 shutdown_connections 2023-10-06T21:23:21.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 1 -- 172.21.15.26:0/2662768668 wait complete. 2023-10-06T21:23:21.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.614+0000 7f6419e94b80 20 client.27163 trim_cache size 0 max 16384 2023-10-06T21:23:21.638 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:23:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:23:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:23:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:23:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:23:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:23:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:23:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:23:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:23:21.640 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:23:21.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 msgr2=0x7f63e00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 0x7f63e00a3b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63f0011bf0 tx=0x7f63f0009e30 comp rx=0 tx=0).stop 2023-10-06T21:23:21.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 msgr2=0x7f6400109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c370 tx=0x7f63f800c680 comp rx=0 tx=0).stop 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 shutdown_connections 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400104220 0x7f6400109c60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63e00a16b0 0x7f63e00a3b70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64000ff080 0x7f64001a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 --2- 172.21.15.26:0/863712996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64000ff9a0 0x7f64001a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 >> 172.21.15.26:0/863712996 conn(0x7f6400076ed0 msgr2=0x7f640010bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.615+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 shutdown_connections 2023-10-06T21:23:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:21.616+0000 7f64065ff700 1 -- 172.21.15.26:0/863712996 wait complete. 2023-10-06T21:23:21.645 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:23:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 760.0 s, 1180.7 tps, lat 84.974 ms stddev 90.009 2023-10-06T21:23:23.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:22 smithi167 ceph-mon[134494]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 644 op/s 2023-10-06T21:23:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:22 smithi026 ceph-mon[98490]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 644 op/s 2023-10-06T21:23:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:22 smithi078 ceph-mon[134896]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.9 MiB/s wr, 644 op/s 2023-10-06T21:23:24.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:24 smithi026 ceph-mon[98490]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 659 op/s 2023-10-06T21:23:24.628 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:24 smithi078 ceph-mon[134896]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 659 op/s 2023-10-06T21:23:24.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:24 smithi167 ceph-mon[134494]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 659 op/s 2023-10-06T21:23:26.883 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:26 smithi078 ceph-mon[134896]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2023-10-06T21:23:27.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:26 smithi167 ceph-mon[134494]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2023-10-06T21:23:27.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:26 smithi026 ceph-mon[98490]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2023-10-06T21:23:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 765.0 s, 1710.0 tps, lat 57.302 ms stddev 67.695 2023-10-06T21:23:28.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:28 smithi078 ceph-mon[134896]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 840 op/s 2023-10-06T21:23:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:28 smithi167 ceph-mon[134494]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 840 op/s 2023-10-06T21:23:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:28 smithi026 ceph-mon[98490]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 840 op/s 2023-10-06T21:23:30.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:30 smithi078 ceph-mon[134896]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 590 op/s 2023-10-06T21:23:31.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:30 smithi167 ceph-mon[134494]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 590 op/s 2023-10-06T21:23:31.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:30 smithi026 ceph-mon[98490]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 590 op/s 2023-10-06T21:23:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 -- 172.21.15.26:0/1737241170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd04073280 msgr2=0x7fcd04073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 --2- 172.21.15.26:0/1737241170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd04073280 0x7fcd04073660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009990 tx=0x7fccf4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 -- 172.21.15.26:0/1737241170 shutdown_connections 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 --2- 172.21.15.26:0/1737241170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd0410d6f0 0x7fcd0410fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 --2- 172.21.15.26:0/1737241170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd04073280 0x7fcd04073660 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 --2- 172.21.15.26:0/1737241170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073ba0 0x7fcd0410d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 -- 172.21.15.26:0/1737241170 >> 172.21.15.26:0/1737241170 conn(0x7fcd040fc940 msgr2=0x7fcd040fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 -- 172.21.15.26:0/1737241170 shutdown_connections 2023-10-06T21:23:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.504+0000 7fcd095ff700 1 -- 172.21.15.26:0/1737241170 wait complete. 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 Processor -- start 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 -- start start 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 0x7fcd04100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04073ba0 0x7fcd04101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 0x7fcd041a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04101bb0 con 0x7fcd04073ba0 2023-10-06T21:23:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041a9c90 con 0x7fcd04073280 2023-10-06T21:23:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.506+0000 7fcd095ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041a9dd0 con 0x7fcd0410d6f0 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 0x7fcd041a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 0x7fcd04100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 0x7fcd041a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55538/0 (socket says 172.21.15.26:55538) 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd037fe700 1 -- 172.21.15.26:0/3571224952 learned_addr learned my addr 172.21.15.26:0/3571224952 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fccfbfff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04073ba0 0x7fcd04101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 -- 172.21.15.26:0/3571224952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 msgr2=0x7fcd041a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 0x7fcd041a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 -- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04073ba0 msgr2=0x7fcd04101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04073ba0 0x7fcd04101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.507+0000 7fcd02ffd700 1 -- 172.21.15.26:0/3571224952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf40095f0 con 0x7fcd04073280 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 0x7fcd04100e10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fccf4000c00 tx=0x7fccf4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd00ff9700 1 -- 172.21.15.26:0/3571224952 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf401d070 con 0x7fcd04073280 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041a9fd0 con 0x7fcd04073280 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd00ff9700 1 -- 172.21.15.26:0/3571224952 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf4021410 con 0x7fcd04073280 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd00ff9700 1 -- 172.21.15.26:0/3571224952 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4017650 con 0x7fcd04073280 2023-10-06T21:23:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.508+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041aa450 con 0x7fcd04073280 2023-10-06T21:23:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.513+0000 7fcd00ff9700 1 -- 172.21.15.26:0/3571224952 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fccf4021580 con 0x7fcd04073280 2023-10-06T21:23:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.514+0000 7fcd1ce85b80 1 Processor -- start 2023-10-06T21:23:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.514+0000 7fcd1ce85b80 1 -- start start 2023-10-06T21:23:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bd0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13bd37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13b2f400 con 0x555a13bd13d0 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13b20bc0 con 0x555a13bc6520 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.515+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13b1e4e0 con 0x555a13bc5c00 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.516+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.516+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55544/0 (socket says 172.21.15.26:55544) 2023-10-06T21:23:31.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.516+0000 7fcd037fe700 1 -- 172.21.15.26:0/3178427388 learned_addr learned my addr 172.21.15.26:0/3178427388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.516+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bd0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.516+0000 7fccfbfff700 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13bd37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 -- 172.21.15.26:0/3178427388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 msgr2=0x555a13bd0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bd0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 -- 172.21.15.26:0/3178427388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 msgr2=0x555a13bd37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13bd37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 -- 172.21.15.26:0/3178427388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a13944d80 con 0x555a13bc5c00 2023-10-06T21:23:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd037fe700 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00acc0 tx=0x7fccfc00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4672e0f4a6537309 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fcd00ff9700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 0x7fccf00a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.517+0000 7fccf97fa700 1 -- 172.21.15.26:0/3178427388 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00fe20 con 0x555a13bc5c00 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.518+0000 7fccf97fa700 1 -- 172.21.15.26:0/3178427388 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccfc018900 con 0x555a13bc5c00 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.518+0000 7fccf97fa700 1 -- 172.21.15.26:0/3178427388 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc010930 con 0x555a13bc5c00 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.518+0000 7fcd00ff9700 1 -- 172.21.15.26:0/3571224952 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccf40ce270 con 0x7fcd04073280 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.518+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 0x7fccf00a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.522 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/3178427388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 msgr2=0x555a13bc5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00acc0 tx=0x7fccfc00f3c0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/3178427388 shutdown_connections 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13bd37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13bc5fe0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/3178427388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bd0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/3178427388 >> 172.21.15.26:0/3178427388 conn(0x555a13b3b200 msgr2=0x555a13bc2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd02ffd700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 0x7fccf00a3bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097770 tx=0x7fccf40d1c60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/3178427388 shutdown_connections 2023-10-06T21:23:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.519+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/3178427388 wait complete. 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.520+0000 7fcd1ce85b80 1 -- start start 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.520+0000 7fcd1ce85b80 1 Processor -- start 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc5c00 0x555a13bce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bd13d0 0x555a13bcebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13b19dd0 con 0x555a13bc5c00 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13ab66b0 con 0x555a13bc6520 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13ab6350 con 0x555a13bd13d0 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.521+0000 7fcd1ce85b80 10 client.0 _ll_register_callbacks cb 0x555a139137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48650/0 (socket says 172.21.15.26:48650) 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 -- 172.21.15.26:0/4207988385 learned_addr learned my addr 172.21.15.26:0/4207988385 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fcd037fe700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc5c00 0x555a13bce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fcd037fe700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bd13d0 0x555a13bcebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 -- 172.21.15.26:0/4207988385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bd13d0 msgr2=0x555a13bcebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.526 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bd13d0 0x555a13bcebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 -- 172.21.15.26:0/4207988385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc5c00 msgr2=0x555a13bce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc5c00 0x555a13bce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.522+0000 7fccfbfff700 1 -- 172.21.15.26:0/4207988385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc00a8f0 con 0x555a13bc6520 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.524+0000 7fccfbfff700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fccec007ba0 tx=0x7fccec007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e1b10b28d82f5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.524+0000 7fccea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.524+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00e040 con 0x555a13bc6520 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.524+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccec003a00 con 0x555a13bc6520 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.525+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a13bb9d50 con 0x555a13bc6520 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.525+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccec003bc0 con 0x555a13bc6520 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.525+0000 7fcd1ce85b80 10 client.27326 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.525+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x555a1392dc00 con 0x555a13bc6520 2023-10-06T21:23:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.525+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a13bb9d50 con 0x555a13bc6520 2023-10-06T21:23:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccec004250 con 0x555a13bc6520 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 10 client.27326 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 10 client.27326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 20 client.27326 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13bf2020 0x555a13bf4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 4 client.27326 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a13b12db0 con 0x555a13bf2020 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.526+0000 7fcd037fe700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13bf2020 0x555a13bf4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.527+0000 7fcd037fe700 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13bf2020 0x555a13bf4440 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688590930f98b6f3 server_cookie=b1048f9280cc3a9d in_seq=0 out_seq=0 2023-10-06T21:23:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.527+0000 7fccea7fc700 10 client.27326 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.527+0000 7fccea7fc700 1 -- 172.21.15.26:0/4207988385 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a13b12db0 con 0x555a13bf2020 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.527+0000 7fccea7fc700 10 client.27326 handle_command_reply: tid=0 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 client.27326 shutdown 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 msgr2=0x555a13bce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fccec007ba0 tx=0x7fccec007bd0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 shutdown_connections 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bd13d0 0x555a13bcebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc5c00 0x555a13bce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13bf2020 0x555a13bf4440 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/4207988385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc6520 0x555a13bce680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 >> 172.21.15.26:0/4207988385 conn(0x555a13b3b200 msgr2=0x555a13bc1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fccea7fc700 0 client.27326 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 shutdown_connections 2023-10-06T21:23:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.528+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/4207988385 wait complete. 2023-10-06T21:23:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.529+0000 7fcd1ce85b80 20 client.27326 trim_cache size 0 max 16384 2023-10-06T21:23:31.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 Processor -- start 2023-10-06T21:23:31.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 -- start start 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13be8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bd13d0 0x555a13bcb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1399a8c0 con 0x555a13bc6520 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13bc8ff0 con 0x555a13bd13d0 2023-10-06T21:23:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.555+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13be9490 con 0x555a13bc5c00 2023-10-06T21:23:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bd13d0 0x555a13bcb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58036/0 (socket says 172.21.15.26:58036) 2023-10-06T21:23:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 -- 172.21.15.26:0/91922812 learned_addr learned my addr 172.21.15.26:0/91922812 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fcd037fe700 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13be8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 -- 172.21.15.26:0/91922812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 msgr2=0x555a13be8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13be8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 -- 172.21.15.26:0/91922812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bd13d0 msgr2=0x555a13bcb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.556+0000 7fccfbfff700 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bd13d0 0x555a13bcb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.557+0000 7fccfbfff700 1 -- 172.21.15.26:0/91922812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a1396b1f0 con 0x555a13bc6520 2023-10-06T21:23:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.557+0000 7fccfbfff700 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccec002230 tx=0x7fccec009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacb3c03be78d613 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.557+0000 7fccf9ffb700 1 -- 172.21.15.26:0/91922812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00e070 con 0x555a13bc6520 2023-10-06T21:23:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.557+0000 7fccf9ffb700 1 -- 172.21.15.26:0/91922812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccec00f040 con 0x555a13bc6520 2023-10-06T21:23:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.557+0000 7fccf9ffb700 1 -- 172.21.15.26:0/91922812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec01f4f0 con 0x555a13bc6520 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.558+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/91922812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 msgr2=0x555a13bcd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.558+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccec002230 tx=0x7fccec009740 comp rx=0 tx=0).stop 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/91922812 shutdown_connections 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bd13d0 0x555a13bcb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc5c00 0x555a13be8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/91922812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bc6520 0x555a13bcd900 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/91922812 >> 172.21.15.26:0/91922812 conn(0x555a13b3b200 msgr2=0x555a13bc9470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/91922812 shutdown_connections 2023-10-06T21:23:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/91922812 wait complete. 2023-10-06T21:23:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.559+0000 7fcd1ce85b80 1 -- start start 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.560+0000 7fcd1ce85b80 1 Processor -- start 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.560+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc5c00 0x555a13be6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.560+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.560+0000 7fcd1ce85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13c44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13c45390 con 0x555a13bd13d0 2023-10-06T21:23:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13c454d0 con 0x555a13bc5c00 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fcd1ce85b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13c45690 con 0x555a13bc6520 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fcd1ce85b80 10 client.0 _ll_register_callbacks cb 0x7fccec007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55564/0 (socket says 172.21.15.26:55564) 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 -- 172.21.15.26:0/875732608 learned_addr learned my addr 172.21.15.26:0/875732608 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fcd037fe700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc5c00 0x555a13be6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 -- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc5c00 msgr2=0x555a13be6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc5c00 0x555a13be6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 -- 172.21.15.26:0/875732608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 msgr2=0x555a13c44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13c44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.561+0000 7fccfbfff700 1 -- 172.21.15.26:0/875732608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a1392dc00 con 0x555a13bc6520 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.562+0000 7fccfbfff700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccec013380 tx=0x7fccec005a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=956eef2af6774d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.562+0000 7fccea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.562+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00e040 con 0x555a13bc6520 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.562+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fccec01f6f0 con 0x555a13bc6520 2023-10-06T21:23:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.562+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccfc018900 con 0x555a13bc6520 2023-10-06T21:23:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.563+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccec01f8f0 con 0x555a13bc6520 2023-10-06T21:23:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.563+0000 7fcd1ce85b80 10 client.27181 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.563+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x555a13943250 con 0x555a13bc6520 2023-10-06T21:23:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.563+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccec03e020 con 0x555a13bc6520 2023-10-06T21:23:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.563+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fccec029be0 con 0x555a13bc6520 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 10 client.27181 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 10 client.27181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 20 client.27181 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13c469c0 0x555a13c48de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 4 client.27181 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a13b12ad0 con 0x555a13c469c0 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd037fe700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13c469c0 0x555a13c48de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.564+0000 7fcd037fe700 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13c469c0 0x555a13c48de0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9531ac598bf300 server_cookie=fb132db563f2eaa4 in_seq=0 out_seq=0 2023-10-06T21:23:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.565+0000 7fccea7fc700 10 client.27181 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.581+0000 7fccea7fc700 1 -- 172.21.15.26:0/875732608 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a13b12ad0 con 0x555a13c469c0 2023-10-06T21:23:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.581+0000 7fccea7fc700 10 client.27181 handle_command_reply: tid=0 2023-10-06T21:23:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.582+0000 7fcd1ce85b80 1 client.27181 shutdown 2023-10-06T21:23:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.582+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 msgr2=0x555a13be70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.582+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fccec013380 tx=0x7fccec005a50 comp rx=0 tx=0).stop 2023-10-06T21:23:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 shutdown_connections 2023-10-06T21:23:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a13bd13d0 0x555a13c44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x555a13c469c0 0x555a13c48de0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555a13bc5c00 0x555a13be6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 --2- 172.21.15.26:0/875732608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a13bc6520 0x555a13be70c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fccea7fc700 0 client.27181 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 >> 172.21.15.26:0/875732608 conn(0x555a13b3b200 msgr2=0x555a13bce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 shutdown_connections 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 1 -- 172.21.15.26:0/875732608 wait complete. 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.583+0000 7fcd1ce85b80 20 client.27181 trim_cache size 0 max 16384 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:23:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:23:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 msgr2=0x7fccf00a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 0x7fccf00a3bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccf4097770 tx=0x7fccf40d1c60 comp rx=0 tx=0).stop 2023-10-06T21:23:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 msgr2=0x7fcd04100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 0x7fcd04100e10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fccf4000c00 tx=0x7fccf4005740 comp rx=0 tx=0).stop 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 shutdown_connections 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd0410d6f0 0x7fcd041a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd04073280 0x7fcd04100e10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fccf00a1730 0x7fccf00a3bf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 --2- 172.21.15.26:0/3571224952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04073ba0 0x7fcd04101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.584+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 >> 172.21.15.26:0/3571224952 conn(0x7fcd040fc940 msgr2=0x7fcd04105150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.585+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 shutdown_connections 2023-10-06T21:23:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:31.585+0000 7fcd095ff700 1 -- 172.21.15.26:0/3571224952 wait complete. 2023-10-06T21:23:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 770.0 s, 739.8 tps, lat 135.587 ms stddev 148.937 2023-10-06T21:23:32.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:32 smithi078 ceph-mon[134896]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-10-06T21:23:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:32 smithi167 ceph-mon[134494]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-10-06T21:23:33.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:32 smithi026 ceph-mon[98490]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-10-06T21:23:34.622 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:34 smithi078 ceph-mon[134896]: pgmap v415: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.9 MiB/s wr, 728 op/s 2023-10-06T21:23:34.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:34 smithi167 ceph-mon[134494]: pgmap v415: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.9 MiB/s wr, 728 op/s 2023-10-06T21:23:34.834 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:34 smithi026 ceph-mon[98490]: pgmap v415: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.9 MiB/s wr, 728 op/s 2023-10-06T21:23:37.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:36 smithi167 ceph-mon[134494]: pgmap v416: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 610 op/s 2023-10-06T21:23:37.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:36 smithi026 ceph-mon[98490]: pgmap v416: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 610 op/s 2023-10-06T21:23:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:36 smithi078 ceph-mon[134896]: pgmap v416: 97 pgs: 97 active+clean; 5.3 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 610 op/s 2023-10-06T21:23:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 775.0 s, 810.2 tps, lat 125.438 ms stddev 161.716 2023-10-06T21:23:39.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:38 smithi167 ceph-mon[134494]: pgmap v417: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:23:39.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:38 smithi026 ceph-mon[98490]: pgmap v417: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:23:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:38 smithi078 ceph-mon[134896]: pgmap v417: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-10-06T21:23:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:40 smithi167 ceph-mon[134494]: pgmap v418: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-10-06T21:23:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:40 smithi026 ceph-mon[98490]: pgmap v418: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-10-06T21:23:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:40 smithi078 ceph-mon[134896]: pgmap v418: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.51k op/s 2023-10-06T21:23:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 780.0 s, 971.4 tps, lat 101.881 ms stddev 109.197 2023-10-06T21:23:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:42 smithi026 ceph-mon[98490]: pgmap v419: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-10-06T21:23:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:42 smithi078 ceph-mon[134896]: pgmap v419: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-10-06T21:23:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:42 smithi167 ceph-mon[134494]: pgmap v419: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.68k op/s 2023-10-06T21:23:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:44 smithi167 ceph-mon[134494]: pgmap v420: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-10-06T21:23:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:44 smithi026 ceph-mon[98490]: pgmap v420: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.831+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/807589537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e280682d0 msgr2=0x7f6e28068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.831+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/807589537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e280682d0 0x7f6e28068750 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009990 tx=0x7f6e18009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.832+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/807589537 shutdown_connections 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.832+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/807589537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 0x7f6e2810dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.832+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/807589537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.832+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/807589537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e280682d0 0x7f6e28068750 secure :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009990 tx=0x7f6e18009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.832+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/807589537 >> 172.21.15.26:0/807589537 conn(0x7f6e28076b70 msgr2=0x7f6e28076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.833+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/807589537 shutdown_connections 2023-10-06T21:23:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.833+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/807589537 wait complete. 2023-10-06T21:23:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 Processor -- start 2023-10-06T21:23:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 -- start start 2023-10-06T21:23:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 0x7f6e28199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 0x7f6e2819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819e980 con 0x7f6e28068c90 2023-10-06T21:23:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819eaf0 con 0x7f6e2819df00 2023-10-06T21:23:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.835+0000 7f6e2f9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819ecb0 con 0x7f6e28106f60 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2cf9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 0x7f6e2819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2cf9a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41064/0 (socket says 172.21.15.26:41064) 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2cf9a700 1 -- 172.21.15.26:0/3281776134 learned_addr learned my addr 172.21.15.26:0/3281776134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 0x7f6e28199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 -- 172.21.15.26:0/3281776134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 msgr2=0x7f6e28199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 -- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 msgr2=0x7f6e28199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 0x7f6e28199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.836+0000 7f6e2df9c700 1 -- 172.21.15.26:0/3281776134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e180095f0 con 0x7f6e2819df00 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 0x7f6e2819e380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007f10 tx=0x7f6e2000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e1e7fc700 1 -- 172.21.15.26:0/3281776134 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20004db0 con 0x7f6e2819df00 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e1e7fc700 1 -- 172.21.15.26:0/3281776134 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2000f040 con 0x7f6e2819df00 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e1e7fc700 1 -- 172.21.15.26:0/3281776134 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e200043a0 con 0x7f6e2819df00 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e281a6210 con 0x7f6e2819df00 2023-10-06T21:23:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.837+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281a66a0 con 0x7f6e2819df00 2023-10-06T21:23:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.844+0000 7f6e432c3b80 1 Processor -- start 2023-10-06T21:23:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.845+0000 7f6e1e7fc700 1 -- 172.21.15.26:0/3281776134 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6e20004500 con 0x7f6e2819df00 2023-10-06T21:23:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.845+0000 7f6e432c3b80 1 -- start start 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0caafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0c14400 con 0x5578b0cb63d0 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0c05bc0 con 0x5578b0caac00 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0c034e0 con 0x5578b0cab520 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:41076/0 (socket says 172.21.15.26:41076) 2023-10-06T21:23:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e2d79b700 1 -- 172.21.15.26:0/634816649 learned_addr learned my addr 172.21.15.26:0/634816649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.846+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0caafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.847+0000 7f6e2d79b700 1 -- 172.21.15.26:0/634816649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 msgr2=0x5578b0caafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.847+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0caafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.847+0000 7f6e2d79b700 1 -- 172.21.15.26:0/634816649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 msgr2=0x5578b0cb87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.847+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.847+0000 7f6e2d79b700 1 -- 172.21.15.26:0/634816649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578b0a29d80 con 0x5578b0cab520 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.848+0000 7f6e1e7fc700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 0x7f6e140a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e1e7fc700 1 -- 172.21.15.26:0/3281776134 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e200cca40 con 0x7f6e2819df00 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009cf0 tx=0x7f6e240103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd47d9e87b4acca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 0x7f6e140a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e137fe700 1 -- 172.21.15.26:0/634816649 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24010d00 con 0x5578b0cab520 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e137fe700 1 -- 172.21.15.26:0/634816649 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24015070 con 0x5578b0cab520 2023-10-06T21:23:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.849+0000 7f6e137fe700 1 -- 172.21.15.26:0/634816649 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011840 con 0x5578b0cab520 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.850+0000 7f6e432c3b80 1 -- 172.21.15.26:0/634816649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 msgr2=0x5578b0cb5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.850+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 0x7f6e140a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e2819ab70 tx=0x7f6e1801a040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.850+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009cf0 tx=0x7f6e240103d0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 -- 172.21.15.26:0/634816649 shutdown_connections 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0caafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/634816649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb5e90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 -- 172.21.15.26:0/634816649 >> 172.21.15.26:0/634816649 conn(0x5578b0c20200 msgr2=0x5578b0ca7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 -- 172.21.15.26:0/634816649 shutdown_connections 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 -- 172.21.15.26:0/634816649 wait complete. 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.851+0000 7f6e432c3b80 1 -- start start 2023-10-06T21:23:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 Processor -- start 2023-10-06T21:23:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0bfedd0 con 0x5578b0cb63d0 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0b9b6b0 con 0x5578b0caac00 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0b9b350 con 0x5578b0cab520 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e432c3b80 10 client.0 _ll_register_callbacks cb 0x5578b09f87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42588/0 (socket says 172.21.15.26:42588) 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.852+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3917847071 learned_addr learned my addr 172.21.15.26:0/3917847071 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3917847071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 msgr2=0x5578b0cb36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3917847071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 msgr2=0x5578b0cb3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3917847071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e24009920 con 0x5578b0caac00 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009600 tx=0x7f6e24004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac035ddb723ad75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.853+0000 7f6e10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24005ce0 con 0x5578b0caac00 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578b0c9ed50 con 0x5578b0caac00 2023-10-06T21:23:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e24017070 con 0x5578b0caac00 2023-10-06T21:23:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e24004520 con 0x5578b0caac00 2023-10-06T21:23:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e432c3b80 10 client.27338 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5578b0a12c00 con 0x5578b0caac00 2023-10-06T21:23:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578b0c9ed50 con 0x5578b0caac00 2023-10-06T21:23:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.854+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6e24004900 con 0x5578b0caac00 2023-10-06T21:23:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 10 client.27338 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 10 client.27338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 20 client.27338 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0cce640 0x5578b0cd0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 4 client.27338 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578b0bf7db0 con 0x5578b0cce640 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.855+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0cce640 0x5578b0cd0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.856+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0cce640 0x5578b0cd0a60 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb013029aa2731c server_cookie=45f124769d5e3a8 in_seq=0 out_seq=0 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.856+0000 7f6e10ff9700 10 client.27338 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.857+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3917847071 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5578b0bf7db0 con 0x5578b0cce640 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.857+0000 7f6e10ff9700 10 client.27338 handle_command_reply: tid=0 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.857+0000 7f6e432c3b80 1 client.27338 shutdown 2023-10-06T21:23:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 msgr2=0x5578b0cb3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009600 tx=0x7f6e24004300 comp rx=0 tx=0).stop 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 shutdown_connections 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0cb3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0cce640 0x5578b0cd0a60 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cb3160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3917847071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0cb36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 >> 172.21.15.26:0/3917847071 conn(0x5578b0c20200 msgr2=0x5578b0ca6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e10ff9700 0 client.27338 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.858+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 shutdown_connections 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.859+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3917847071 wait complete. 2023-10-06T21:23:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.859+0000 7f6e432c3b80 20 client.27338 trim_cache size 0 max 16384 2023-10-06T21:23:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:44 smithi078 ceph-mon[134896]: pgmap v420: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2023-10-06T21:23:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.882+0000 7f6e432c3b80 1 Processor -- start 2023-10-06T21:23:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.882+0000 7f6e432c3b80 1 -- start start 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cab520 0x5578b0cae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 0x5578b0caea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0a7f8c0 con 0x5578b0cab520 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0cc5c70 con 0x5578b0caac00 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0cb1700 con 0x5578b0cb63d0 2023-10-06T21:23:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42600/0 (socket says 172.21.15.26:42600) 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e2df9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 0x5578b0caea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e2d79b700 1 -- 172.21.15.26:0/1799095233 learned_addr learned my addr 172.21.15.26:0/1799095233 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.883+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cab520 0x5578b0cae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 -- 172.21.15.26:0/1799095233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 msgr2=0x5578b0caea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 0x5578b0caea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 -- 172.21.15.26:0/1799095233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cab520 msgr2=0x5578b0cae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cab520 0x5578b0cae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 -- 172.21.15.26:0/1799095233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578b0a501f0 con 0x5578b0caac00 2023-10-06T21:23:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2df9c700 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 0x5578b0caea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e240059d0 tx=0x7f6e240103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8206a49efc2b43e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e13fff700 1 -- 172.21.15.26:0/1799095233 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003680 con 0x5578b0caac00 2023-10-06T21:23:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e13fff700 1 -- 172.21.15.26:0/1799095233 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24003df0 con 0x5578b0caac00 2023-10-06T21:23:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.884+0000 7f6e13fff700 1 -- 172.21.15.26:0/1799095233 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004680 con 0x5578b0caac00 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.885+0000 7f6e432c3b80 1 -- 172.21.15.26:0/1799095233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 msgr2=0x5578b0cc5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.885+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e240059d0 tx=0x7f6e240103d0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 -- 172.21.15.26:0/1799095233 shutdown_connections 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cb63d0 0x5578b0caea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0cc5730 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/1799095233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cab520 0x5578b0cae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 -- 172.21.15.26:0/1799095233 >> 172.21.15.26:0/1799095233 conn(0x5578b0c20200 msgr2=0x5578b0cb3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 -- 172.21.15.26:0/1799095233 shutdown_connections 2023-10-06T21:23:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 -- 172.21.15.26:0/1799095233 wait complete. 2023-10-06T21:23:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.886+0000 7f6e432c3b80 1 -- start start 2023-10-06T21:23:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.887+0000 7f6e432c3b80 1 Processor -- start 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0d16b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0d1b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0d1b970 con 0x5578b0cb63d0 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0d1bae0 con 0x5578b0caac00 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578b0d1bc50 con 0x5578b0cab520 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e432c3b80 10 client.0 _ll_register_callbacks cb 0x7f6e24009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e2d79b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42608/0 (socket says 172.21.15.26:42608) 2023-10-06T21:23:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.890+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3509976454 learned_addr learned my addr 172.21.15.26:0/3509976454 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3509976454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 msgr2=0x5578b0d16b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:23:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0d16b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3509976454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 msgr2=0x5578b0d1b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0d1b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 -- 172.21.15.26:0/3509976454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e200b7e70 con 0x5578b0caac00 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e2d79b700 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009cf0 tx=0x7f6e24003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f302b05a9f091510 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003c00 con 0x5578b0caac00 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e24000e40 con 0x5578b0caac00 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.891+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e24015070 con 0x5578b0caac00 2023-10-06T21:23:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e24004420 con 0x5578b0caac00 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 10 client.27344 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5578b0a12c00 con 0x5578b0caac00 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24015070 con 0x5578b0caac00 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f6e2403f070 con 0x5578b0caac00 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 10 client.27344 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 10 client.27344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 20 client.27344 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0d33fc0 0x5578b0d363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 4 client.27344 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.892+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5578b0bf7ad0 con 0x5578b0d33fc0 2023-10-06T21:23:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.893+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0d33fc0 0x5578b0d363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.893+0000 7f6e2cf9a700 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0d33fc0 0x5578b0d363e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445174bdee4a03f3 server_cookie=f9489c1cb6e3c939 in_seq=0 out_seq=0 2023-10-06T21:23:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.893+0000 7f6e10ff9700 10 client.27344 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.909+0000 7f6e10ff9700 1 -- 172.21.15.26:0/3509976454 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5578b0bf7ad0 con 0x5578b0d33fc0 2023-10-06T21:23:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.909+0000 7f6e10ff9700 10 client.27344 handle_command_reply: tid=0 2023-10-06T21:23:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.909+0000 7f6e432c3b80 1 client.27344 shutdown 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.910+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 msgr2=0x5578b0d165d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.910+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009cf0 tx=0x7f6e24003f60 comp rx=0 tx=0).stop 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.910+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 shutdown_connections 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.910+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578b0cb63d0 0x5578b0d1b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5578b0d33fc0 0x5578b0d363e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5578b0caac00 0x5578b0d165d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 --2- 172.21.15.26:0/3509976454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578b0cab520 0x5578b0d16b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 >> 172.21.15.26:0/3509976454 conn(0x5578b0c20200 msgr2=0x5578b0cae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e10ff9700 0 client.27344 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 shutdown_connections 2023-10-06T21:23:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.911+0000 7f6e432c3b80 1 -- 172.21.15.26:0/3509976454 wait complete. 2023-10-06T21:23:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e432c3b80 20 client.27344 trim_cache size 0 max 16384 2023-10-06T21:23:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:23:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:23:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:23:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:23:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 msgr2=0x7f6e140a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 0x7f6e140a3b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e2819ab70 tx=0x7f6e1801a040 comp rx=0 tx=0).stop 2023-10-06T21:23:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 msgr2=0x7f6e2819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 0x7f6e2819e380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007f10 tx=0x7f6e2000d3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 shutdown_connections 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e2819df00 0x7f6e2819e380 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6e140a1670 0x7f6e140a3b30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e28068c90 0x7f6e28199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 --2- 172.21.15.26:0/3281776134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e28106f60 0x7f6e28199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.912+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 >> 172.21.15.26:0/3281776134 conn(0x7f6e28076b70 msgr2=0x7f6e280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.913+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 shutdown_connections 2023-10-06T21:23:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:44.913+0000 7f6e2f9ff700 1 -- 172.21.15.26:0/3281776134 wait complete. 2023-10-06T21:23:47.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:46 smithi167 ceph-mon[134494]: pgmap v421: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-10-06T21:23:47.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:46 smithi026 ceph-mon[98490]: pgmap v421: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-10-06T21:23:47.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:46 smithi078 ceph-mon[134896]: pgmap v421: 97 pgs: 97 active+clean; 5.4 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-10-06T21:23:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 785.0 s, 1148.9 tps, lat 88.047 ms stddev 98.097 2023-10-06T21:23:49.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:48 smithi167 ceph-mon[134494]: pgmap v422: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-10-06T21:23:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:48 smithi026 ceph-mon[98490]: pgmap v422: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-10-06T21:23:49.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:48 smithi078 ceph-mon[134896]: pgmap v422: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 25 MiB/s wr, 1.72k op/s 2023-10-06T21:23:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:50 smithi167 ceph-mon[134494]: pgmap v423: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 533 op/s 2023-10-06T21:23:51.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:50 smithi026 ceph-mon[98490]: pgmap v423: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 533 op/s 2023-10-06T21:23:51.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:50 smithi078 ceph-mon[134896]: pgmap v423: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 533 op/s 2023-10-06T21:23:51.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:23:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.056+0000 7fb8d63ff700 1 -- 172.21.15.26:0/164006936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0108420 msgr2=0x7fb8d010ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.056+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/164006936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0108420 0x7fb8d010ffa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b0a0 tx=0x7fb8c800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 -- 172.21.15.26:0/164006936 shutdown_connections 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/164006936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0108420 0x7fb8d010ffa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/164006936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0103300 0x7fb8d01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/164006936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d0103cb0 0x7fb8d0107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 -- 172.21.15.26:0/164006936 >> 172.21.15.26:0/164006936 conn(0x7fb8d00feb40 msgr2=0x7fb8d0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 -- 172.21.15.26:0/164006936 shutdown_connections 2023-10-06T21:23:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.057+0000 7fb8d63ff700 1 -- 172.21.15.26:0/164006936 wait complete. 2023-10-06T21:23:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.059+0000 7fb8d63ff700 1 Processor -- start 2023-10-06T21:23:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.059+0000 7fb8d63ff700 1 -- start start 2023-10-06T21:23:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 0x7fb8d0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d019dbe0 0x7fb8d019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d019e660 con 0x7fb8d019dbe0 2023-10-06T21:23:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d019e7d0 con 0x7fb8d0103300 2023-10-06T21:23:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8d63ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d019e990 con 0x7fb8d0108420 2023-10-06T21:23:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38318/0 (socket says 172.21.15.26:38318) 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8cffff700 1 -- 172.21.15.26:0/1611499907 learned_addr learned my addr 172.21.15.26:0/1611499907 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.060+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 0x7fb8d0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 -- 172.21.15.26:0/1611499907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 msgr2=0x7fb8d0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 0x7fb8d0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 -- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d019dbe0 msgr2=0x7fb8d019e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d019dbe0 0x7fb8d019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 -- 172.21.15.26:0/1611499907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8009d00 con 0x7fb8d0103300 2023-10-06T21:23:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 0x7fb8d0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b530 tx=0x7fb8c000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.061+0000 7fb8cdffb700 1 -- 172.21.15.26:0/1611499907 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0011830 con 0x7fb8d0103300 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.062+0000 7fb8cdffb700 1 -- 172.21.15.26:0/1611499907 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8c0011e50 con 0x7fb8d0103300 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.062+0000 7fb8cdffb700 1 -- 172.21.15.26:0/1611499907 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000f2e0 con 0x7fb8d0103300 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.062+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d01a5ef0 con 0x7fb8d0103300 2023-10-06T21:23:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.062+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d01a6320 con 0x7fb8d0103300 2023-10-06T21:23:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.066+0000 7fb8cdffb700 1 -- 172.21.15.26:0/1611499907 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb8c00105f0 con 0x7fb8d0103300 2023-10-06T21:23:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.068+0000 7fb8cdffb700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 0x7fb8bc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.068+0000 7fb8cdffb700 1 -- 172.21.15.26:0/1611499907 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb8c00ccae0 con 0x7fb8d0103300 2023-10-06T21:23:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.070+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 0x7fb8bc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.071+0000 7fb8e9d1cb80 1 Processor -- start 2023-10-06T21:23:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.072+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 0x7fb8bc0a3b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8d019a850 tx=0x7fb8b8009190 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.072+0000 7fb8e9d1cb80 1 -- start start 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5dfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f53b460 con 0x556a7f5d29e0 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f52cc20 con 0x556a7f5dd890 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.073+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f52a540 con 0x556a7f5d20c0 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34108/0 (socket says 172.21.15.26:34108) 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60964/0 (socket says 172.21.15.26:60964) 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1671498563 learned_addr learned my addr 172.21.15.26:0/1671498563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8d499c700 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5dfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1671498563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 msgr2=0x556a7f5d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1671498563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 msgr2=0x556a7f5dfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5dfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1671498563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a7f350de0 con 0x556a7f5d29e0 2023-10-06T21:23:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.074+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:23:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.075+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800edf0 tx=0x7fb8b8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01d2c5876232efc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.075+0000 7fb8c67fc700 1 -- 172.21.15.26:0/1671498563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800eee0 con 0x556a7f5d29e0 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.075+0000 7fb8c67fc700 1 -- 172.21.15.26:0/1671498563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8b8015d50 con 0x556a7f5d29e0 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.075+0000 7fb8c67fc700 1 -- 172.21.15.26:0/1671498563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8013da0 con 0x556a7f5d29e0 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1671498563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 msgr2=0x556a7f5dd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800edf0 tx=0x7fb8b8011000 comp rx=0 tx=0).stop 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1671498563 shutdown_connections 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5dfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1671498563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f5dd350 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1671498563 >> 172.21.15.26:0/1671498563 conn(0x556a7f547260 msgr2=0x556a7f5ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1671498563 shutdown_connections 2023-10-06T21:23:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.076+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1671498563 wait complete. 2023-10-06T21:23:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.077+0000 7fb8e9d1cb80 1 -- start start 2023-10-06T21:23:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.077+0000 7fb8e9d1cb80 1 Processor -- start 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 0x556a7f5dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f525e30 con 0x556a7f5dd890 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f4c2710 con 0x556a7f5d29e0 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f4c23b0 con 0x556a7f5d20c0 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8e9d1cb80 10 client.0 _ll_register_callbacks cb 0x556a7f31f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.078+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38338/0 (socket says 172.21.15.26:38338) 2023-10-06T21:23:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1477676685 learned_addr learned my addr 172.21.15.26:0/1477676685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8d499c700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 0x556a7f5dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1477676685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 msgr2=0x556a7f5dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1477676685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 msgr2=0x556a7f5dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 0x556a7f5dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1477676685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d01a6320 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.079+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80048a0 tx=0x7fb8b8015bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931da641b2a7f5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8d499c700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 0x556a7f5dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8013d90 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb8b8032410 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.080+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a7f5c5f10 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b8013ef0 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 10 client.27356 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fb8b800eaa0 con 0x556a7f5d29e0 2023-10-06T21:23:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a7f5c5f10 con 0x556a7f5d29e0 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb8b801f660 con 0x556a7f5d29e0 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 10 client.27356 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 10 client.27356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 20 client.27356 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f5f5af0 0x556a7f5f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 4 client.27356 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.081+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a7f51ee10 con 0x556a7f5f5af0 2023-10-06T21:23:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.082+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f5f5af0 0x556a7f5f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.082+0000 7fb8cffff700 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f5f5af0 0x556a7f5f7f10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9ca9eec447a7 server_cookie=e07932c26805aa77 in_seq=0 out_seq=0 2023-10-06T21:23:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.082+0000 7fb8af7fe700 10 client.27356 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8af7fe700 1 -- 172.21.15.26:0/1477676685 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a7f51ee10 con 0x556a7f5f5af0 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8af7fe700 10 client.27356 handle_command_reply: tid=0 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8e9d1cb80 1 client.27356 shutdown 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 msgr2=0x556a7f5da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80048a0 tx=0x7fb8b8015bc0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.083+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 shutdown_connections 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5dd890 0x556a7f5dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f5dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f5f5af0 0x556a7f5f7f10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/1477676685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5da520 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8af7fe700 0 client.27356 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 >> 172.21.15.26:0/1477676685 conn(0x556a7f547260 msgr2=0x556a7f5cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 shutdown_connections 2023-10-06T21:23:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/1477676685 wait complete. 2023-10-06T21:23:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.084+0000 7fb8e9d1cb80 20 client.27356 trim_cache size 0 max 16384 2023-10-06T21:23:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.111+0000 7fb8e9d1cb80 1 Processor -- start 2023-10-06T21:23:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.111+0000 7fb8e9d1cb80 1 -- start start 2023-10-06T21:23:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 0x556a7f5d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5dd890 0x556a7f5da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f3a6920 con 0x556a7f5d20c0 2023-10-06T21:23:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f5d32e0 con 0x556a7f5d29e0 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f5caa20 con 0x556a7f5dd890 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 0x556a7f5d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8c7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38354/0 (socket says 172.21.15.26:38354) 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 0x556a7f5d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34130/0 (socket says 172.21.15.26:34130) 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8c7fff700 1 -- 172.21.15.26:0/243654364 learned_addr learned my addr 172.21.15.26:0/243654364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.112+0000 7fb8d499c700 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5dd890 0x556a7f5da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 -- 172.21.15.26:0/243654364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5dd890 msgr2=0x556a7f5da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5dd890 0x556a7f5da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 -- 172.21.15.26:0/243654364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 msgr2=0x556a7f5d5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 0x556a7f5d5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 -- 172.21.15.26:0/243654364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a7f339c60 con 0x556a7f5d29e0 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8004740 tx=0x7fb8b8009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8095051b6cb29a3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c6ffd700 1 -- 172.21.15.26:0/243654364 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8013b90 con 0x556a7f5d29e0 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c6ffd700 1 -- 172.21.15.26:0/243654364 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8b8013cf0 con 0x556a7f5d29e0 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.113+0000 7fb8c6ffd700 1 -- 172.21.15.26:0/243654364 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8032c70 con 0x556a7f5d29e0 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.114+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/243654364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 msgr2=0x556a7f5d9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.114+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8004740 tx=0x7fb8b8009b20 comp rx=0 tx=0).stop 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/243654364 shutdown_connections 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5dd890 0x556a7f5da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d20c0 0x556a7f5d5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/243654364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5d29e0 0x556a7f5d9d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/243654364 >> 172.21.15.26:0/243654364 conn(0x556a7f547260 msgr2=0x556a7f5d58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/243654364 shutdown_connections 2023-10-06T21:23:52.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.115+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/243654364 wait complete. 2023-10-06T21:23:52.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.116+0000 7fb8e9d1cb80 1 -- start start 2023-10-06T21:23:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.116+0000 7fb8e9d1cb80 1 Processor -- start 2023-10-06T21:23:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f642ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f643220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f5ec8a0 con 0x556a7f5d29e0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f5eca10 con 0x556a7f5dd890 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7f5ecbd0 con 0x556a7f5d20c0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8e9d1cb80 10 client.0 _ll_register_callbacks cb 0x7fb8c800b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8d499c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8d499c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38366/0 (socket says 172.21.15.26:38366) 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.117+0000 7fb8d499c700 1 -- 172.21.15.26:0/889053952 learned_addr learned my addr 172.21.15.26:0/889053952 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f643220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8cffff700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f642ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 -- 172.21.15.26:0/889053952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 msgr2=0x556a7f642ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f642ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 -- 172.21.15.26:0/889053952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 msgr2=0x556a7f643220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f643220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 -- 172.21.15.26:0/889053952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b800eaa0 con 0x556a7f5dd890 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8d499c700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b0a0 tx=0x7fb8c8003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c96af4b3f44686e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.118+0000 7fb8af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8020070 con 0x556a7f5dd890 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb8c8007820 con 0x556a7f5dd890 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8b8015d50 con 0x556a7f5dd890 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8c800e040 con 0x556a7f5dd890 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8e9d1cb80 10 client.27362 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.119+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x556a7f377250 con 0x556a7f5dd890 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8b8015d50 con 0x556a7f5dd890 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fb8c802b020 con 0x556a7f5dd890 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 10 client.27362 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 20 client.27362 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f64abe0 0x556a7f64d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 4 client.27362 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.120+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556a7f51eb30 con 0x556a7f64abe0 2023-10-06T21:23:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.121+0000 7fb8cffff700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f64abe0 0x556a7f64d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.121+0000 7fb8af7fe700 10 client.27362 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.121+0000 7fb8cffff700 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f64abe0 0x556a7f64d000 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1895e751d0ef6b6 server_cookie=7b1397949a3cd690 in_seq=0 out_seq=0 2023-10-06T21:23:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.122+0000 7fb8af7fe700 1 -- 172.21.15.26:0/889053952 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556a7f51eb30 con 0x556a7f64abe0 2023-10-06T21:23:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.122+0000 7fb8af7fe700 10 client.27362 handle_command_reply: tid=0 2023-10-06T21:23:52.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.122+0000 7fb8e9d1cb80 1 client.27362 shutdown 2023-10-06T21:23:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 790.0 s, 1054.7 tps, lat 93.780 ms stddev 102.618 2023-10-06T21:23:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.122+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 msgr2=0x556a7f5ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.122+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b0a0 tx=0x7fb8c8003f40 comp rx=0 tx=0).stop 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 shutdown_connections 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556a7f64abe0 0x556a7f64d000 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a7f5dd890 0x556a7f5ec290 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7f5d20c0 0x556a7f642ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 --2- 172.21.15.26:0/889053952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556a7f5d29e0 0x556a7f643220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 >> 172.21.15.26:0/889053952 conn(0x556a7f547260 msgr2=0x556a7f5dbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8af7fe700 0 client.27362 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:52.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 shutdown_connections 2023-10-06T21:23:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.126+0000 7fb8e9d1cb80 1 -- 172.21.15.26:0/889053952 wait complete. 2023-10-06T21:23:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.127+0000 7fb8e9d1cb80 20 client.27362 trim_cache size 0 max 16384 2023-10-06T21:23:52.129 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:23:52.130 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 msgr2=0x7fb8bc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 0x7fb8bc0a3b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8d019a850 tx=0x7fb8b8009190 comp rx=0 tx=0).stop 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 msgr2=0x7fb8d0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000b530 tx=0x7fb8c000b840 comp rx=0 tx=0).stop 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 shutdown_connections 2023-10-06T21:23:52.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb8bc0a16b0 0x7fb8bc0a3b70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d019dbe0 0x7fb8d019e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d0103300 0x7fb8d0199240 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 --2- 172.21.15.26:0/1611499907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8d0108420 0x7fb8d0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 >> 172.21.15.26:0/1611499907 conn(0x7fb8d00feb40 msgr2=0x7fb8d0106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 shutdown_connections 2023-10-06T21:23:52.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:23:52.128+0000 7fb8d63ff700 1 -- 172.21.15.26:0/1611499907 wait complete. 2023-10-06T21:23:52.156 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:23:53.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:52 smithi167 ceph-mon[134494]: pgmap v424: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-06T21:23:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:52 smithi026 ceph-mon[98490]: pgmap v424: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-06T21:23:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:52 smithi078 ceph-mon[134896]: pgmap v424: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 664 op/s 2023-10-06T21:23:54.626 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:54 smithi078 ceph-mon[134896]: pgmap v425: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 642 op/s 2023-10-06T21:23:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:54 smithi167 ceph-mon[134494]: pgmap v425: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 642 op/s 2023-10-06T21:23:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:54 smithi026 ceph-mon[98490]: pgmap v425: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 642 op/s 2023-10-06T21:23:55.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:55 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:55.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:55 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:55.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:55 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:23:56.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:56 smithi078 ceph-mon[134896]: pgmap v426: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-06T21:23:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:56 smithi167 ceph-mon[134494]: pgmap v426: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-06T21:23:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:56 smithi026 ceph-mon[98490]: pgmap v426: 97 pgs: 97 active+clean; 5.5 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 544 op/s 2023-10-06T21:23:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 795.0 s, 904.8 tps, lat 109.621 ms stddev 114.082 2023-10-06T21:23:57.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:57 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:57.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:57 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:57.621 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:57 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:23:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:57 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:57 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:57.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:57 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:23:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:57 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:23:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:57 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:23:57.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:57 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:23:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.194+0000 7f6271fff700 1 -- 172.21.15.26:0/2567761002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c108420 msgr2=0x7f626c10ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.194+0000 7f6271fff700 1 --2- 172.21.15.26:0/2567761002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c108420 0x7f626c10ffa0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f625c009990 tx=0x7f625c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.195+0000 7f6271fff700 1 -- 172.21.15.26:0/2567761002 shutdown_connections 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.195+0000 7f6271fff700 1 --2- 172.21.15.26:0/2567761002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c108420 0x7f626c10ffa0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.195+0000 7f6271fff700 1 --2- 172.21.15.26:0/2567761002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c103300 0x7f626c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.195+0000 7f6271fff700 1 --2- 172.21.15.26:0/2567761002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103cb0 0x7f626c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.195+0000 7f6271fff700 1 -- 172.21.15.26:0/2567761002 >> 172.21.15.26:0/2567761002 conn(0x7f626c0feb20 msgr2=0x7f626c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.196+0000 7f6271fff700 1 -- 172.21.15.26:0/2567761002 shutdown_connections 2023-10-06T21:23:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.196+0000 7f6271fff700 1 -- 172.21.15.26:0/2567761002 wait complete. 2023-10-06T21:23:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 Processor -- start 2023-10-06T21:23:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 -- start start 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103300 0x7f626c10f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c108420 0x7f626c10fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c10dd00 con 0x7f626c0729d0 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c10de70 con 0x7f626c103300 2023-10-06T21:23:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.198+0000 7f6271fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c10e030 con 0x7f626c108420 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.199+0000 7f626bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.199+0000 7f626bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34160/0 (socket says 172.21.15.26:34160) 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.199+0000 7f626bfff700 1 -- 172.21.15.26:0/857207145 learned_addr learned my addr 172.21.15.26:0/857207145 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626affd700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c108420 0x7f626c10fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626b7fe700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103300 0x7f626c10f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626bfff700 1 -- 172.21.15.26:0/857207145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c108420 msgr2=0x7f626c10fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626bfff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c108420 0x7f626c10fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626bfff700 1 -- 172.21.15.26:0/857207145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103300 msgr2=0x7f626c10f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626bfff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103300 0x7f626c10f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.200+0000 7f626bfff700 1 -- 172.21.15.26:0/857207145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0095f0 con 0x7f626c0729d0 2023-10-06T21:23:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.201+0000 7f626bfff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f625c005910 tx=0x7f625c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.202+0000 7f6268ff9700 1 -- 172.21.15.26:0/857207145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c01d070 con 0x7f626c0729d0 2023-10-06T21:23:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.202+0000 7f6268ff9700 1 -- 172.21.15.26:0/857207145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f625c021410 con 0x7f626c0729d0 2023-10-06T21:23:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.202+0000 7f6268ff9700 1 -- 172.21.15.26:0/857207145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c017650 con 0x7f626c0729d0 2023-10-06T21:23:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.202+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c10e2b0 con 0x7f626c0729d0 2023-10-06T21:23:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.202+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c1a2130 con 0x7f626c0729d0 2023-10-06T21:23:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.209+0000 7f6268ff9700 1 -- 172.21.15.26:0/857207145 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f625c021580 con 0x7f626c0729d0 2023-10-06T21:23:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.210+0000 7f628581fb80 1 Processor -- start 2023-10-06T21:23:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.210+0000 7f628581fb80 1 -- start start 2023-10-06T21:23:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce851400 con 0x5654ce8e7c00 2023-10-06T21:23:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce842bc0 con 0x5654ce8f33d0 2023-10-06T21:23:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f628581fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce8404e0 con 0x5654ce8e8520 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.211+0000 7f626affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.212+0000 7f626affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8e7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34176/0 (socket says 172.21.15.26:34176) 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.212+0000 7f626affd700 1 -- 172.21.15.26:0/1551635363 learned_addr learned my addr 172.21.15.26:0/1551635363 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.212+0000 7f626b7fe700 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.212+0000 7f626affd700 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.212+0000 7f6268ff9700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 0x7f62540a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f6268ff9700 1 -- 172.21.15.26:0/857207145 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f625c0ce3e0 con 0x7f626c0729d0 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 -- 172.21.15.26:0/1551635363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 msgr2=0x5654ce8f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 -- 172.21.15.26:0/1551635363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 msgr2=0x5654ce8e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8e7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 -- 172.21.15.26:0/1551635363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654ce666d80 con 0x5654ce8f33d0 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.213+0000 7f626affd700 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8f57c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f62600125b0 tx=0x7f62600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d9a92ddc808c1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.214+0000 7f6251ffb700 1 -- 172.21.15.26:0/1551635363 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260012e40 con 0x5654ce8f33d0 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.214+0000 7f6251ffb700 1 -- 172.21.15.26:0/1551635363 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6260004500 con 0x5654ce8f33d0 2023-10-06T21:23:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.214+0000 7f6251ffb700 1 -- 172.21.15.26:0/1551635363 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260005280 con 0x5654ce8f33d0 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.214+0000 7f626bfff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 0x7f62540a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.215+0000 7f626bfff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 0x7f62540a3cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f625c0a32c0 tx=0x7f625c005bc0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.215+0000 7f628581fb80 1 -- 172.21.15.26:0/1551635363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 msgr2=0x5654ce8f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.215+0000 7f628581fb80 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8f57c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f62600125b0 tx=0x7f62600047c0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 -- 172.21.15.26:0/1551635363 shutdown_connections 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8f57c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 --2- 172.21.15.26:0/1551635363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 -- 172.21.15.26:0/1551635363 >> 172.21.15.26:0/1551635363 conn(0x5654ce85d200 msgr2=0x5654ce8e4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 -- 172.21.15.26:0/1551635363 shutdown_connections 2023-10-06T21:23:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.216+0000 7f628581fb80 1 -- 172.21.15.26:0/1551635363 wait complete. 2023-10-06T21:23:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.217+0000 7f628581fb80 1 -- start start 2023-10-06T21:23:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.217+0000 7f628581fb80 1 Processor -- start 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e8520 0x5654ce8f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce83bdd0 con 0x5654ce8e7c00 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce7d86b0 con 0x5654ce8e8520 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce7d8350 con 0x5654ce8f33d0 2023-10-06T21:23:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.219+0000 7f628581fb80 10 client.0 _ll_register_callbacks cb 0x5654ce6357a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.220+0000 7f626affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.220+0000 7f626affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:32816/0 (socket says 172.21.15.26:32816) 2023-10-06T21:23:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.220+0000 7f626affd700 1 -- 172.21.15.26:0/213857339 learned_addr learned my addr 172.21.15.26:0/213857339 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.220+0000 7f626b7fe700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e8520 0x5654ce8f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.220+0000 7f626affd700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 -- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e8520 msgr2=0x5654ce8f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e8520 0x5654ce8f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 -- 172.21.15.26:0/213857339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 msgr2=0x5654ce8f0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8f0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 -- 172.21.15.26:0/213857339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654ce64fc00 con 0x5654ce8f33d0 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f626affd700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f626000b4f0 tx=0x7f6260004670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ecbdf371a290252 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.221+0000 7f624affd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004b00 con 0x5654ce8f33d0 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654ce8dbd50 con 0x5654ce8f33d0 2023-10-06T21:23:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6260012e40 con 0x5654ce8f33d0 2023-10-06T21:23:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6260004d70 con 0x5654ce8f33d0 2023-10-06T21:23:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f628581fb80 10 client.27220 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5654ce68d1f0 con 0x5654ce8f33d0 2023-10-06T21:23:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.222+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6260031020 con 0x5654ce8f33d0 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f626002c070 con 0x5654ce8f33d0 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 10 client.27220 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 10 client.27220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 20 client.27220 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce913fa0 0x5654ce9163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 4 client.27220 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.223+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654ce834db0 con 0x5654ce913fa0 2023-10-06T21:23:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.224+0000 7f626b7fe700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce913fa0 0x5654ce9163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.226+0000 7f626b7fe700 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce913fa0 0x5654ce9163c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c7684962022819 server_cookie=47c117510cc0809d in_seq=0 out_seq=0 2023-10-06T21:23:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.226+0000 7f624affd700 10 client.27220 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.227+0000 7f624affd700 1 -- 172.21.15.26:0/213857339 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5654ce834db0 con 0x5654ce913fa0 2023-10-06T21:23:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.227+0000 7f624affd700 10 client.27220 handle_command_reply: tid=0 2023-10-06T21:23:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.227+0000 7f628581fb80 1 client.27220 shutdown 2023-10-06T21:23:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.227+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 msgr2=0x5654ce8f0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.227+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f626000b4f0 tx=0x7f6260004670 comp rx=0 tx=0).stop 2023-10-06T21:23:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 shutdown_connections 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8f33d0 0x5654ce8f0bf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e7c00 0x5654ce8f0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce913fa0 0x5654ce9163c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 --2- 172.21.15.26:0/213857339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e8520 0x5654ce8f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f624affd700 0 client.27220 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 >> 172.21.15.26:0/213857339 conn(0x5654ce85d200 msgr2=0x5654ce8e3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 shutdown_connections 2023-10-06T21:23:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.228+0000 7f628581fb80 1 -- 172.21.15.26:0/213857339 wait complete. 2023-10-06T21:23:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.229+0000 7f628581fb80 20 client.27220 trim_cache size 0 max 16384 2023-10-06T21:23:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.259+0000 7f628581fb80 1 Processor -- start 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.259+0000 7f628581fb80 1 -- start start 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e8520 0x5654ce8ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce6bc8c0 con 0x5654ce8e8520 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce8eaff0 con 0x5654ce8f33d0 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f628581fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce8fa450 con 0x5654ce8e7c00 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f626affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f626affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:32822/0 (socket says 172.21.15.26:32822) 2023-10-06T21:23:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f626affd700 1 -- 172.21.15.26:0/59392540 learned_addr learned my addr 172.21.15.26:0/59392540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.260+0000 7f626affd700 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 -- 172.21.15.26:0/59392540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 msgr2=0x5654ce8ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 -- 172.21.15.26:0/59392540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e8520 msgr2=0x5654ce8ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e8520 0x5654ce8ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 -- 172.21.15.26:0/59392540 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654ce665250 con 0x5654ce8e7c00 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f626affd700 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6260005dd0 tx=0x7f62600107e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf891032810cefc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.261+0000 7f62527fc700 1 -- 172.21.15.26:0/59392540 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260005950 con 0x5654ce8e7c00 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.262+0000 7f62527fc700 1 -- 172.21.15.26:0/59392540 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6260016020 con 0x5654ce8e7c00 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.262+0000 7f62527fc700 1 -- 172.21.15.26:0/59392540 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626001e8f0 con 0x5654ce8e7c00 2023-10-06T21:23:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 -- 172.21.15.26:0/59392540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 msgr2=0x5654ce8f9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6260005dd0 tx=0x7f62600107e0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 -- 172.21.15.26:0/59392540 shutdown_connections 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8f33d0 0x5654ce8ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9f10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 --2- 172.21.15.26:0/59392540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8e8520 0x5654ce8ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 -- 172.21.15.26:0/59392540 >> 172.21.15.26:0/59392540 conn(0x5654ce85d200 msgr2=0x5654ce8eb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 -- 172.21.15.26:0/59392540 shutdown_connections 2023-10-06T21:23:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.263+0000 7f628581fb80 1 -- 172.21.15.26:0/59392540 wait complete. 2023-10-06T21:23:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.264+0000 7f628581fb80 1 -- start start 2023-10-06T21:23:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.264+0000 7f628581fb80 1 Processor -- start 2023-10-06T21:23:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8f33d0 0x5654ce966dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce8f9fc0 con 0x5654ce8f33d0 2023-10-06T21:23:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce967310 con 0x5654ce8e7c00 2023-10-06T21:23:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654ce967480 con 0x5654ce8e8520 2023-10-06T21:23:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.265+0000 7f628581fb80 10 client.0 _ll_register_callbacks cb 0x7f6260012110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:23:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38408/0 (socket says 172.21.15.26:38408) 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 -- 172.21.15.26:0/1073338563 learned_addr learned my addr 172.21.15.26:0/1073338563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 -- 172.21.15.26:0/1073338563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 msgr2=0x5654ce8f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 -- 172.21.15.26:0/1073338563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8f33d0 msgr2=0x5654ce966dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:23:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.266+0000 7f626affd700 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8f33d0 0x5654ce966dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.267+0000 7f626affd700 1 -- 172.21.15.26:0/1073338563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654ce68d1f0 con 0x5654ce8e7c00 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.267+0000 7f626affd700 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6260014f90 tx=0x7f6260014fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db4b277cd939b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.267+0000 7f624affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.267+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260005950 con 0x5654ce8e7c00 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.267+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6260012e40 con 0x5654ce8e7c00 2023-10-06T21:23:58.271 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.268+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6260004500 con 0x5654ce8e7c00 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.268+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62600108d0 con 0x5654ce8e7c00 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.268+0000 7f628581fb80 10 client.27377 fetch_fsmap learned FSMap version 53 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.268+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5654ce733a20 con 0x5654ce8e7c00 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6260004500 con 0x5654ce8e7c00 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 53) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f626002c030 con 0x5654ce8e7c00 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 10 client.27377 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 10 client.27377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:23:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 20 client.27377 populate_metadata read hostname 'smithi026' 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce968720 0x5654ce96ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 4 client.27377 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5654ce834ad0 con 0x5654ce968720 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.269+0000 7f626b7fe700 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce968720 0x5654ce96ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.270+0000 7f624affd700 10 client.27377 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.270+0000 7f626b7fe700 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce968720 0x5654ce96ab10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6080451b88b0435 server_cookie=10e551cdbb2620e2 in_seq=0 out_seq=0 2023-10-06T21:23:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.284+0000 7f624affd700 1 -- 172.21.15.26:0/1073338563 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5654ce834ad0 con 0x5654ce968720 2023-10-06T21:23:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.284+0000 7f624affd700 10 client.27377 handle_command_reply: tid=0 2023-10-06T21:23:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.284+0000 7f628581fb80 1 client.27377 shutdown 2023-10-06T21:23:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 msgr2=0x5654ce8f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6260014f90 tx=0x7f6260014fc0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 shutdown_connections 2023-10-06T21:23:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654ce8f33d0 0x5654ce966dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.288 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5654ce968720 0x5654ce96ab10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654ce8e7c00 0x5654ce8f9140 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 --2- 172.21.15.26:0/1073338563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654ce8e8520 0x5654ce8f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 >> 172.21.15.26:0/1073338563 conn(0x5654ce85d200 msgr2=0x5654ce8f00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f624affd700 0 client.27377 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.285+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 shutdown_connections 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.286+0000 7f628581fb80 1 -- 172.21.15.26:0/1073338563 wait complete. 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.286+0000 7f628581fb80 20 client.27377 trim_cache size 0 max 16384 2023-10-06T21:23:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:23:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:23:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:23:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.287+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 msgr2=0x7f62540a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.287+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 0x7f62540a3cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f625c0a32c0 tx=0x7f625c005bc0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.287+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 msgr2=0x7f626c10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.287+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f625c005910 tx=0x7f625c005740 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 shutdown_connections 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62540a1800 0x7f62540a3cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f626c0729d0 0x7f626c10d7c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c103300 0x7f626c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 --2- 172.21.15.26:0/857207145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626c108420 0x7f626c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:23:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.288+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 >> 172.21.15.26:0/857207145 conn(0x7f626c0feb20 msgr2=0x7f626c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:23:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.290+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 shutdown_connections 2023-10-06T21:23:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:23:58.290+0000 7f6271fff700 1 -- 172.21.15.26:0/857207145 wait complete. 2023-10-06T21:23:58.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:23:58 smithi167 ceph-mon[134494]: pgmap v427: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 834 op/s 2023-10-06T21:23:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:23:58 smithi026 ceph-mon[98490]: pgmap v427: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 834 op/s 2023-10-06T21:23:58.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:23:58 smithi078 ceph-mon[134896]: pgmap v427: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 834 op/s 2023-10-06T21:24:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:00 smithi167 ceph-mon[134494]: pgmap v428: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 563 op/s 2023-10-06T21:24:01.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:00 smithi026 ceph-mon[98490]: pgmap v428: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 563 op/s 2023-10-06T21:24:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:00 smithi078 ceph-mon[134896]: pgmap v428: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 563 op/s 2023-10-06T21:24:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 800.0 s, 1067.5 tps, lat 94.289 ms stddev 137.528 2023-10-06T21:24:03.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:02 smithi167 ceph-mon[134494]: pgmap v429: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-10-06T21:24:03.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:02 smithi026 ceph-mon[98490]: pgmap v429: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-10-06T21:24:03.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:02 smithi078 ceph-mon[134896]: pgmap v429: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-10-06T21:24:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:04 smithi167 ceph-mon[134494]: pgmap v430: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-10-06T21:24:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:04 smithi026 ceph-mon[98490]: pgmap v430: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-10-06T21:24:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:04 smithi078 ceph-mon[134896]: pgmap v430: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-10-06T21:24:06.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:06 smithi167 ceph-mon[134494]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:24:06.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:06 smithi167 ceph-mon[134494]: pgmap v432: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 17 MiB/s wr, 1.44k op/s 2023-10-06T21:24:06.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:06 smithi026 ceph-mon[98490]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:24:06.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:06 smithi026 ceph-mon[98490]: pgmap v432: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 17 MiB/s wr, 1.44k op/s 2023-10-06T21:24:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:06 smithi078 ceph-mon[134896]: osdmap e104: 12 total, 12 up, 12 in 2023-10-06T21:24:06.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:06 smithi078 ceph-mon[134896]: pgmap v432: 97 pgs: 97 active+clean; 5.6 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 17 MiB/s wr, 1.44k op/s 2023-10-06T21:24:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 805.0 s, 968.4 tps, lat 103.801 ms stddev 137.732 2023-10-06T21:24:09.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:08 smithi167 ceph-mon[134494]: pgmap v433: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:08 smithi026 ceph-mon[98490]: pgmap v433: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:08 smithi078 ceph-mon[134896]: pgmap v433: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:10 smithi167 ceph-mon[134494]: pgmap v434: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:11.080 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:10 smithi026 ceph-mon[98490]: pgmap v434: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:11.130 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:10 smithi078 ceph-mon[134896]: pgmap v434: 97 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 78 active+clean; 5.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-10-06T21:24:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.548+0000 7f995ffff700 1 -- 172.21.15.26:0/1540739506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958108690 msgr2=0x7f9958114300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.548+0000 7f995ffff700 1 --2- 172.21.15.26:0/1540739506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958108690 0x7f9958114300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9954009920 tx=0x7f9954009c30 comp rx=0 tx=0).stop 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 -- 172.21.15.26:0/1540739506 shutdown_connections 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 --2- 172.21.15.26:0/1540739506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958108690 0x7f9958114300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 --2- 172.21.15.26:0/1540739506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f995810d630 0x7f995810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 --2- 172.21.15.26:0/1540739506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9958107cf0 0x7f9958108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 -- 172.21.15.26:0/1540739506 >> 172.21.15.26:0/1540739506 conn(0x7f9958076c70 msgr2=0x7f9958079090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 -- 172.21.15.26:0/1540739506 shutdown_connections 2023-10-06T21:24:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.549+0000 7f995ffff700 1 -- 172.21.15.26:0/1540739506 wait complete. 2023-10-06T21:24:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.550+0000 7f995ffff700 1 Processor -- start 2023-10-06T21:24:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.551+0000 7f995ffff700 1 -- start start 2023-10-06T21:24:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.551+0000 7f995ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958107cf0 0x7f99581a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.551+0000 7f995ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.551+0000 7f995ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995810d630 0x7f995819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.551+0000 7f995ffff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819db20 con 0x7f9958107cf0 2023-10-06T21:24:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995ffff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819dc90 con 0x7f9958108690 2023-10-06T21:24:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995ffff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819de50 con 0x7f995810d630 2023-10-06T21:24:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995d59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995d59a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43284/0 (socket says 172.21.15.26:43284) 2023-10-06T21:24:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995d59a700 1 -- 172.21.15.26:0/412105901 learned_addr learned my addr 172.21.15.26:0/412105901 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.552+0000 7f995dd9b700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958107cf0 0x7f99581a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 -- 172.21.15.26:0/412105901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995810d630 msgr2=0x7f995819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995810d630 0x7f995819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 -- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958107cf0 msgr2=0x7f99581a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958107cf0 0x7f99581a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 -- 172.21.15.26:0/412105901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954009520 con 0x7f9958108690 2023-10-06T21:24:11.556 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995d59a700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9950007fa0 tx=0x7f995000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f994effd700 1 -- 172.21.15.26:0/412105901 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000dda0 con 0x7f9958108690 2023-10-06T21:24:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f994effd700 1 -- 172.21.15.26:0/412105901 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995000f040 con 0x7f9958108690 2023-10-06T21:24:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.553+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995819e130 con 0x7f9958108690 2023-10-06T21:24:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.554+0000 7f994effd700 1 -- 172.21.15.26:0/412105901 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950014690 con 0x7f9958108690 2023-10-06T21:24:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.554+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99581aa640 con 0x7f9958108690 2023-10-06T21:24:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 Processor -- start 2023-10-06T21:24:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f994effd700 1 -- 172.21.15.26:0/412105901 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f99500159c0 con 0x7f9958108690 2023-10-06T21:24:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 -- start start 2023-10-06T21:24:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x55874734ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.560+0000 7f99738a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587472a9400 con 0x55874733fc00 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f99738a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874729abc0 con 0x55874734b3d0 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f99738a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587472984e0 con 0x558747340520 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59256/0 (socket says 172.21.15.26:59256) 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995e59c700 1 -- 172.21.15.26:0/2501181261 learned_addr learned my addr 172.21.15.26:0/2501181261 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995d59a700 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995e59c700 1 -- 172.21.15.26:0/2501181261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 msgr2=0x55874734ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.561+0000 7f995e59c700 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x55874734ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f995e59c700 1 -- 172.21.15.26:0/2501181261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 msgr2=0x55874734d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f995e59c700 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f995e59c700 1 -- 172.21.15.26:0/2501181261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587470bed80 con 0x55874733fc00 2023-10-06T21:24:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f995e59c700 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99540094f0 tx=0x7f9954010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea2c2a464a9003 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f9943fff700 1 -- 172.21.15.26:0/2501181261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995401c070 con 0x55874733fc00 2023-10-06T21:24:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f994effd700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 0x7f99440a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f9943fff700 1 -- 172.21.15.26:0/2501181261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9954010df0 con 0x55874733fc00 2023-10-06T21:24:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.562+0000 7f9943fff700 1 -- 172.21.15.26:0/2501181261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400b790 con 0x55874733fc00 2023-10-06T21:24:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.563+0000 7f994effd700 1 -- 172.21.15.26:0/412105901 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f99500ccb40 con 0x7f9958108690 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.563+0000 7f995dd9b700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 0x7f99440a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.563+0000 7f99738a5b80 1 -- 172.21.15.26:0/2501181261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 msgr2=0x55874733ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.563+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f99540094f0 tx=0x7f9954010d50 comp rx=0 tx=0).stop 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 -- 172.21.15.26:0/2501181261 shutdown_connections 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874733fc00 0x55874733ffe0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2501181261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x55874734ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 -- 172.21.15.26:0/2501181261 >> 172.21.15.26:0/2501181261 conn(0x5587472b5200 msgr2=0x55874733c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f995dd9b700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 0x7f99440a3b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f994800b6c0 tx=0x7f994800b650 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 -- 172.21.15.26:0/2501181261 shutdown_connections 2023-10-06T21:24:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 -- 172.21.15.26:0/2501181261 wait complete. 2023-10-06T21:24:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.564+0000 7f99738a5b80 1 -- start start 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 Processor -- start 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x558747348670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874734b3d0 0x558747348bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558747293dd0 con 0x55874734b3d0 2023-10-06T21:24:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587472306b0 con 0x55874733fc00 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558747230350 con 0x558747340520 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.565+0000 7f99738a5b80 10 client.0 _ll_register_callbacks cb 0x55874708d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43304/0 (socket says 172.21.15.26:43304) 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 -- 172.21.15.26:0/2736808899 learned_addr learned my addr 172.21.15.26:0/2736808899 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995e59c700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874734b3d0 0x558747348bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995d59a700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x558747348670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 -- 172.21.15.26:0/2736808899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 msgr2=0x558747348670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x558747348670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 -- 172.21.15.26:0/2736808899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874734b3d0 msgr2=0x558747348bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874734b3d0 0x558747348bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995dd9b700 1 -- 172.21.15.26:0/2736808899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9958114cd0 con 0x55874733fc00 2023-10-06T21:24:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.566+0000 7f995d59a700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x558747348670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f995dd9b700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9948010fc0 tx=0x7f994800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d8a32cc2203d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99417fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994800bdd0 con 0x55874733fc00 2023-10-06T21:24:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558747333d50 con 0x55874733fc00 2023-10-06T21:24:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f9948017070 con 0x55874733fc00 2023-10-06T21:24:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994801e620 con 0x55874733fc00 2023-10-06T21:24:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99738a5b80 10 client.27389 fetch_fsmap learned FSMap version 54 2023-10-06T21:24:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.567+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5587470a7c00 con 0x55874733fc00 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558747333d50 con 0x55874733fc00 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f994802c070 con 0x55874733fc00 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 10 client.27389 fetch_fsmap finished waiting for FSMap version 54 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 10 client.27389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 20 client.27389 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55874735af20 0x55874735d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 4 client.27389 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.570+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55874728cdb0 con 0x55874735af20 2023-10-06T21:24:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.571+0000 7f995d59a700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55874735af20 0x55874735d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.571+0000 7f995d59a700 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55874735af20 0x55874735d340 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42152803ed98a5e4 server_cookie=ac5efb9eee39d6f6 in_seq=0 out_seq=0 2023-10-06T21:24:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.571+0000 7f99417fa700 10 client.27389 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99417fa700 1 -- 172.21.15.26:0/2736808899 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55874728cdb0 con 0x55874735af20 2023-10-06T21:24:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99417fa700 10 client.27389 handle_command_reply: tid=0 2023-10-06T21:24:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99738a5b80 1 client.27389 shutdown 2023-10-06T21:24:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 msgr2=0x558747348130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9948010fc0 tx=0x7f994800b410 comp rx=0 tx=0).stop 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 shutdown_connections 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.572+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55874734b3d0 0x558747348bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55874735af20 0x55874735d340 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747348130 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 --2- 172.21.15.26:0/2736808899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558747340520 0x558747348670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 >> 172.21.15.26:0/2736808899 conn(0x5587472b5200 msgr2=0x55874733bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99417fa700 0 client.27389 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 shutdown_connections 2023-10-06T21:24:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 1 -- 172.21.15.26:0/2736808899 wait complete. 2023-10-06T21:24:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.573+0000 7f99738a5b80 20 client.27389 trim_cache size 0 max 16384 2023-10-06T21:24:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 Processor -- start 2023-10-06T21:24:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 -- start start 2023-10-06T21:24:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x5587473453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874734b3d0 0x5587473458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587471148c0 con 0x558747340520 2023-10-06T21:24:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55874733d3c0 con 0x55874733fc00 2023-10-06T21:24:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.593+0000 7f99738a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x558747343f80 con 0x55874734b3d0 2023-10-06T21:24:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43318/0 (socket says 172.21.15.26:43318) 2023-10-06T21:24:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 -- 172.21.15.26:0/771109688 learned_addr learned my addr 172.21.15.26:0/771109688 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995e59c700 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874734b3d0 0x5587473458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995d59a700 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x5587473453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 -- 172.21.15.26:0/771109688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874734b3d0 msgr2=0x5587473458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874734b3d0 0x5587473458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 -- 172.21.15.26:0/771109688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 msgr2=0x5587473453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x5587473453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 -- 172.21.15.26:0/771109688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587470e51f0 con 0x55874733fc00 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.594+0000 7f995dd9b700 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9948011010 tx=0x7f9948011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d759ae59242864c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.595+0000 7f994cff9700 1 -- 172.21.15.26:0/771109688 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480265f0 con 0x55874733fc00 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.595+0000 7f994cff9700 1 -- 172.21.15.26:0/771109688 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9948015070 con 0x55874733fc00 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.595+0000 7f994cff9700 1 -- 172.21.15.26:0/771109688 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480048f0 con 0x55874733fc00 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- 172.21.15.26:0/771109688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 msgr2=0x558747351e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9948011010 tx=0x7f9948011ac0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- 172.21.15.26:0/771109688 shutdown_connections 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874734b3d0 0x5587473458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874733fc00 0x558747351e80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 --2- 172.21.15.26:0/771109688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x5587473453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- 172.21.15.26:0/771109688 >> 172.21.15.26:0/771109688 conn(0x5587472b5200 msgr2=0x558747343470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- 172.21.15.26:0/771109688 shutdown_connections 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- 172.21.15.26:0/771109688 wait complete. 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.596+0000 7f99738a5b80 1 -- start start 2023-10-06T21:24:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 Processor -- start 2023-10-06T21:24:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x55874734e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587473ade40 con 0x558747340520 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587473adf80 con 0x55874734b3d0 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587473ae140 con 0x55874733fc00 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f99738a5b80 10 client.0 _ll_register_callbacks cb 0x55874708d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:50722/0 (socket says 172.21.15.26:50722) 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.597+0000 7f995dd9b700 1 -- 172.21.15.26:0/1189026371 learned_addr learned my addr 172.21.15.26:0/1189026371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995e59c700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995d59a700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x55874734e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 -- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 msgr2=0x55874734e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 -- 172.21.15.26:0/1189026371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 msgr2=0x55874734e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x55874734e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 -- 172.21.15.26:0/1189026371 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587470a7c00 con 0x55874733fc00 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995d59a700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x55874734e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f995dd9b700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9948026d80 tx=0x7f9948026e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61f520f2c0f7594 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f99417fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.598+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948026a30 con 0x55874733fc00 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9954010df0 con 0x55874733fc00 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f9948017070 con 0x55874733fc00 2023-10-06T21:24:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9948005450 con 0x55874733fc00 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 10 client.27235 fetch_fsmap learned FSMap version 54 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5587470bd250 con 0x55874733fc00 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9954010df0 con 0x55874733fc00 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9948011440 con 0x55874733fc00 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 10 client.27235 fetch_fsmap finished waiting for FSMap version 54 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 10 client.27235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 20 client.27235 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5587473af420 0x5587473b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 4 client.27235 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.599+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55874728cad0 con 0x5587473af420 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.600+0000 7f995d59a700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5587473af420 0x5587473b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.600+0000 7f995d59a700 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5587473af420 0x5587473b1840 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58afd0271a5781d7 server_cookie=f158666e7ffbf123 in_seq=0 out_seq=0 2023-10-06T21:24:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.600+0000 7f99417fa700 10 client.27235 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:11.633 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99417fa700 1 -- 172.21.15.26:0/1189026371 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55874728cad0 con 0x5587473af420 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99417fa700 10 client.27235 handle_command_reply: tid=0 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 client.27235 shutdown 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 msgr2=0x55874734de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9948026d80 tx=0x7f9948026e00 comp rx=0 tx=0).stop 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 shutdown_connections 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5587473af420 0x5587473b1840 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55874734b3d0 0x55874734e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55874733fc00 0x55874734de60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99417fa700 0 client.27235 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 --2- 172.21.15.26:0/1189026371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558747340520 0x55874734e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.630+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 >> 172.21.15.26:0/1189026371 conn(0x5587472b5200 msgr2=0x5587473480d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.631+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 shutdown_connections 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.631+0000 7f99738a5b80 1 -- 172.21.15.26:0/1189026371 wait complete. 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.631+0000 7f99738a5b80 20 client.27235 trim_cache size 0 max 16384 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:24:11.635 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:24:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.633+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 msgr2=0x7f99440a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.633+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 0x7f99440a3b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f994800b6c0 tx=0x7f994800b650 comp rx=0 tx=0).stop 2023-10-06T21:24:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.633+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 msgr2=0x7f99581a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.633+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9950007fa0 tx=0x7f995000d3b0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 shutdown_connections 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f995810d630 0x7f995819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f99440a1670 0x7f99440a3b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9958107cf0 0x7f99581a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 --2- 172.21.15.26:0/412105901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9958108690 0x7f99581a3b90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.634+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 >> 172.21.15.26:0/412105901 conn(0x7f9958076c70 msgr2=0x7f9958079250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.635+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 shutdown_connections 2023-10-06T21:24:11.638 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:11.635+0000 7f995ffff700 1 -- 172.21.15.26:0/412105901 wait complete. 2023-10-06T21:24:12.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:11 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:11 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:12.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:11 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:12.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 810.0 s, 1037.0 tps, lat 96.234 ms stddev 106.082 2023-10-06T21:24:13.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:12 smithi167 ceph-mon[134494]: pgmap v435: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 11 MiB/s wr, 857 op/s 2023-10-06T21:24:13.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:12 smithi026 ceph-mon[98490]: pgmap v435: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 11 MiB/s wr, 857 op/s 2023-10-06T21:24:13.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:12 smithi078 ceph-mon[134896]: pgmap v435: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 11 MiB/s wr, 857 op/s 2023-10-06T21:24:14.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:14 smithi167 ceph-mon[134494]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:24:14.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:14 smithi167 ceph-mon[134494]: pgmap v437: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-06T21:24:14.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:14 smithi026 ceph-mon[98490]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:24:14.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:14 smithi026 ceph-mon[98490]: pgmap v437: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-06T21:24:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:14 smithi078 ceph-mon[134896]: osdmap e105: 12 total, 12 up, 12 in 2023-10-06T21:24:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:14 smithi078 ceph-mon[134896]: pgmap v437: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-06T21:24:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:16 smithi167 ceph-mon[134494]: pgmap v438: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.0 MiB/s wr, 578 op/s 2023-10-06T21:24:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:16 smithi026 ceph-mon[98490]: pgmap v438: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.0 MiB/s wr, 578 op/s 2023-10-06T21:24:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:16 smithi078 ceph-mon[134896]: pgmap v438: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.0 MiB/s wr, 578 op/s 2023-10-06T21:24:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 815.0 s, 1066.3 tps, lat 93.914 ms stddev 96.827 2023-10-06T21:24:19.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:18 smithi167 ceph-mon[134494]: pgmap v439: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.5 MiB/s wr, 557 op/s 2023-10-06T21:24:19.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:18 smithi026 ceph-mon[98490]: pgmap v439: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.5 MiB/s wr, 557 op/s 2023-10-06T21:24:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:18 smithi078 ceph-mon[134896]: pgmap v439: 97 pgs: 97 active+clean; 5.0 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.5 MiB/s wr, 557 op/s 2023-10-06T21:24:21.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:20 smithi167 ceph-mon[134494]: pgmap v440: 97 pgs: 97 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.8 MiB/s wr, 601 op/s 2023-10-06T21:24:21.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:20 smithi026 ceph-mon[98490]: pgmap v440: 97 pgs: 97 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.8 MiB/s wr, 601 op/s 2023-10-06T21:24:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:20 smithi078 ceph-mon[134896]: pgmap v440: 97 pgs: 97 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.8 MiB/s wr, 601 op/s 2023-10-06T21:24:22.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:21 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:22.085 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:21 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:22.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:21 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 820.0 s, 1322.8 tps, lat 74.733 ms stddev 93.757 2023-10-06T21:24:22.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:24:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 -- 172.21.15.26:0/774165093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b301083f0 msgr2=0x7f3b30108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 --2- 172.21.15.26:0/774165093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b301083f0 0x7f3b30108870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009990 tx=0x7f3b20009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 -- 172.21.15.26:0/774165093 shutdown_connections 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 --2- 172.21.15.26:0/774165093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3011a690 0x7f3b3011ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 --2- 172.21.15.26:0/774165093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b3010f640 0x7f3b30107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 --2- 172.21.15.26:0/774165093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b301083f0 0x7f3b30108870 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.517+0000 7f3b363ff700 1 -- 172.21.15.26:0/774165093 >> 172.21.15.26:0/774165093 conn(0x7f3b3006d1a0 msgr2=0x7f3b3006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.518+0000 7f3b363ff700 1 -- 172.21.15.26:0/774165093 shutdown_connections 2023-10-06T21:24:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.518+0000 7f3b363ff700 1 -- 172.21.15.26:0/774165093 wait complete. 2023-10-06T21:24:22.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.519+0000 7f3b363ff700 1 Processor -- start 2023-10-06T21:24:22.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 -- start start 2023-10-06T21:24:22.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 0x7f3b30113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3011a690 0x7f3b301181d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30118710 con 0x7f3b3010f640 2023-10-06T21:24:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30118880 con 0x7f3b301083f0 2023-10-06T21:24:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.520+0000 7f3b363ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30118a40 con 0x7f3b3011a690 2023-10-06T21:24:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48550/0 (socket says 172.21.15.26:48550) 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 -- 172.21.15.26:0/417327005 learned_addr learned my addr 172.21.15.26:0/417327005 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b3499c700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3011a690 0x7f3b301181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 -- 172.21.15.26:0/417327005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3011a690 msgr2=0x7f3b301181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3011a690 0x7f3b301181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 0x7f3b30113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 -- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 msgr2=0x7f3b30113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 0x7f3b30113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2ffff700 1 -- 172.21.15.26:0/417327005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200095f0 con 0x7f3b301083f0 2023-10-06T21:24:22.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.521+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 0x7f3b30113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:24:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.523+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800ef10 tx=0x7f3b2800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.523+0000 7f3b2d7fa700 1 -- 172.21.15.26:0/417327005 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800cad0 con 0x7f3b301083f0 2023-10-06T21:24:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.523+0000 7f3b2d7fa700 1 -- 172.21.15.26:0/417327005 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2800f040 con 0x7f3b301083f0 2023-10-06T21:24:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.523+0000 7f3b2d7fa700 1 -- 172.21.15.26:0/417327005 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28007ea0 con 0x7f3b301083f0 2023-10-06T21:24:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.523+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b30118cc0 con 0x7f3b301083f0 2023-10-06T21:24:22.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.524+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b301bb500 con 0x7f3b301083f0 2023-10-06T21:24:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.529+0000 7f3b2d7fa700 1 -- 172.21.15.26:0/417327005 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f3b28007500 con 0x7f3b301083f0 2023-10-06T21:24:22.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.531+0000 7f3b49e1db80 1 Processor -- start 2023-10-06T21:24:22.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.531+0000 7f3b49e1db80 1 -- start start 2023-10-06T21:24:22.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46c9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46bf9460 con 0x563d46c9b890 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46beac20 con 0x563d46c900c0 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46be8540 con 0x563d46c909e0 2023-10-06T21:24:22.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2d7fa700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2d7fa700 1 -- 172.21.15.26:0/417327005 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b280cd560 con 0x7f3b301083f0 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b3499c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60754/0 (socket says 172.21.15.26:60754) 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b3499c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48562/0 (socket says 172.21.15.26:48562) 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2ffff700 1 -- 172.21.15.26:0/3523186903 learned_addr learned my addr 172.21.15.26:0/3523186903 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46c9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.532+0000 7f3b3499c700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 -- 172.21.15.26:0/3523186903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 msgr2=0x563d46c904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 -- 172.21.15.26:0/3523186903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 msgr2=0x563d46c9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46c9dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 -- 172.21.15.26:0/3523186903 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d46a0ede0 con 0x563d46c909e0 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3b280090a0 tx=0x7f3b28096940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e34e0cd0a6f5e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b3499c700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 0x7f3b180a3b70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006ea0 tx=0x7f3b24006e30 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b167fc700 1 -- 172.21.15.26:0/3523186903 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2809abd0 con 0x563d46c909e0 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b167fc700 1 -- 172.21.15.26:0/3523186903 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2800cc30 con 0x563d46c909e0 2023-10-06T21:24:22.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.533+0000 7f3b167fc700 1 -- 172.21.15.26:0/3523186903 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2809a730 con 0x563d46c909e0 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3523186903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 msgr2=0x563d46c9b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3b280090a0 tx=0x7f3b28096940 comp rx=0 tx=0).stop 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3523186903 shutdown_connections 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46c9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3523186903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46c9b350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.534+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3523186903 >> 172.21.15.26:0/3523186903 conn(0x563d46c05260 msgr2=0x563d46c8c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.535+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3523186903 shutdown_connections 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.535+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3523186903 wait complete. 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.535+0000 7f3b49e1db80 1 -- start start 2023-10-06T21:24:22.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.536+0000 7f3b49e1db80 1 Processor -- start 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.536+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.536+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c98590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c98c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46be3e30 con 0x563d46c909e0 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46b80710 con 0x563d46c900c0 2023-10-06T21:24:22.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46b803b0 con 0x563d46c9b890 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b49e1db80 10 client.0 _ll_register_callbacks cb 0x563d469dd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48568/0 (socket says 172.21.15.26:48568) 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 -- 172.21.15.26:0/941849946 learned_addr learned my addr 172.21.15.26:0/941849946 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b3499c700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c98c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c98590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 -- 172.21.15.26:0/941849946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 msgr2=0x563d46c98c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c98c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 -- 172.21.15.26:0/941849946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 msgr2=0x563d46c98590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c98590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.537+0000 7f3b2ffff700 1 -- 172.21.15.26:0/941849946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b301bb500 con 0x563d46c900c0 2023-10-06T21:24:22.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009c10 tx=0x7f3b28009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9cb1ce5a482d841 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:22.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2809abd0 con 0x563d46c900c0 2023-10-06T21:24:22.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b2809a4d0 con 0x563d46c900c0 2023-10-06T21:24:22.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d46c83f10 con 0x563d46c900c0 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.538+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b280cabb0 con 0x563d46c900c0 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.541+0000 7f3b49e1db80 10 client.27407 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.541+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x563d469f7c60 con 0x563d46c900c0 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.541+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d46c83f10 con 0x563d46c900c0 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3b280af050 con 0x563d46c900c0 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 10 client.27407 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:22.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 20 client.27407 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cab020 0x563d46cad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 4 client.27407 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d46bdce10 con 0x563d46cab020 2023-10-06T21:24:22.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.542+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cab020 0x563d46cad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.543+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cab020 0x563d46cad440 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f729152577cf3e9 server_cookie=2c0d677597d1b8f3 in_seq=0 out_seq=0 2023-10-06T21:24:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.543+0000 7f3b0f7fe700 10 client.27407 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.544+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/941849946 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d46bdce10 con 0x563d46cab020 2023-10-06T21:24:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.544+0000 7f3b0f7fe700 10 client.27407 handle_command_reply: tid=0 2023-10-06T21:24:22.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.544+0000 7f3b49e1db80 1 client.27407 shutdown 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.544+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 msgr2=0x563d46c9b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.544+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009c10 tx=0x7f3b28009eb0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 shutdown_connections 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c98c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cab020 0x563d46cad440 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9b310 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/941849946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c98590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 >> 172.21.15.26:0/941849946 conn(0x563d46c05260 msgr2=0x563d46c8bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b0f7fe700 0 client.27407 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 shutdown_connections 2023-10-06T21:24:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 1 -- 172.21.15.26:0/941849946 wait complete. 2023-10-06T21:24:22.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.545+0000 7f3b49e1db80 20 client.27407 trim_cache size 0 max 16384 2023-10-06T21:24:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.571+0000 7f3b49e1db80 1 Processor -- start 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.572+0000 7f3b49e1db80 1 -- start start 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c95820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c95d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46a64920 con 0x563d46c909e0 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46c8b240 con 0x563d46c900c0 2023-10-06T21:24:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46c94400 con 0x563d46c9b890 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48582/0 (socket says 172.21.15.26:48582) 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.573+0000 7f3b2ffff700 1 -- 172.21.15.26:0/801167564 learned_addr learned my addr 172.21.15.26:0/801167564 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b3499c700 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c95d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b2ffff700 1 -- 172.21.15.26:0/801167564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 msgr2=0x563d46c95d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c95d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b2ffff700 1 -- 172.21.15.26:0/801167564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 msgr2=0x563d46c95820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c95820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.574+0000 7f3b2ffff700 1 -- 172.21.15.26:0/801167564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d46a35250 con 0x563d46c900c0 2023-10-06T21:24:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.575+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3b28098820 tx=0x7f3b2809a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b783c3457929f48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.575+0000 7f3b16ffd700 1 -- 172.21.15.26:0/801167564 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2809a4d0 con 0x563d46c900c0 2023-10-06T21:24:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.575+0000 7f3b16ffd700 1 -- 172.21.15.26:0/801167564 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b280a2860 con 0x563d46c900c0 2023-10-06T21:24:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.575+0000 7f3b16ffd700 1 -- 172.21.15.26:0/801167564 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280a2630 con 0x563d46c900c0 2023-10-06T21:24:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.576+0000 7f3b49e1db80 1 -- 172.21.15.26:0/801167564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 msgr2=0x563d46c9aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.576+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3b28098820 tx=0x7f3b2809a090 comp rx=0 tx=0).stop 2023-10-06T21:24:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 -- 172.21.15.26:0/801167564 shutdown_connections 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c9b890 0x563d46c95d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46c9aeb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/801167564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c909e0 0x563d46c95820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 -- 172.21.15.26:0/801167564 >> 172.21.15.26:0/801167564 conn(0x563d46c05260 msgr2=0x563d46c938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 -- 172.21.15.26:0/801167564 shutdown_connections 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 -- 172.21.15.26:0/801167564 wait complete. 2023-10-06T21:24:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.577+0000 7f3b49e1db80 1 -- start start 2023-10-06T21:24:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 Processor -- start 2023-10-06T21:24:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46ca0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46ca0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46cfdef0 con 0x563d46c9b890 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46cfe060 con 0x563d46c900c0 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.578+0000 7f3b49e1db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d46cfe220 con 0x563d46c909e0 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b49e1db80 10 client.0 _ll_register_callbacks cb 0x7f3b2809a2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48590/0 (socket says 172.21.15.26:48590) 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2ffff700 1 -- 172.21.15.26:0/3233167026 learned_addr learned my addr 172.21.15.26:0/3233167026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46ca0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b3499c700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46ca0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 -- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 msgr2=0x563d46ca03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 -- 172.21.15.26:0/3233167026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 msgr2=0x563d46ca0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46ca0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2f7fe700 1 -- 172.21.15.26:0/3233167026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d469f7c60 con 0x563d46c909e0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.579+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b2f7fe700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46ca0900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3b30114e40 tx=0x7f3b20005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f579949f2c36628a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20020040 con 0x563d46c909e0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f3b200289f0 con 0x563d46c909e0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2800cc30 con 0x563d46c909e0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b20028bb0 con 0x563d46c909e0 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b49e1db80 10 client.27253 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.580+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x563d46a0d2b0 con 0x563d46c909e0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2800cc30 con 0x563d46c909e0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3b20004500 con 0x563d46c909e0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 10 client.27253 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 10 client.27253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 20 client.27253 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cffaf0 0x563d46d01f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 4 client.27253 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d46bdcb30 con 0x563d46cffaf0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.581+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cffaf0 0x563d46d01f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b2ffff700 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cffaf0 0x563d46d01f10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8301000361759b5 server_cookie=dfdf15ab5e967c33 in_seq=0 out_seq=0 2023-10-06T21:24:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b0f7fe700 10 client.27253 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b0f7fe700 1 -- 172.21.15.26:0/3233167026 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563d46bdcb30 con 0x563d46cffaf0 2023-10-06T21:24:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b0f7fe700 10 client.27253 handle_command_reply: tid=0 2023-10-06T21:24:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 client.27253 shutdown 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 msgr2=0x563d46ca0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46ca0900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3b30114e40 tx=0x7f3b20005c20 comp rx=0 tx=0).stop 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 shutdown_connections 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563d46c9b890 0x563d46ca0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563d46cffaf0 0x563d46d01f10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d46c900c0 0x563d46ca03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 --2- 172.21.15.26:0/3233167026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d46c909e0 0x563d46ca0900 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 >> 172.21.15.26:0/3233167026 conn(0x563d46c05260 msgr2=0x563d46c8b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.582+0000 7f3b0f7fe700 0 client.27253 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 shutdown_connections 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b49e1db80 1 -- 172.21.15.26:0/3233167026 wait complete. 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b49e1db80 20 client.27253 trim_cache size 0 max 16384 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:24:22.589 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:24:22.590 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 msgr2=0x7f3b180a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 0x7f3b180a3b70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006ea0 tx=0x7f3b24006e30 comp rx=0 tx=0).stop 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 msgr2=0x7f3b30113650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.583+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800ef10 tx=0x7f3b2800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 shutdown_connections 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b3011a690 0x7f3b301181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b301083f0 0x7f3b30113650 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f3b180a16b0 0x7f3b180a3b70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 --2- 172.21.15.26:0/417327005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3010f640 0x7f3b30113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 >> 172.21.15.26:0/417327005 conn(0x7f3b3006d1a0 msgr2=0x7f3b30071880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 shutdown_connections 2023-10-06T21:24:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:22.584+0000 7f3b363ff700 1 -- 172.21.15.26:0/417327005 wait complete. 2023-10-06T21:24:22.612 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:24:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:22 smithi167 ceph-mon[134494]: pgmap v441: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.6 MiB/s wr, 625 op/s 2023-10-06T21:24:23.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:22 smithi026 ceph-mon[98490]: pgmap v441: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.6 MiB/s wr, 625 op/s 2023-10-06T21:24:23.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:22 smithi078 ceph-mon[134896]: pgmap v441: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.6 MiB/s wr, 625 op/s 2023-10-06T21:24:24.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:24 smithi167 ceph-mon[134494]: pgmap v442: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-10-06T21:24:24.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:24 smithi026 ceph-mon[98490]: pgmap v442: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.866+0000 7f0a2ebff700 1 -- 172.21.15.26:0/3829235056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 msgr2=0x7f0a2810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.866+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/3829235056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a2810dd70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b0a0 tx=0x7f0a2000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 -- 172.21.15.26:0/3829235056 shutdown_connections 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/3829235056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a2810dd70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/3829235056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/3829235056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a28101a80 0x7f0a28105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 -- 172.21.15.26:0/3829235056 >> 172.21.15.26:0/3829235056 conn(0x7f0a280fc950 msgr2=0x7f0a280fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 -- 172.21.15.26:0/3829235056 shutdown_connections 2023-10-06T21:24:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.867+0000 7f0a2ebff700 1 -- 172.21.15.26:0/3829235056 wait complete. 2023-10-06T21:24:24.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:24 smithi078 ceph-mon[134896]: pgmap v442: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.3 MiB/s wr, 606 op/s 2023-10-06T21:24:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.868+0000 7f0a2ebff700 1 Processor -- start 2023-10-06T21:24:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 -- start start 2023-10-06T21:24:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 0x7f0a2810cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d560 con 0x7f0a281010d0 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d6d0 con 0x7f0a281061f0 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2ebff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d890 con 0x7f0a2810cab0 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2c99b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2c99b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36658/0 (socket says 172.21.15.26:36658) 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a2c99b700 1 -- 172.21.15.26:0/2013776082 learned_addr learned my addr 172.21.15.26:0/2013776082 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.869+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48600/0 (socket says 172.21.15.26:48600) 2023-10-06T21:24:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 0x7f0a2810cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 -- 172.21.15.26:0/2013776082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 msgr2=0x7f0a2810cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 0x7f0a2810cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 -- 172.21.15.26:0/2013776082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 msgr2=0x7f0a281a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 -- 172.21.15.26:0/2013776082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20009d00 con 0x7f0a281010d0 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 0x7f0a2810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a27fff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.870+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00e9a0 tx=0x7f0a1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.871+0000 7f0a25ffb700 1 -- 172.21.15.26:0/2013776082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c009790 con 0x7f0a281010d0 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.871+0000 7f0a25ffb700 1 -- 172.21.15.26:0/2013776082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a1c00cbd0 con 0x7f0a281010d0 2023-10-06T21:24:24.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.871+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a281aa3e0 con 0x7f0a281010d0 2023-10-06T21:24:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.871+0000 7f0a25ffb700 1 -- 172.21.15.26:0/2013776082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c018680 con 0x7f0a281010d0 2023-10-06T21:24:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.871+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a281aa910 con 0x7f0a281010d0 2023-10-06T21:24:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.877+0000 7f0a424aab80 1 Processor -- start 2023-10-06T21:24:24.881 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.878+0000 7f0a424aab80 1 -- start start 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb3dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3e520 0x5622ffb48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffaa7400 con 0x5622ffb3dc00 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffa98bc0 con 0x5622ffb493d0 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a424aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffa964e0 con 0x5622ffb3e520 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48608/0 (socket says 172.21.15.26:48608) 2023-10-06T21:24:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a27fff700 1 -- 172.21.15.26:0/2207151954 learned_addr learned my addr 172.21.15.26:0/2207151954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.879+0000 7f0a25ffb700 1 -- 172.21.15.26:0/2013776082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0a1c00cd40 con 0x7f0a281010d0 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.880+0000 7f0a27fff700 1 -- 172.21.15.26:0/2207151954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3e520 msgr2=0x5622ffb48e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.880+0000 7f0a27fff700 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3e520 0x5622ffb48e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.880+0000 7f0a27fff700 1 -- 172.21.15.26:0/2207151954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 msgr2=0x5622ffb3dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.880+0000 7f0a27fff700 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb3dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.883 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.880+0000 7f0a27fff700 1 -- 172.21.15.26:0/2207151954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622ff8bcd80 con 0x5622ffb493d0 2023-10-06T21:24:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.881+0000 7f0a27fff700 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010400 tx=0x7f0a14010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10200e281082e1c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.881+0000 7f0a12ffd700 1 -- 172.21.15.26:0/2207151954 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14005ce0 con 0x5622ffb493d0 2023-10-06T21:24:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.881+0000 7f0a12ffd700 1 -- 172.21.15.26:0/2207151954 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a14015070 con 0x5622ffb493d0 2023-10-06T21:24:24.884 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.881+0000 7f0a12ffd700 1 -- 172.21.15.26:0/2207151954 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011a00 con 0x5622ffb493d0 2023-10-06T21:24:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 -- 172.21.15.26:0/2207151954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 msgr2=0x5622ffb4b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010400 tx=0x7f0a14010710 comp rx=0 tx=0).stop 2023-10-06T21:24:24.885 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a25ffb700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 0x7f0a180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 -- 172.21.15.26:0/2207151954 shutdown_connections 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffb4b7c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb3dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 --2- 172.21.15.26:0/2207151954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3e520 0x5622ffb48e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.882+0000 7f0a424aab80 1 -- 172.21.15.26:0/2207151954 >> 172.21.15.26:0/2207151954 conn(0x5622ffab3200 msgr2=0x5622ffb3a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.883+0000 7f0a25ffb700 1 -- 172.21.15.26:0/2013776082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0a1c014070 con 0x7f0a281010d0 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.883+0000 7f0a424aab80 1 -- 172.21.15.26:0/2207151954 shutdown_connections 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.883+0000 7f0a424aab80 1 -- 172.21.15.26:0/2207151954 wait complete. 2023-10-06T21:24:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.883+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 0x7f0a180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.884+0000 7f0a424aab80 1 -- start start 2023-10-06T21:24:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.884+0000 7f0a424aab80 1 Processor -- start 2023-10-06T21:24:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 0x7f0a180a3b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009610 tx=0x7f0a2000bbd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffb46790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffa91dd0 con 0x5622ffb3e520 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffa2e6b0 con 0x5622ffb3dc00 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffa2e350 con 0x5622ffb493d0 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a424aab80 10 client.0 _ll_register_callbacks cb 0x5622ff88b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a2c99b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a2d19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffb46790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a2c99b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48618/0 (socket says 172.21.15.26:48618) 2023-10-06T21:24:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.885+0000 7f0a2c99b700 1 -- 172.21.15.26:0/3194017 learned_addr learned my addr 172.21.15.26:0/3194017 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 -- 172.21.15.26:0/3194017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 msgr2=0x5622ffb46cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb46cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 -- 172.21.15.26:0/3194017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 msgr2=0x5622ffb46790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffb46790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 -- 172.21.15.26:0/3194017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1400bc50 con 0x5622ffb3dc00 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00e970 tx=0x7f0a1c099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d494bfd59150c3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a07fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.886+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c09a2f0 con 0x5622ffb3dc00 2023-10-06T21:24:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.887+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0a1c09e440 con 0x5622ffb3dc00 2023-10-06T21:24:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.887+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622ffb31d50 con 0x5622ffb3dc00 2023-10-06T21:24:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a1c0a2680 con 0x5622ffb3dc00 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a424aab80 10 client.27425 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5622ff8a5c00 con 0x5622ffb3dc00 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a1c0b6030 con 0x5622ffb3dc00 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0a1c096170 con 0x5622ffb3dc00 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.888+0000 7f0a424aab80 10 client.27425 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a424aab80 10 client.27425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a424aab80 20 client.27425 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffb58f70 0x5622ffb5b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a424aab80 4 client.27425 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622ffa8adb0 con 0x5622ffb58f70 2023-10-06T21:24:24.892 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.889+0000 7f0a27fff700 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffb58f70 0x5622ffb5b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.890+0000 7f0a07fff700 10 client.27425 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.890+0000 7f0a27fff700 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffb58f70 0x5622ffb5b390 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab946cc80c948700 server_cookie=410008204e2c0285 in_seq=0 out_seq=0 2023-10-06T21:24:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.890+0000 7f0a07fff700 1 -- 172.21.15.26:0/3194017 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5622ffa8adb0 con 0x5622ffb58f70 2023-10-06T21:24:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.890+0000 7f0a07fff700 10 client.27425 handle_command_reply: tid=0 2023-10-06T21:24:24.893 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.890+0000 7f0a424aab80 1 client.27425 shutdown 2023-10-06T21:24:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 msgr2=0x5622ffb46250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00e970 tx=0x7f0a1c099cd0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.894 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 shutdown_connections 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb46cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffb58f70 0x5622ffb5b390 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3dc00 0x5622ffb46250 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 --2- 172.21.15.26:0/3194017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffb46790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 >> 172.21.15.26:0/3194017 conn(0x5622ffab3200 msgr2=0x5622ffb39c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a07fff700 0 client.27425 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 shutdown_connections 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.891+0000 7f0a424aab80 1 -- 172.21.15.26:0/3194017 wait complete. 2023-10-06T21:24:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.892+0000 7f0a424aab80 20 client.27425 trim_cache size 0 max 16384 2023-10-06T21:24:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.917+0000 7f0a424aab80 1 Processor -- start 2023-10-06T21:24:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 -- start start 2023-10-06T21:24:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3e520 0x5622ffb43500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ff9128c0 con 0x5622ffb3dc00 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffb47220 con 0x5622ffb3e520 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.924+0000 7f0a424aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffb31920 con 0x5622ffb493d0 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.927+0000 7f0a2d19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.927+0000 7f0a2d19c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:60822/0 (socket says 172.21.15.26:60822) 2023-10-06T21:24:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.927+0000 7f0a2d19c700 1 -- 172.21.15.26:0/1430024198 learned_addr learned my addr 172.21.15.26:0/1430024198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.933+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a27fff700 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3e520 0x5622ffb43500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a2d19c700 1 -- 172.21.15.26:0/1430024198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3e520 msgr2=0x5622ffb43500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3e520 0x5622ffb43500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a2d19c700 1 -- 172.21.15.26:0/1430024198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 msgr2=0x5622ffb4ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb4ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.936+0000 7f0a2d19c700 1 -- 172.21.15.26:0/1430024198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622ff8e31f0 con 0x5622ffb493d0 2023-10-06T21:24:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.964+0000 7f0a2d19c700 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a20015000 tx=0x7f0a2001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed62abd99c79bf01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.975+0000 7f0a137fe700 1 -- 172.21.15.26:0/1430024198 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200075a0 con 0x5622ffb493d0 2023-10-06T21:24:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.975+0000 7f0a137fe700 1 -- 172.21.15.26:0/1430024198 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a20007700 con 0x5622ffb493d0 2023-10-06T21:24:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.975+0000 7f0a137fe700 1 -- 172.21.15.26:0/1430024198 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2001b690 con 0x5622ffb493d0 2023-10-06T21:24:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.976+0000 7f0a424aab80 1 -- 172.21.15.26:0/1430024198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 msgr2=0x5622ffb43a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.976+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a20015000 tx=0x7f0a2001b000 comp rx=0 tx=0).stop 2023-10-06T21:24:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.977+0000 7f0a424aab80 1 -- 172.21.15.26:0/1430024198 shutdown_connections 2023-10-06T21:24:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.977+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb493d0 0x5622ffb43a40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.977+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3dc00 0x5622ffb4ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.977+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1430024198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb3e520 0x5622ffb43500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.977+0000 7f0a424aab80 1 -- 172.21.15.26:0/1430024198 >> 172.21.15.26:0/1430024198 conn(0x5622ffab3200 msgr2=0x5622ffb41570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.984+0000 7f0a424aab80 1 -- 172.21.15.26:0/1430024198 shutdown_connections 2023-10-06T21:24:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.985+0000 7f0a424aab80 1 -- 172.21.15.26:0/1430024198 wait complete. 2023-10-06T21:24:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.986+0000 7f0a424aab80 1 -- start start 2023-10-06T21:24:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.986+0000 7f0a424aab80 1 Processor -- start 2023-10-06T21:24:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3dc00 0x5622ffba6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffba5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffba57d0 con 0x5622ffb3e520 2023-10-06T21:24:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffba5940 con 0x5622ffb493d0 2023-10-06T21:24:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ffba5b00 con 0x5622ffb3dc00 2023-10-06T21:24:24.991 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.987+0000 7f0a424aab80 10 client.0 _ll_register_callbacks cb 0x7f0a1c09a0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.990+0000 7f0a2c99b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3dc00 0x5622ffba6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.990+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.990+0000 7f0a27fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36682/0 (socket says 172.21.15.26:36682) 2023-10-06T21:24:24.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.990+0000 7f0a27fff700 1 -- 172.21.15.26:0/1962185213 learned_addr learned my addr 172.21.15.26:0/1962185213 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.992+0000 7f0a27fff700 1 -- 172.21.15.26:0/1962185213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3dc00 msgr2=0x5622ffba6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.992+0000 7f0a27fff700 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3dc00 0x5622ffba6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.992+0000 7f0a27fff700 1 -- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 msgr2=0x5622ffba5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.992+0000 7f0a27fff700 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffba5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.992+0000 7f0a27fff700 1 -- 172.21.15.26:0/1962185213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2810e5e0 con 0x5622ffb3e520 2023-10-06T21:24:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.993+0000 7f0a27fff700 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010400 tx=0x7f0a14005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f45f6147753b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.994+0000 7f0a07fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:24:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.995+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14020040 con 0x5622ffb3e520 2023-10-06T21:24:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.995+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0a14017070 con 0x5622ffb3e520 2023-10-06T21:24:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.995+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a14015070 con 0x5622ffb3e520 2023-10-06T21:24:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.996+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a1402be30 con 0x5622ffb3e520 2023-10-06T21:24:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.996+0000 7f0a424aab80 10 client.17052 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.996+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5622ff8bb250 con 0x5622ffb3e520 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a14015070 con 0x5622ffb3e520 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0a14005370 con 0x5622ffb3e520 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 10 client.17052 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 20 client.17052 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffbbf1f0 0x5622ffbc1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 4 client.17052 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:24.997+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622ffa8aad0 con 0x5622ffbbf1f0 2023-10-06T21:24:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.005+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffbbf1f0 0x5622ffbc1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.008+0000 7f0a2c99b700 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffbbf1f0 0x5622ffbc1610 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21aacf6e2385e5a1 server_cookie=dfe908b6f8d5a010 in_seq=0 out_seq=0 2023-10-06T21:24:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.008+0000 7f0a07fff700 10 client.17052 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.046+0000 7f0a07fff700 1 -- 172.21.15.26:0/1962185213 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622ffa8aad0 con 0x5622ffbbf1f0 2023-10-06T21:24:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.046+0000 7f0a07fff700 10 client.17052 handle_command_reply: tid=0 2023-10-06T21:24:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.046+0000 7f0a424aab80 1 client.17052 shutdown 2023-10-06T21:24:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.047+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 msgr2=0x5622ffba4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.047+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0a14010400 tx=0x7f0a14005d30 comp rx=0 tx=0).stop 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 shutdown_connections 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5622ffbbf1f0 0x5622ffbc1610 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622ffb493d0 0x5622ffba5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622ffb3dc00 0x5622ffba6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 --2- 172.21.15.26:0/1962185213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ffb3e520 0x5622ffba4d50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 >> 172.21.15.26:0/1962185213 conn(0x5622ffab3200 msgr2=0x5622ffb44a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a07fff700 0 client.17052 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 shutdown_connections 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 1 -- 172.21.15.26:0/1962185213 wait complete. 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.048+0000 7f0a424aab80 20 client.17052 trim_cache size 0 max 16384 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:24:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 msgr2=0x7f0a180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 0x7f0a180a3b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009610 tx=0x7f0a2000bbd0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 msgr2=0x7f0a281a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00e9a0 tx=0x7f0a1c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 shutdown_connections 2023-10-06T21:24:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2810cab0 0x7f0a2810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0a180a1670 0x7f0a180a3b30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a281010d0 0x7f0a281a23b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 --2- 172.21.15.26:0/2013776082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281061f0 0x7f0a281a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.049+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 >> 172.21.15.26:0/2013776082 conn(0x7f0a280fc950 msgr2=0x7f0a280fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.050+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 shutdown_connections 2023-10-06T21:24:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:25.050+0000 7f0a2ebff700 1 -- 172.21.15.26:0/2013776082 wait complete. 2023-10-06T21:24:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:26 smithi167 ceph-mon[134494]: pgmap v443: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 513 op/s 2023-10-06T21:24:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:26 smithi026 ceph-mon[98490]: pgmap v443: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 513 op/s 2023-10-06T21:24:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:26 smithi078 ceph-mon[134896]: pgmap v443: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.0 MiB/s wr, 513 op/s 2023-10-06T21:24:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 825.0 s, 1096.0 tps, lat 93.032 ms stddev 103.509 2023-10-06T21:24:29.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:28 smithi167 ceph-mon[134494]: pgmap v444: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 779 op/s 2023-10-06T21:24:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:28 smithi026 ceph-mon[98490]: pgmap v444: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 779 op/s 2023-10-06T21:24:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:28 smithi078 ceph-mon[134896]: pgmap v444: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 779 op/s 2023-10-06T21:24:31.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:30 smithi167 ceph-mon[134494]: pgmap v445: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 609 op/s 2023-10-06T21:24:31.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:30 smithi026 ceph-mon[98490]: pgmap v445: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 609 op/s 2023-10-06T21:24:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:30 smithi078 ceph-mon[134896]: pgmap v445: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.2 MiB/s wr, 609 op/s 2023-10-06T21:24:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 830.0 s, 1194.0 tps, lat 82.877 ms stddev 93.750 2023-10-06T21:24:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:32 smithi167 ceph-mon[134494]: pgmap v446: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2023-10-06T21:24:33.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:32 smithi026 ceph-mon[98490]: pgmap v446: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2023-10-06T21:24:33.127 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:32 smithi078 ceph-mon[134896]: pgmap v446: 97 pgs: 97 active+clean; 5.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.3 MiB/s wr, 709 op/s 2023-10-06T21:24:34.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:34 smithi167 ceph-mon[134494]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 658 op/s 2023-10-06T21:24:34.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:34 smithi026 ceph-mon[98490]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 658 op/s 2023-10-06T21:24:34.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:34 smithi078 ceph-mon[134896]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 658 op/s 2023-10-06T21:24:37.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:36 smithi167 ceph-mon[134494]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 561 op/s 2023-10-06T21:24:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:36 smithi026 ceph-mon[98490]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 561 op/s 2023-10-06T21:24:37.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:36 smithi078 ceph-mon[134896]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.5 MiB/s wr, 561 op/s 2023-10-06T21:24:37.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 835.0 s, 889.2 tps, lat 112.642 ms stddev 151.129 2023-10-06T21:24:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3817146035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069c00 msgr2=0x7f63b810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3817146035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069c00 0x7f63b810dcd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b0d0 tx=0x7f63b000b3e0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3817146035 shutdown_connections 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3817146035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069c00 0x7f63b810dcd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3817146035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8105e60 0x7f63b8106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3817146035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8069240 0x7f63b80696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3817146035 >> 172.21.15.26:0/3817146035 conn(0x7f63b8076ea0 msgr2=0x7f63b80772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3817146035 shutdown_connections 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.292+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3817146035 wait complete. 2023-10-06T21:24:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.294+0000 7f63bf1ff700 1 Processor -- start 2023-10-06T21:24:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.294+0000 7f63bf1ff700 1 -- start start 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 0x7f63b8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 0x7f63b8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8105e60 0x7f63b819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b819e540 con 0x7f63b8069c00 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b819e6b0 con 0x7f63b8069240 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bf1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b819e870 con 0x7f63b8105e60 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bcf9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 0x7f63b8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bcf9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 0x7f63b8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54506/0 (socket says 172.21.15.26:54506) 2023-10-06T21:24:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.295+0000 7f63bcf9b700 1 -- 172.21.15.26:0/3332859031 learned_addr learned my addr 172.21.15.26:0/3332859031 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.296+0000 7f63affff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 0x7f63b8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.296+0000 7f63bd79c700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8105e60 0x7f63b819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.297+0000 7f63affff700 1 -- 172.21.15.26:0/3332859031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8105e60 msgr2=0x7f63b819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.297+0000 7f63affff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8105e60 0x7f63b819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.297+0000 7f63affff700 1 -- 172.21.15.26:0/3332859031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 msgr2=0x7f63b8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.297+0000 7f63affff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 0x7f63b8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.297+0000 7f63affff700 1 -- 172.21.15.26:0/3332859031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009520 con 0x7f63b8069c00 2023-10-06T21:24:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.298+0000 7f63affff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 0x7f63b8199890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9f0 tx=0x7f63b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.299+0000 7f63adffb700 1 -- 172.21.15.26:0/3332859031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b400cb70 con 0x7f63b8069c00 2023-10-06T21:24:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.299+0000 7f63adffb700 1 -- 172.21.15.26:0/3332859031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b400ee90 con 0x7f63b8069c00 2023-10-06T21:24:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.299+0000 7f63adffb700 1 -- 172.21.15.26:0/3332859031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4003cf0 con 0x7f63b8069c00 2023-10-06T21:24:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.299+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b0009d00 con 0x7f63b8069c00 2023-10-06T21:24:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.299+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b8107c60 con 0x7f63b8069c00 2023-10-06T21:24:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.305+0000 7f63adffb700 1 -- 172.21.15.26:0/3332859031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63b400ccd0 con 0x7f63b8069c00 2023-10-06T21:24:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.305+0000 7f63d2b0bb80 1 Processor -- start 2023-10-06T21:24:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 -- start start 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc93b400 con 0x55bfbc9d1c00 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc92cbc0 con 0x55bfbc9dd3d0 2023-10-06T21:24:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc92a4e0 con 0x55bfbc9d2520 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63bcf9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.306+0000 7f63bd79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:42778/0 (socket says 172.21.15.26:42778) 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 -- 172.21.15.26:0/4186235306 learned_addr learned my addr 172.21.15.26:0/4186235306 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54522/0 (socket says 172.21.15.26:54522) 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 -- 172.21.15.26:0/4186235306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 msgr2=0x55bfbc9df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 -- 172.21.15.26:0/4186235306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 msgr2=0x55bfbc9d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 -- 172.21.15.26:0/4186235306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfbc750d80 con 0x55bfbc9d2520 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63adffb700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 0x7f63a00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.307+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009e50 tx=0x7f63a8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46510799ca510d98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.308+0000 7f639effd700 1 -- 172.21.15.26:0/4186235306 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8004a30 con 0x55bfbc9d2520 2023-10-06T21:24:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.308+0000 7f63adffb700 1 -- 172.21.15.26:0/3332859031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63b4014070 con 0x7f63b8069c00 2023-10-06T21:24:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.308+0000 7f639effd700 1 -- 172.21.15.26:0/4186235306 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a80054e0 con 0x55bfbc9d2520 2023-10-06T21:24:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.308+0000 7f63bd79c700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 0x7f63a00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.308+0000 7f639effd700 1 -- 172.21.15.26:0/4186235306 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8019930 con 0x55bfbc9d2520 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/4186235306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 msgr2=0x55bfbc9dce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009e50 tx=0x7f63a8011720 comp rx=0 tx=0).stop 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63bd79c700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 0x7f63a00a3c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b770 tx=0x7f63b000b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/4186235306 shutdown_connections 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/4186235306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d2520 0x55bfbc9dce90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/4186235306 >> 172.21.15.26:0/4186235306 conn(0x55bfbc947200 msgr2=0x55bfbc9ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/4186235306 shutdown_connections 2023-10-06T21:24:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/4186235306 wait complete. 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.309+0000 7f63d2b0bb80 1 -- start start 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 Processor -- start 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc925dd0 con 0x55bfbc9d1c00 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc8c26b0 con 0x55bfbc9d2520 2023-10-06T21:24:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc8c2350 con 0x55bfbc9dd3d0 2023-10-06T21:24:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.310+0000 7f63d2b0bb80 10 client.0 _ll_register_callbacks cb 0x55bfbc71f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.312+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.312+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54524/0 (socket says 172.21.15.26:54524) 2023-10-06T21:24:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.312+0000 7f63affff700 1 -- 172.21.15.26:0/2473755630 learned_addr learned my addr 172.21.15.26:0/2473755630 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63bd79c700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63affff700 1 -- 172.21.15.26:0/2473755630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 msgr2=0x55bfbc9daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63affff700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63affff700 1 -- 172.21.15.26:0/2473755630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 msgr2=0x55bfbc9da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63affff700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.313+0000 7f63affff700 1 -- 172.21.15.26:0/2473755630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a8009a80 con 0x55bfbc9d2520 2023-10-06T21:24:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.314+0000 7f63affff700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f63b400c5b0 tx=0x7f63b4095ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4b6da404ec0287 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.314+0000 7f6397fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.314+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40cf2f0 con 0x55bfbc9d2520 2023-10-06T21:24:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.315+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63b409a8b0 con 0x55bfbc9d2520 2023-10-06T21:24:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.315+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfbc9c5d50 con 0x55bfbc9d2520 2023-10-06T21:24:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.315+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b409e130 con 0x55bfbc9d2520 2023-10-06T21:24:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.315+0000 7f63d2b0bb80 10 client.27440 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.315+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55bfbc739c00 con 0x55bfbc9d2520 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfbc9c5d50 con 0x55bfbc9d2520 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63b409a410 con 0x55bfbc9d2520 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 10 client.27440 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 10 client.27440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 20 client.27440 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbc9ecff0 0x55bfbc9ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 4 client.27440 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfbc91edb0 con 0x55bfbc9ecff0 2023-10-06T21:24:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.316+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbc9ecff0 0x55bfbc9ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.317+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbc9ecff0 0x55bfbc9ef410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c116194b1f58073a server_cookie=5facb711099e973 in_seq=0 out_seq=0 2023-10-06T21:24:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.317+0000 7f6397fff700 10 client.27440 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f6397fff700 1 -- 172.21.15.26:0/2473755630 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bfbc91edb0 con 0x55bfbc9ecff0 2023-10-06T21:24:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f6397fff700 10 client.27440 handle_command_reply: tid=0 2023-10-06T21:24:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 client.27440 shutdown 2023-10-06T21:24:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 msgr2=0x55bfbc9da660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f63b400c5b0 tx=0x7f63b4095ef0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 shutdown_connections 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbc9ecff0 0x55bfbc9ef410 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/2473755630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9da660 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 >> 172.21.15.26:0/2473755630 conn(0x55bfbc947200 msgr2=0x55bfbc9cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.318+0000 7f6397fff700 0 client.27440 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.319+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 shutdown_connections 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.319+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/2473755630 wait complete. 2023-10-06T21:24:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.319+0000 7f63d2b0bb80 20 client.27440 trim_cache size 0 max 16384 2023-10-06T21:24:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.346+0000 7f63d2b0bb80 1 Processor -- start 2023-10-06T21:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.346+0000 7f63d2b0bb80 1 -- start start 2023-10-06T21:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc7a68c0 con 0x55bfbc9dd3d0 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc9ca880 con 0x55bfbc9d2520 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbc9cf640 con 0x55bfbc9d1c00 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54534/0 (socket says 172.21.15.26:54534) 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 -- 172.21.15.26:0/3229488073 learned_addr learned my addr 172.21.15.26:0/3229488073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63bd79c700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 -- 172.21.15.26:0/3229488073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d1c00 msgr2=0x55bfbc9d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 -- 172.21.15.26:0/3229488073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 msgr2=0x55bfbc9ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63affff700 1 -- 172.21.15.26:0/3229488073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfbc7771f0 con 0x55bfbc9d2520 2023-10-06T21:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.347+0000 7f63bd79c700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:24:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.348+0000 7f63affff700 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f63b409a770 tx=0x7f63b4096620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9859778dd51562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.348+0000 7f639f7fe700 1 -- 172.21.15.26:0/3229488073 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b409e130 con 0x55bfbc9d2520 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.348+0000 7f639f7fe700 1 -- 172.21.15.26:0/3229488073 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b4099b10 con 0x55bfbc9d2520 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.348+0000 7f639f7fe700 1 -- 172.21.15.26:0/3229488073 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4099740 con 0x55bfbc9d2520 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/3229488073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 msgr2=0x55bfbc9cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f63b409a770 tx=0x7f63b4096620 comp rx=0 tx=0).stop 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/3229488073 shutdown_connections 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9dd3d0 0x55bfbc9ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/3229488073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbc9cdbb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/3229488073 >> 172.21.15.26:0/3229488073 conn(0x55bfbc947200 msgr2=0x55bfbc9d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/3229488073 shutdown_connections 2023-10-06T21:24:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.349+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/3229488073 wait complete. 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.350+0000 7f63d2b0bb80 1 -- start start 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.350+0000 7f63d2b0bb80 1 Processor -- start 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbca38dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbca3cb50 con 0x55bfbc9d1c00 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbca39310 con 0x55bfbc9d2520 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfbca394d0 con 0x55bfbc9dd3d0 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63d2b0bb80 10 client.0 _ll_register_callbacks cb 0x7f63b4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54548/0 (socket says 172.21.15.26:54548) 2023-10-06T21:24:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 -- 172.21.15.26:0/1363533442 learned_addr learned my addr 172.21.15.26:0/1363533442 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63bd79c700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbca38dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 -- 172.21.15.26:0/1363533442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 msgr2=0x55bfbca38dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbca38dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 -- 172.21.15.26:0/1363533442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 msgr2=0x55bfbc9e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.351+0000 7f63affff700 1 -- 172.21.15.26:0/1363533442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfbc739c00 con 0x55bfbc9d2520 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f63affff700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9c0 tx=0x7f63b40d0890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5dd09552daf418c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f6397fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b40a22b0 con 0x55bfbc9d2520 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63b409e300 con 0x55bfbc9d2520 2023-10-06T21:24:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a80054e0 con 0x55bfbc9d2520 2023-10-06T21:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b4099950 con 0x55bfbc9d2520 2023-10-06T21:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f63d2b0bb80 10 client.27446 fetch_fsmap learned FSMap version 55 2023-10-06T21:24:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.352+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55bfbc74f250 con 0x55bfbc9d2520 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a80054e0 con 0x55bfbc9d2520 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 55) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f63b40961a0 con 0x55bfbc9d2520 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 10 client.27446 fetch_fsmap finished waiting for FSMap version 55 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 10 client.27446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 20 client.27446 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbca52620 0x55bfbca54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 4 client.27446 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bfbc91ead0 con 0x55bfbca52620 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.353+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbca52620 0x55bfbca54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.354+0000 7f63bcf9b700 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbca52620 0x55bfbca54a10 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf91316f4c601ac server_cookie=94ede267c7a10e5f in_seq=0 out_seq=0 2023-10-06T21:24:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.354+0000 7f6397fff700 10 client.27446 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f6397fff700 1 -- 172.21.15.26:0/1363533442 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bfbc91ead0 con 0x55bfbca52620 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f6397fff700 10 client.27446 handle_command_reply: tid=0 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 client.27446 shutdown 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 msgr2=0x55bfbca3c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9c0 tx=0x7f63b40d0890 comp rx=0 tx=0).stop 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 shutdown_connections 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfbc9dd3d0 0x55bfbca38dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfbc9d1c00 0x55bfbc9e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bfbca52620 0x55bfbca54a10 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 --2- 172.21.15.26:0/1363533442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bfbc9d2520 0x55bfbca3c450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 >> 172.21.15.26:0/1363533442 conn(0x55bfbc947200 msgr2=0x55bfbc9d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f6397fff700 0 client.27446 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 shutdown_connections 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.369+0000 7f63d2b0bb80 1 -- 172.21.15.26:0/1363533442 wait complete. 2023-10-06T21:24:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.370+0000 7f63d2b0bb80 20 client.27446 trim_cache size 0 max 16384 2023-10-06T21:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:24:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.393+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 msgr2=0x7f63a00a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.393+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 0x7f63a00a3c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b770 tx=0x7f63b000b040 comp rx=0 tx=0).stop 2023-10-06T21:24:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.393+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 msgr2=0x7f63b8199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.393+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 0x7f63b8199890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9f0 tx=0x7f63b400ed00 comp rx=0 tx=0).stop 2023-10-06T21:24:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 shutdown_connections 2023-10-06T21:24:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63b8105e60 0x7f63b819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b8069240 0x7f63b8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63a00a1740 0x7f63a00a3c00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 --2- 172.21.15.26:0/3332859031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8069c00 0x7f63b8199890 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.411+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 >> 172.21.15.26:0/3332859031 conn(0x7f63b8076ea0 msgr2=0x7f63b80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.435+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 shutdown_connections 2023-10-06T21:24:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:38.437+0000 7f63bf1ff700 1 -- 172.21.15.26:0/3332859031 wait complete. 2023-10-06T21:24:39.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:38 smithi167 ceph-mon[134494]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2023-10-06T21:24:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:38 smithi026 ceph-mon[98490]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2023-10-06T21:24:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:38 smithi078 ceph-mon[134896]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2023-10-06T21:24:41.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:40 smithi167 ceph-mon[134494]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2023-10-06T21:24:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:40 smithi026 ceph-mon[98490]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2023-10-06T21:24:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:40 smithi078 ceph-mon[134896]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2023-10-06T21:24:42.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 840.0 s, 1180.4 tps, lat 82.595 ms stddev 96.797 2023-10-06T21:24:43.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:42 smithi167 ceph-mon[134494]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:24:43.079 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:42 smithi026 ceph-mon[98490]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:24:43.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:42 smithi078 ceph-mon[134896]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 1.48k op/s 2023-10-06T21:24:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:44 smithi167 ceph-mon[134494]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:24:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:44 smithi026 ceph-mon[98490]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:24:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:44 smithi078 ceph-mon[134896]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2023-10-06T21:24:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:46 smithi167 ceph-mon[134494]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:24:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:46 smithi026 ceph-mon[98490]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:24:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:46 smithi078 ceph-mon[134896]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:24:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 845.0 s, 1150.8 tps, lat 89.566 ms stddev 106.737 2023-10-06T21:24:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:48 smithi167 ceph-mon[134494]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-10-06T21:24:49.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:48 smithi026 ceph-mon[98490]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-10-06T21:24:49.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:48 smithi078 ceph-mon[134896]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 18 MiB/s wr, 1.54k op/s 2023-10-06T21:24:50.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:49 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:50.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:49 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:50.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:49 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:24:51.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:50 smithi167 ceph-mon[134494]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 714 op/s 2023-10-06T21:24:51.080 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:50 smithi026 ceph-mon[98490]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 714 op/s 2023-10-06T21:24:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:50 smithi078 ceph-mon[134896]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 714 op/s 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3080134075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38107840 msgr2=0x7f0d38107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3080134075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38107840 0x7f0d38107c20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d340099c0 tx=0x7f0d34009cd0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3080134075 shutdown_connections 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3080134075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3810c880 0x7f0d38114400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3080134075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38107840 0x7f0d38107c20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3080134075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d381081f0 0x7f0d3810c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3080134075 >> 172.21.15.26:0/3080134075 conn(0x7f0d38076d50 msgr2=0x7f0d38105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3080134075 shutdown_connections 2023-10-06T21:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.688+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3080134075 wait complete. 2023-10-06T21:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.690+0000 7f0d3fdff700 1 Processor -- start 2023-10-06T21:24:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.690+0000 7f0d3fdff700 1 -- start start 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3810c880 0x7f0d381a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3819d3c0 0x7f0d3819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3819de40 con 0x7f0d3819d3c0 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3819df80 con 0x7f0d3810c880 2023-10-06T21:24:51.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3fdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3819e140 con 0x7f0d38107840 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55272/0 (socket says 172.21.15.26:55272) 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.691+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3983664607 learned_addr learned my addr 172.21.15.26:0/3983664607 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3e39c700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3819d3c0 0x7f0d3819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3983664607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3810c880 msgr2=0x7f0d381a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3810c880 0x7f0d381a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3819d3c0 msgr2=0x7f0d3819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3819d3c0 0x7f0d3819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3983664607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30009c40 con 0x7f0d38107840 2023-10-06T21:24:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.692+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400b580 tx=0x7f0d3400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.694+0000 7f0d2effd700 1 -- 172.21.15.26:0/3983664607 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3401d070 con 0x7f0d38107840 2023-10-06T21:24:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.694+0000 7f0d2effd700 1 -- 172.21.15.26:0/3983664607 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d34005510 con 0x7f0d38107840 2023-10-06T21:24:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.694+0000 7f0d2effd700 1 -- 172.21.15.26:0/3983664607 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34004bd0 con 0x7f0d38107840 2023-10-06T21:24:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.694+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340095f0 con 0x7f0d38107840 2023-10-06T21:24:51.697 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.694+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d381aa670 con 0x7f0d38107840 2023-10-06T21:24:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.702+0000 7f0d53658b80 1 Processor -- start 2023-10-06T21:24:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.703+0000 7f0d2effd700 1 -- 172.21.15.26:0/3983664607 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0d34004d30 con 0x7f0d38107840 2023-10-06T21:24:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.703+0000 7f0d53658b80 1 -- start start 2023-10-06T21:24:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb7c00 0x556095cb7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cc2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095c21400 con 0x556095cb8520 2023-10-06T21:24:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095c12bc0 con 0x556095cc33d0 2023-10-06T21:24:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.704+0000 7f0d53658b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095c104e0 con 0x556095cb7c00 2023-10-06T21:24:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3e39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb7c00 0x556095cb7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53682/0 (socket says 172.21.15.26:53682) 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 -- 172.21.15.26:0/3788036341 learned_addr learned my addr 172.21.15.26:0/3788036341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cc2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 -- 172.21.15.26:0/3788036341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb7c00 msgr2=0x556095cb7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb7c00 0x556095cb7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.712 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 -- 172.21.15.26:0/3788036341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 msgr2=0x556095cc2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cc2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.705+0000 7f0d3d39a700 1 -- 172.21.15.26:0/3788036341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556095a36d80 con 0x556095cc33d0 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.706+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800aac0 tx=0x7f0d2800add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdc87d107433b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.706+0000 7f0d23fff700 1 -- 172.21.15.26:0/3788036341 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800cc10 con 0x556095cc33d0 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.706+0000 7f0d23fff700 1 -- 172.21.15.26:0/3788036341 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d2800cd70 con 0x556095cc33d0 2023-10-06T21:24:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.706+0000 7f0d23fff700 1 -- 172.21.15.26:0/3788036341 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28018730 con 0x556095cc33d0 2023-10-06T21:24:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.706+0000 7f0d2effd700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 0x7f0d240a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d2effd700 1 -- 172.21.15.26:0/3983664607 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0d340cdf00 con 0x7f0d38107840 2023-10-06T21:24:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d3e39c700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 0x7f0d240a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 -- 172.21.15.26:0/3788036341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 msgr2=0x556095cc57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800aac0 tx=0x7f0d2800add0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 -- 172.21.15.26:0/3788036341 shutdown_connections 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cc33d0 0x556095cc57c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb7c00 0x556095cb7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3788036341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cc2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 -- 172.21.15.26:0/3788036341 >> 172.21.15.26:0/3788036341 conn(0x556095c2d200 msgr2=0x556095cb4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 -- 172.21.15.26:0/3788036341 shutdown_connections 2023-10-06T21:24:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.707+0000 7f0d53658b80 1 -- 172.21.15.26:0/3788036341 wait complete. 2023-10-06T21:24:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.708+0000 7f0d3e39c700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 0x7f0d240a3b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fd0 tx=0x7f0d3000b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.708+0000 7f0d53658b80 1 -- start start 2023-10-06T21:24:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.708+0000 7f0d53658b80 1 Processor -- start 2023-10-06T21:24:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.718 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb8520 0x556095cc0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cc33d0 0x556095cc0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095c0bdd0 con 0x556095cc33d0 2023-10-06T21:24:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095ba86b0 con 0x556095cb7c00 2023-10-06T21:24:51.719 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095ba8350 con 0x556095cb8520 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.709+0000 7f0d53658b80 10 client.0 _ll_register_callbacks cb 0x556095a057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3e39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cc33d0 0x556095cc0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53696/0 (socket says 172.21.15.26:53696) 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3470636828 learned_addr learned my addr 172.21.15.26:0/3470636828 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb8520 0x556095cc0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3470636828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb8520 msgr2=0x556095cc0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb8520 0x556095cc0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.710+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3470636828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cc33d0 msgr2=0x556095cc0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cc33d0 0x556095cc0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d3db9b700 1 -- 172.21.15.26:0/3470636828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2800a6f0 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005b40 tx=0x7f0d3400fe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67605edd3268352a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d217fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340d1220 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0d34005680 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556095cabd50 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d3409b600 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d53658b80 10 client.27455 fetch_fsmap learned FSMap version 56 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.711+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556095a1fc00 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556095cabd50 con 0x556095cb7c00 2023-10-06T21:24:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0d340cf070 con 0x556095cb7c00 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 10 client.27455 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 10 client.27455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 20 client.27455 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095cd2c20 0x556095cd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 4 client.27455 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556095c04db0 con 0x556095cd2c20 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.712+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095cd2c20 0x556095cd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.713+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095cd2c20 0x556095cd5040 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6a614fe48d692f server_cookie=ed25c61f615128ff in_seq=0 out_seq=0 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.713+0000 7f0d217fa700 10 client.27455 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.713+0000 7f0d217fa700 1 -- 172.21.15.26:0/3470636828 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556095c04db0 con 0x556095cd2c20 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.713+0000 7f0d217fa700 10 client.27455 handle_command_reply: tid=0 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 client.27455 shutdown 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 msgr2=0x556095cc2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005b40 tx=0x7f0d3400fe60 comp rx=0 tx=0).stop 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 shutdown_connections 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cc33d0 0x556095cc0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095cd2c20 0x556095cd5040 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cc2e90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 --2- 172.21.15.26:0/3470636828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cb8520 0x556095cc0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d217fa700 0 client.27455 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 >> 172.21.15.26:0/3470636828 conn(0x556095c2d200 msgr2=0x556095cb3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 shutdown_connections 2023-10-06T21:24:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.714+0000 7f0d53658b80 1 -- 172.21.15.26:0/3470636828 wait complete. 2023-10-06T21:24:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.715+0000 7f0d53658b80 20 client.27455 trim_cache size 0 max 16384 2023-10-06T21:24:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.743+0000 7f0d53658b80 1 Processor -- start 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 -- start start 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095cc29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095a8c8c0 con 0x556095cb8520 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095cc1680 con 0x556095cb7c00 2023-10-06T21:24:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d53658b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095cb0860 con 0x556095cc33d0 2023-10-06T21:24:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d3db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53700/0 (socket says 172.21.15.26:53700) 2023-10-06T21:24:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.744+0000 7f0d3db9b700 1 -- 172.21.15.26:0/1284293027 learned_addr learned my addr 172.21.15.26:0/1284293027 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3e39c700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095cc29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 -- 172.21.15.26:0/1284293027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 msgr2=0x556095cc29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095cc29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 -- 172.21.15.26:0/1284293027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 msgr2=0x556095cb27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cb27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 -- 172.21.15.26:0/1284293027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556095a5d1f0 con 0x556095cb7c00 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3e39c700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095cc29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cb27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:24:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0d3409ac00 tx=0x7f0d340ce160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e0f377a19ca29cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d2cbff700 1 -- 172.21.15.26:0/1284293027 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3409a640 con 0x556095cb7c00 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d2cbff700 1 -- 172.21.15.26:0/1284293027 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d34005680 con 0x556095cb7c00 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.745+0000 7f0d2cbff700 1 -- 172.21.15.26:0/1284293027 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340a2d70 con 0x556095cb7c00 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.746+0000 7f0d53658b80 1 -- 172.21.15.26:0/1284293027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 msgr2=0x556095cbabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.746+0000 7f0d53658b80 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0d3409ac00 tx=0x7f0d340ce160 comp rx=0 tx=0).stop 2023-10-06T21:24:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.747+0000 7f0d53658b80 1 -- 172.21.15.26:0/1284293027 shutdown_connections 2023-10-06T21:24:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.747+0000 7f0d53658b80 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095cc29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.747+0000 7f0d53658b80 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb7c00 0x556095cbabe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.747+0000 7f0d53658b80 1 --2- 172.21.15.26:0/1284293027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb8520 0x556095cb27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.747+0000 7f0d53658b80 1 -- 172.21.15.26:0/1284293027 >> 172.21.15.26:0/1284293027 conn(0x556095c2d200 msgr2=0x556095cd99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.749+0000 7f0d53658b80 1 -- 172.21.15.26:0/1284293027 shutdown_connections 2023-10-06T21:24:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.749+0000 7f0d53658b80 1 -- 172.21.15.26:0/1284293027 wait complete. 2023-10-06T21:24:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.750+0000 7f0d53658b80 1 -- start start 2023-10-06T21:24:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.750+0000 7f0d53658b80 1 Processor -- start 2023-10-06T21:24:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb7c00 0x556095cc8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 0x556095cc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095d36a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095d36fa0 con 0x556095cb7c00 2023-10-06T21:24:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095d370e0 con 0x556095cb8520 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556095d37220 con 0x556095cc33d0 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.751+0000 7f0d53658b80 10 client.0 _ll_register_callbacks cb 0x556095a057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3e39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095d36a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 0x556095cc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3e39c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095d36a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55322/0 (socket says 172.21.15.26:55322) 2023-10-06T21:24:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3e39c700 1 -- 172.21.15.26:0/2464261033 learned_addr learned my addr 172.21.15.26:0/2464261033 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb7c00 0x556095cc8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 -- 172.21.15.26:0/2464261033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 msgr2=0x556095d36a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095d36a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 -- 172.21.15.26:0/2464261033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb7c00 msgr2=0x556095cc8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb7c00 0x556095cc8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.752+0000 7f0d3d39a700 1 -- 172.21.15.26:0/2464261033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556095a1fc00 con 0x556095cb8520 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d3d39a700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 0x556095cc8660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000c00 tx=0x7f0d2800fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34b620c66fa88cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d217fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28028c80 con 0x556095cb8520 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0d28028de0 con 0x556095cb8520 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d2800cd70 con 0x556095cb8520 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d280280f0 con 0x556095cb8520 2023-10-06T21:24:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d53658b80 10 client.27461 fetch_fsmap learned FSMap version 56 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.753+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556095a35250 con 0x556095cb8520 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d2800cd70 con 0x556095cb8520 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0d2800aaf0 con 0x556095cb8520 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 10 client.27461 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 10 client.27461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 20 client.27461 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095d38930 0x556095d3ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 4 client.27461 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556095c04ad0 con 0x556095d38930 2023-10-06T21:24:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.754+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095d38930 0x556095d3ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.755+0000 7f0d3db9b700 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095d38930 0x556095d3ad50 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe72cce24c31f2b server_cookie=9dc9723331adcac8 in_seq=0 out_seq=0 2023-10-06T21:24:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.755+0000 7f0d217fa700 10 client.27461 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.766+0000 7f0d217fa700 1 -- 172.21.15.26:0/2464261033 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556095c04ad0 con 0x556095d38930 2023-10-06T21:24:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.766+0000 7f0d217fa700 10 client.27461 handle_command_reply: tid=0 2023-10-06T21:24:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.766+0000 7f0d53658b80 1 client.27461 shutdown 2023-10-06T21:24:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 msgr2=0x556095cc8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 0x556095cc8660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000c00 tx=0x7f0d2800fec0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 shutdown_connections 2023-10-06T21:24:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556095cc33d0 0x556095d36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556095cb7c00 0x556095cc8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556095d38930 0x556095d3ad50 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 --2- 172.21.15.26:0/2464261033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556095cb8520 0x556095cc8660 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 >> 172.21.15.26:0/2464261033 conn(0x556095c2d200 msgr2=0x556095cce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d217fa700 0 client.27461 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 shutdown_connections 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.767+0000 7f0d53658b80 1 -- 172.21.15.26:0/2464261033 wait complete. 2023-10-06T21:24:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d53658b80 20 client.27461 trim_cache size 0 max 16384 2023-10-06T21:24:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:24:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 msgr2=0x7f0d240a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 0x7f0d240a3b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fd0 tx=0x7f0d3000b040 comp rx=0 tx=0).stop 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 msgr2=0x7f0d381a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400b580 tx=0x7f0d3400f690 comp rx=0 tx=0).stop 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 shutdown_connections 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0d240a1670 0x7f0d240a3b30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d3819d3c0 0x7f0d3819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d38107840 0x7f0d381a3560 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 --2- 172.21.15.26:0/3983664607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d3810c880 0x7f0d381a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.768+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 >> 172.21.15.26:0/3983664607 conn(0x7f0d38076d50 msgr2=0x7f0d38077150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.769+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 shutdown_connections 2023-10-06T21:24:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:24:51.769+0000 7f0d3fdff700 1 -- 172.21.15.26:0/3983664607 wait complete. 2023-10-06T21:24:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 850.0 s, 1289.6 tps, lat 76.505 ms stddev 86.609 2023-10-06T21:24:52.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:24:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 -- 172.21.15.26:0/1277202764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc470102050 msgr2=0x7fc470102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 --2- 172.21.15.26:0/1277202764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc470102050 0x7fc470102430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009990 tx=0x7fc46c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 -- 172.21.15.26:0/1277202764 shutdown_connections 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 --2- 172.21.15.26:0/1277202764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc47010b420 0x7fc47010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 --2- 172.21.15.26:0/1277202764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc470102050 0x7fc470102430 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 --2- 172.21.15.26:0/1277202764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 0x7fc47010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.959+0000 7fc477dff700 1 -- 172.21.15.26:0/1277202764 >> 172.21.15.26:0/1277202764 conn(0x7fc4700fb830 msgr2=0x7fc4700fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.960+0000 7fc477dff700 1 -- 172.21.15.26:0/1277202764 shutdown_connections 2023-10-06T21:24:52.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.960+0000 7fc477dff700 1 -- 172.21.15.26:0/1277202764 wait complete. 2023-10-06T21:24:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.961+0000 7fc477dff700 1 Processor -- start 2023-10-06T21:24:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.964+0000 7fc477dff700 1 -- start start 2023-10-06T21:24:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.964+0000 7fc477dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.964+0000 7fc477dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 0x7fc470197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.964+0000 7fc477dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47010b420 0x7fc47019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc477dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c000 con 0x7fc470102970 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc477dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c170 con 0x7fc47010b420 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc477dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c330 con 0x7fc470102050 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55336/0 (socket says 172.21.15.26:55336) 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 -- 172.21.15.26:0/3675138519 learned_addr learned my addr 172.21.15.26:0/3675138519 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc47639c700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47010b420 0x7fc47019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc47539a700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 0x7fc470197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 -- 172.21.15.26:0/3675138519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47010b420 msgr2=0x7fc47019b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47010b420 0x7fc47019b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 -- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 msgr2=0x7fc470197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 0x7fc470197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 -- 172.21.15.26:0/3675138519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0095f0 con 0x7fc470102050 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.965+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005910 tx=0x7fc46c004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.966+0000 7fc466ffd700 1 -- 172.21.15.26:0/3675138519 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c01d070 con 0x7fc470102050 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.966+0000 7fc466ffd700 1 -- 172.21.15.26:0/3675138519 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46c022470 con 0x7fc470102050 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.966+0000 7fc466ffd700 1 -- 172.21.15.26:0/3675138519 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c017650 con 0x7fc470102050 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.966+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47019c5b0 con 0x7fc470102050 2023-10-06T21:24:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.966+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47019c830 con 0x7fc470102050 2023-10-06T21:24:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.971+0000 7fc466ffd700 1 -- 172.21.15.26:0/3675138519 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fc46c0225e0 con 0x7fc470102050 2023-10-06T21:24:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.972+0000 7fc48b805b80 1 Processor -- start 2023-10-06T21:24:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 -- start start 2023-10-06T21:24:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cdc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9c33460 con 0x55c8a9cca9e0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9c24c20 con 0x55c8a9cca0c0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc48b805b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9c22540 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc47639c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc47639c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53732/0 (socket says 172.21.15.26:53732) 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc47639c700 1 -- 172.21.15.26:0/3393653509 learned_addr learned my addr 172.21.15.26:0/3393653509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc47539a700 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cdc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.973+0000 7fc466ffd700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 0x7fc45c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 -- 172.21.15.26:0/3393653509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 msgr2=0x55c8a9cca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 -- 172.21.15.26:0/3393653509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 msgr2=0x55c8a9cd9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 -- 172.21.15.26:0/3393653509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a9a48de0 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc466ffd700 1 -- 172.21.15.26:0/3675138519 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc46c0ce220 con 0x7fc470102050 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47639c700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 0x7fc45c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc47539a700 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cdc090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc460009bd0 tx=0x7fc460009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a3e0e66577b154 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.974+0000 7fc45bfff700 1 -- 172.21.15.26:0/3393653509 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460011b90 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.975+0000 7fc45bfff700 1 -- 172.21.15.26:0/3393653509 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc460011cf0 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.975+0000 7fc45bfff700 1 -- 172.21.15.26:0/3393653509 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000f6a0 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.976+0000 7fc48b805b80 1 -- 172.21.15.26:0/3393653509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 msgr2=0x55c8a9cdc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.976+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cdc090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc460009bd0 tx=0x7fc460009ee0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc47639c700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 0x7fc45c0a3b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc468017fd0 tx=0x7fc468019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc48b805b80 1 -- 172.21.15.26:0/3393653509 shutdown_connections 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cdc090 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3393653509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.977+0000 7fc48b805b80 1 -- 172.21.15.26:0/3393653509 >> 172.21.15.26:0/3393653509 conn(0x55c8a9c3f260 msgr2=0x55c8a9cc6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.978+0000 7fc48b805b80 1 -- 172.21.15.26:0/3393653509 shutdown_connections 2023-10-06T21:24:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.978+0000 7fc48b805b80 1 -- 172.21.15.26:0/3393653509 wait complete. 2023-10-06T21:24:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.978+0000 7fc48b805b80 1 -- start start 2023-10-06T21:24:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 Processor -- start 2023-10-06T21:24:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9c1de30 con 0x55c8a9cd9ca0 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9bba710 con 0x55c8a9cca0c0 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.979+0000 7fc48b805b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9bba3b0 con 0x55c8a9cca9e0 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc48b805b80 10 client.0 _ll_register_callbacks cb 0x55c8a9a17800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53744/0 (socket says 172.21.15.26:53744) 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 -- 172.21.15.26:0/2284554966 learned_addr learned my addr 172.21.15.26:0/2284554966 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc47639c700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc47539a700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 -- 172.21.15.26:0/2284554966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 msgr2=0x55c8a9cd5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 -- 172.21.15.26:0/2284554966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 msgr2=0x55c8a9cd56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.980+0000 7fc475b9b700 1 -- 172.21.15.26:0/2284554966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc460009830 con 0x55c8a9cca0c0 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc475b9b700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc46c09af80 tx=0x7fc46c0975c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7d1b47152249c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc4597fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c09adc0 con 0x55c8a9cca0c0 2023-10-06T21:24:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc46c09b980 con 0x55c8a9cca0c0 2023-10-06T21:24:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8a9cbdf10 con 0x55c8a9cca0c0 2023-10-06T21:24:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc46c09abe0 con 0x55c8a9cca0c0 2023-10-06T21:24:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc48b805b80 10 client.27473 fetch_fsmap learned FSMap version 56 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.981+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c8a9a31c60 con 0x55c8a9cca0c0 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8a9cbdf10 con 0x55c8a9cca0c0 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc46c0b92a0 con 0x55c8a9cca0c0 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 10 client.27473 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 20 client.27473 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9ce4fe0 0x55c8a9ce7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 4 client.27473 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.982+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8a9c16e10 con 0x55c8a9ce4fe0 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.983+0000 7fc47539a700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9ce4fe0 0x55c8a9ce7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.983+0000 7fc4597fa700 10 client.27473 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.983+0000 7fc47539a700 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9ce4fe0 0x55c8a9ce7400 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e08f3566352274 server_cookie=d5094555d32c77df in_seq=0 out_seq=0 2023-10-06T21:24:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.984+0000 7fc4597fa700 1 -- 172.21.15.26:0/2284554966 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8a9c16e10 con 0x55c8a9ce4fe0 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.984+0000 7fc4597fa700 10 client.27473 handle_command_reply: tid=0 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.984+0000 7fc48b805b80 1 client.27473 shutdown 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.984+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 msgr2=0x55c8a9cd4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.984+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc46c09af80 tx=0x7fc46c0975c0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 shutdown_connections 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9ce4fe0 0x55c8a9ce7400 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd4c30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 --2- 172.21.15.26:0/2284554966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 >> 172.21.15.26:0/2284554966 conn(0x55c8a9c3f260 msgr2=0x55c8a9cc5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc4597fa700 0 client.27473 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 shutdown_connections 2023-10-06T21:24:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 1 -- 172.21.15.26:0/2284554966 wait complete. 2023-10-06T21:24:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:52.985+0000 7fc48b805b80 20 client.27473 trim_cache size 0 max 16384 2023-10-06T21:24:53.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.009+0000 7fc48b805b80 1 Processor -- start 2023-10-06T21:24:53.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.009+0000 7fc48b805b80 1 -- start start 2023-10-06T21:24:53.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9a9e920 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9cd5da0 con 0x55c8a9cca0c0 2023-10-06T21:24:53.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc48b805b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9c16e40 con 0x55c8a9cca9e0 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53748/0 (socket says 172.21.15.26:53748) 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 -- 172.21.15.26:0/3910574676 learned_addr learned my addr 172.21.15.26:0/3910574676 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc47639c700 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc47539a700 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 -- 172.21.15.26:0/3910574676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 msgr2=0x55c8a9cd23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 -- 172.21.15.26:0/3910574676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 msgr2=0x55c8a9cd4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.010+0000 7fc475b9b700 1 -- 172.21.15.26:0/3910574676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a9cd47a0 con 0x55c8a9cca0c0 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.011+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc46c097cf0 tx=0x7fc46c0094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a762f65c06c5949 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.011+0000 7fc464bff700 1 -- 172.21.15.26:0/3910574676 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c09adc0 con 0x55c8a9cca0c0 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.011+0000 7fc464bff700 1 -- 172.21.15.26:0/3910574676 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46c07b070 con 0x55c8a9cca0c0 2023-10-06T21:24:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.011+0000 7fc464bff700 1 -- 172.21.15.26:0/3910574676 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c09b310 con 0x55c8a9cca0c0 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 -- 172.21.15.26:0/3910574676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 msgr2=0x55c8a9cd1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc46c097cf0 tx=0x7fc46c0094b0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 -- 172.21.15.26:0/3910574676 shutdown_connections 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9cd4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cd1ea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3910574676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cd23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 -- 172.21.15.26:0/3910574676 >> 172.21.15.26:0/3910574676 conn(0x55c8a9c3f260 msgr2=0x55c8a9ccff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 -- 172.21.15.26:0/3910574676 shutdown_connections 2023-10-06T21:24:53.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.012+0000 7fc48b805b80 1 -- 172.21.15.26:0/3910574676 wait complete. 2023-10-06T21:24:53.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.013+0000 7fc48b805b80 1 -- start start 2023-10-06T21:24:53.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.013+0000 7fc48b805b80 1 Processor -- start 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.013+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cdbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cdc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9d32320 con 0x55c8a9cca9e0 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9d32490 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a9d32650 con 0x55c8a9cca0c0 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc48b805b80 10 client.0 _ll_register_callbacks cb 0x7fc46c022bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc47639c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc47639c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:53752/0 (socket says 172.21.15.26:53752) 2023-10-06T21:24:53.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.014+0000 7fc47639c700 1 -- 172.21.15.26:0/3110078514 learned_addr learned my addr 172.21.15.26:0/3110078514 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:24:53.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47539a700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cdc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 -- 172.21.15.26:0/3110078514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca0c0 msgr2=0x55c8a9cdbf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:24:53.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cdbf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 -- 172.21.15.26:0/3110078514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 msgr2=0x55c8a9cdc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:24:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cdc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 -- 172.21.15.26:0/3110078514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a9a31c60 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47539a700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cdc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:24:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc47639c700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc4680236b0 tx=0x7fc468004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c49b3088410f9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:24:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.016+0000 7fc4597fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.017+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800e040 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.017+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc468031740 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.017+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc460011cf0 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc468031900 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 10 client.27479 fetch_fsmap learned FSMap version 56 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c8a9a6f250 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc460011cf0 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc468030b60 con 0x55c8a9cd9ca0 2023-10-06T21:24:53.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 10 client.27479 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:24:53.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 10 client.27479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 20 client.27479 populate_metadata read hostname 'smithi026' 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9d39ee0 0x55c8a9d3c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 4 client.27479 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.018+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8a9c16b30 con 0x55c8a9d39ee0 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.019+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9d39ee0 0x55c8a9d3c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.019+0000 7fc475b9b700 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9d39ee0 0x55c8a9d3c300 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d1776aa6abb5d9 server_cookie=e47a951045ba031f in_seq=0 out_seq=0 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.019+0000 7fc4597fa700 10 client.27479 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc4597fa700 1 -- 172.21.15.26:0/3110078514 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c8a9c16b30 con 0x55c8a9d39ee0 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc4597fa700 10 client.27479 handle_command_reply: tid=0 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 client.27479 shutdown 2023-10-06T21:24:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 msgr2=0x55c8a9d31de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc4680236b0 tx=0x7fc468004ad0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 shutdown_connections 2023-10-06T21:24:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55c8a9d39ee0 0x55c8a9d3c300 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8a9cd9ca0 0x55c8a9d31de0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c8a9cca0c0 0x55c8a9cdbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 --2- 172.21.15.26:0/3110078514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8a9cca9e0 0x55c8a9cdc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 >> 172.21.15.26:0/3110078514 conn(0x55c8a9c3f260 msgr2=0x55c8a9cd33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.020+0000 7fc4597fa700 0 client.27479 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.021+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 shutdown_connections 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.021+0000 7fc48b805b80 1 -- 172.21.15.26:0/3110078514 wait complete. 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.021+0000 7fc48b805b80 20 client.27479 trim_cache size 0 max 16384 2023-10-06T21:24:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:24:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 msgr2=0x7fc45c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 0x7fc45c0a3b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc468017fd0 tx=0x7fc468019040 comp rx=0 tx=0).stop 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 msgr2=0x7fc470197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005910 tx=0x7fc46c004710 comp rx=0 tx=0).stop 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 shutdown_connections 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc47010b420 0x7fc47019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470102050 0x7fc470197020 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fc45c0a1670 0x7fc45c0a3b30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 --2- 172.21.15.26:0/3675138519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc470102970 0x7fc470197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:24:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 >> 172.21.15.26:0/3675138519 conn(0x7fc4700fb830 msgr2=0x7fc47010d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:24:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 shutdown_connections 2023-10-06T21:24:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:24:53.022+0000 7fc477dff700 1 -- 172.21.15.26:0/3675138519 wait complete. 2023-10-06T21:24:53.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:52 smithi167 ceph-mon[134494]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2023-10-06T21:24:53.053 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:24:53.054 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:52 smithi026 ceph-mon[98490]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2023-10-06T21:24:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:52 smithi078 ceph-mon[134896]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2023-10-06T21:24:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:54 smithi167 ceph-mon[134494]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-10-06T21:24:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:54 smithi026 ceph-mon[98490]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-10-06T21:24:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:54 smithi078 ceph-mon[134896]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-10-06T21:24:57.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:56 smithi167 ceph-mon[134494]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 512 op/s 2023-10-06T21:24:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:56 smithi026 ceph-mon[98490]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 512 op/s 2023-10-06T21:24:57.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:56 smithi078 ceph-mon[134896]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 512 op/s 2023-10-06T21:24:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 855.0 s, 1343.6 tps, lat 75.529 ms stddev 89.802 2023-10-06T21:24:58.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:57 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:58.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:57 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:58.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:57 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:24:59.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:58 smithi167 ceph-mon[134494]: pgmap v459: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 752 op/s 2023-10-06T21:24:59.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:58 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:59.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:58 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:59.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:24:58 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:24:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:58 smithi026 ceph-mon[98490]: pgmap v459: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 752 op/s 2023-10-06T21:24:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:58 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:59.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:58 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:59.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:24:58 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:24:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:58 smithi078 ceph-mon[134896]: pgmap v459: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 752 op/s 2023-10-06T21:24:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:58 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:24:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:58 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:24:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:24:58 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:25:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:00 smithi167 ceph-mon[134494]: pgmap v460: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2023-10-06T21:25:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:00 smithi026 ceph-mon[98490]: pgmap v460: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2023-10-06T21:25:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:00 smithi078 ceph-mon[134896]: pgmap v460: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 571 op/s 2023-10-06T21:25:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 860.0 s, 1026.4 tps, lat 96.458 ms stddev 100.880 2023-10-06T21:25:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:02 smithi167 ceph-mon[134494]: pgmap v461: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 677 op/s 2023-10-06T21:25:03.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:02 smithi026 ceph-mon[98490]: pgmap v461: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 677 op/s 2023-10-06T21:25:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:02 smithi078 ceph-mon[134896]: pgmap v461: 97 pgs: 97 active+clean; 5.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.5 MiB/s wr, 677 op/s 2023-10-06T21:25:04.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:04 smithi167 ceph-mon[134494]: pgmap v462: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2023-10-06T21:25:04.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:04 smithi026 ceph-mon[98490]: pgmap v462: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2023-10-06T21:25:04.882 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:04 smithi078 ceph-mon[134896]: pgmap v462: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.4 MiB/s wr, 832 op/s 2023-10-06T21:25:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.035+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/3683665259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410fa90 msgr2=0x7f0a041083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.035+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/3683665259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410fa90 0x7f0a041083f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09ec01a9a0 tx=0x7f09ec01acb0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/3683665259 shutdown_connections 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/3683665259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0411a690 0x7f0a0411ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/3683665259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0410f620 0x7f0a04107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/3683665259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410fa90 0x7f0a041083f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/3683665259 >> 172.21.15.26:0/3683665259 conn(0x7f0a0406cfa0 msgr2=0x7f0a0406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/3683665259 shutdown_connections 2023-10-06T21:25:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.036+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/3683665259 wait complete. 2023-10-06T21:25:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.037+0000 7f0a0a1ff700 1 Processor -- start 2023-10-06T21:25:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 -- start start 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0410fa90 0x7f0a04115e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0411a690 0x7f0a041a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041a39f0 con 0x7f0a0410fa90 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041a3b60 con 0x7f0a0411a690 2023-10-06T21:25:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.038+0000 7f0a0a1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041a3d20 con 0x7f0a0410f620 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0410fa90 0x7f0a04115e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51584/0 (socket says 172.21.15.26:51584) 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 -- 172.21.15.26:0/659000491 learned_addr learned my addr 172.21.15.26:0/659000491 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a03fff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0411a690 0x7f0a041a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 -- 172.21.15.26:0/659000491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0411a690 msgr2=0x7f0a041a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0411a690 0x7f0a041a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 -- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0410fa90 msgr2=0x7f0a04115e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0410fa90 0x7f0a04115e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.039+0000 7f0a037fe700 1 -- 172.21.15.26:0/659000491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec01a600 con 0x7f0a0410f620 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.040+0000 7f0a037fe700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09f400c9b0 tx=0x7f09f400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.040+0000 7f0a017fa700 1 -- 172.21.15.26:0/659000491 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4007b00 con 0x7f0a0410f620 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.041+0000 7f0a017fa700 1 -- 172.21.15.26:0/659000491 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f400ce50 con 0x7f0a0410f620 2023-10-06T21:25:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.041+0000 7f0a017fa700 1 -- 172.21.15.26:0/659000491 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4018660 con 0x7f0a0410f620 2023-10-06T21:25:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.041+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a041a4000 con 0x7f0a0410f620 2023-10-06T21:25:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.041+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a041aa620 con 0x7f0a0410f620 2023-10-06T21:25:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.052+0000 7f0a1db39b80 1 Processor -- start 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.052+0000 7f0a1db39b80 1 -- start start 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93892e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b937f1400 con 0x556b938933d0 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b937e2bc0 con 0x556b93887c00 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b937e04e0 con 0x556b93888520 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33148/0 (socket says 172.21.15.26:33148) 2023-10-06T21:25:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.053+0000 7f0a03fff700 1 -- 172.21.15.26:0/657710386 learned_addr learned my addr 172.21.15.26:0/657710386 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 -- 172.21.15.26:0/657710386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 msgr2=0x556b93892e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T21:25:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93892e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 -- 172.21.15.26:0/657710386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 msgr2=0x556b938957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 -- 172.21.15.26:0/657710386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b93606d80 con 0x556b93887c00 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a03fff700 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00a400 tx=0x7f09fc009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da79c03c7493f794 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f0a017fa700 1 -- 172.21.15.26:0/659000491 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f09f400f450 con 0x7f0a0410f620 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f09f9ffb700 1 -- 172.21.15.26:0/657710386 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010040 con 0x556b93887c00 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f09f9ffb700 1 -- 172.21.15.26:0/657710386 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc014ae0 con 0x556b93887c00 2023-10-06T21:25:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.054+0000 7f09f9ffb700 1 -- 172.21.15.26:0/657710386 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc013a80 con 0x556b93887c00 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.055+0000 7f0a1db39b80 1 -- 172.21.15.26:0/657710386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 msgr2=0x556b93887fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.055+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00a400 tx=0x7f09fc009650 comp rx=0 tx=0).stop 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.056+0000 7f0a1db39b80 1 -- 172.21.15.26:0/657710386 shutdown_connections 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.056+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.056+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b93887fe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.056+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/657710386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93892e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.056+0000 7f0a1db39b80 1 -- 172.21.15.26:0/657710386 >> 172.21.15.26:0/657710386 conn(0x556b937fd200 msgr2=0x556b93884080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.057+0000 7f0a1db39b80 1 -- 172.21.15.26:0/657710386 shutdown_connections 2023-10-06T21:25:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.057+0000 7f0a1db39b80 1 -- 172.21.15.26:0/657710386 wait complete. 2023-10-06T21:25:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.057+0000 7f0a017fa700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 0x7f09e40a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.058+0000 7f0a1db39b80 1 -- start start 2023-10-06T21:25:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.058+0000 7f0a017fa700 1 -- 172.21.15.26:0/659000491 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09f40ccff0 con 0x7f0a0410f620 2023-10-06T21:25:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.058+0000 7f0a03fff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 0x7f09e40a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.058+0000 7f0a1db39b80 1 Processor -- start 2023-10-06T21:25:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938901d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93890710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b937dbdd0 con 0x556b938933d0 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b937786b0 con 0x556b93887c00 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b93778350 con 0x556b93888520 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a03fff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 0x7f09e40a3be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09fc013560 tx=0x7f09fc00c040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.059+0000 7f0a1db39b80 10 client.0 _ll_register_callbacks cb 0x556b935d57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.060+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.060+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35658/0 (socket says 172.21.15.26:35658) 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.060+0000 7f0a03fff700 1 -- 172.21.15.26:0/696003268 learned_addr learned my addr 172.21.15.26:0/696003268 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.060+0000 7f09fbfff700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93890710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.060+0000 7f0a037fe700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938901d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 -- 172.21.15.26:0/696003268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 msgr2=0x556b93890710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93890710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 -- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 msgr2=0x556b938901d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938901d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 -- 172.21.15.26:0/696003268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc009c60 con 0x556b938933d0 2023-10-06T21:25:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f09fbfff700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93890710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a037fe700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938901d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:25:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a03fff700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f09fc018000 tx=0x7f09fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8354f5707053e4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f09f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:25:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010050 con 0x556b938933d0 2023-10-06T21:25:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09fc006e40 con 0x556b938933d0 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.061+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b9387bd50 con 0x556b938933d0 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc014980 con 0x556b938933d0 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f0a1db39b80 10 client.17106 fetch_fsmap learned FSMap version 56 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556b935efc00 con 0x556b938933d0 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b9387bd50 con 0x556b938933d0 2023-10-06T21:25:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f09fc01e7b0 con 0x556b938933d0 2023-10-06T21:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.062+0000 7f0a1db39b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a1db39b80 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a1db39b80 20 client.17106 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938a3050 0x556b938a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a1db39b80 4 client.17106 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b937d4db0 con 0x556b938a3050 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a037fe700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938a3050 0x556b938a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f0a037fe700 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938a3050 0x556b938a5470 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293405e0880c41f5 server_cookie=dfa97114dbdbe4fb in_seq=0 out_seq=0 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.063+0000 7f09f2ffd700 10 client.17106 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.064+0000 7f09f2ffd700 1 -- 172.21.15.26:0/696003268 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556b937d4db0 con 0x556b938a3050 2023-10-06T21:25:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.064+0000 7f09f2ffd700 10 client.17106 handle_command_reply: tid=0 2023-10-06T21:25:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.064+0000 7f0a1db39b80 1 client.17106 shutdown 2023-10-06T21:25:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 msgr2=0x556b93890c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f09fc018000 tx=0x7f09fc004ad0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 shutdown_connections 2023-10-06T21:25:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b93890c50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938a3050 0x556b938a5470 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938901d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f09f2ffd700 0 client.17106 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/696003268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b93890710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 >> 172.21.15.26:0/696003268 conn(0x556b937fd200 msgr2=0x556b93883c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.065+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 shutdown_connections 2023-10-06T21:25:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.066+0000 7f0a1db39b80 1 -- 172.21.15.26:0/696003268 wait complete. 2023-10-06T21:25:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.066+0000 7f0a1db39b80 20 client.17106 trim_cache size 0 max 16384 2023-10-06T21:25:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.099+0000 7f0a1db39b80 1 Processor -- start 2023-10-06T21:25:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.099+0000 7f0a1db39b80 1 -- start start 2023-10-06T21:25:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.099+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.099+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b93888520 0x556b9388d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b938933d0 0x556b9388d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9365c8c0 con 0x556b93888520 2023-10-06T21:25:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b93880890 con 0x556b938933d0 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9389a460 con 0x556b93887c00 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51604/0 (socket says 172.21.15.26:51604) 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 -- 172.21.15.26:0/4268041179 learned_addr learned my addr 172.21.15.26:0/4268041179 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a03fff700 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b938933d0 0x556b9388d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.106 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f09fbfff700 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b93888520 0x556b9388d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 -- 172.21.15.26:0/4268041179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b938933d0 msgr2=0x556b9388d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b938933d0 0x556b9388d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 -- 172.21.15.26:0/4268041179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b93888520 msgr2=0x556b9388d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b93888520 0x556b9388d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.100+0000 7f0a037fe700 1 -- 172.21.15.26:0/4268041179 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b938aa060 con 0x556b93887c00 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.101+0000 7f0a037fe700 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09f4099ef0 tx=0x7f09f4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1582f179b96b92fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.101+0000 7f09fa7fc700 1 -- 172.21.15.26:0/4268041179 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f409a660 con 0x556b93887c00 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.101+0000 7f09fa7fc700 1 -- 172.21.15.26:0/4268041179 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f4099f60 con 0x556b93887c00 2023-10-06T21:25:05.107 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.101+0000 7f09fa7fc700 1 -- 172.21.15.26:0/4268041179 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40ca950 con 0x556b93887c00 2023-10-06T21:25:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 -- 172.21.15.26:0/4268041179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 msgr2=0x556b93899f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09f4099ef0 tx=0x7f09f4020000 comp rx=0 tx=0).stop 2023-10-06T21:25:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 -- 172.21.15.26:0/4268041179 shutdown_connections 2023-10-06T21:25:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b938933d0 0x556b9388d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.108 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93887c00 0x556b93899f20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/4268041179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b93888520 0x556b9388d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 -- 172.21.15.26:0/4268041179 >> 172.21.15.26:0/4268041179 conn(0x556b937fd200 msgr2=0x556b9388b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 -- 172.21.15.26:0/4268041179 shutdown_connections 2023-10-06T21:25:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.102+0000 7f0a1db39b80 1 -- 172.21.15.26:0/4268041179 wait complete. 2023-10-06T21:25:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.103+0000 7f0a1db39b80 1 -- start start 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.103+0000 7f0a1db39b80 1 Processor -- start 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b938f5ee0 con 0x556b938933d0 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b938f6020 con 0x556b93887c00 2023-10-06T21:25:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b938f61e0 con 0x556b93888520 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f0a1db39b80 10 client.0 _ll_register_callbacks cb 0x7f09fc0147e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:51612/0 (socket says 172.21.15.26:51612) 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.104+0000 7f09fbfff700 1 -- 172.21.15.26:0/2528754767 learned_addr learned my addr 172.21.15.26:0/2528754767 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 -- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 msgr2=0x556b938eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 -- 172.21.15.26:0/2528754767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 msgr2=0x556b938eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 -- 172.21.15.26:0/2528754767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a041aa620 con 0x556b93888520 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09fbfff700 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09ec01a8f0 tx=0x7f09ec005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d86f1787694d30c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec02d040 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f09ec004660 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.105+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09fc014ae0 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.106+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09ec004820 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.107+0000 7f0a1db39b80 10 client.27328 fetch_fsmap learned FSMap version 56 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.107+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556b9362d1f0 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.107+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc014ae0 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 56) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f09ec03a020 con 0x556b93888520 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 10 client.27328 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T21:25:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 10 client.27328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 20 client.27328 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938f7920 0x556b938f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 4 client.27328 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.108+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b937d4ad0 con 0x556b938f7920 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.110+0000 7f0a037fe700 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938f7920 0x556b938f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.111+0000 7f0a037fe700 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938f7920 0x556b938f9d40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50f922d8e50f4b0 server_cookie=b907e8ca086b7360 in_seq=0 out_seq=0 2023-10-06T21:25:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.111+0000 7f09f2ffd700 10 client.27328 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.129+0000 7f09f2ffd700 1 -- 172.21.15.26:0/2528754767 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b937d4ad0 con 0x556b938f7920 2023-10-06T21:25:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.129+0000 7f09f2ffd700 10 client.27328 handle_command_reply: tid=0 2023-10-06T21:25:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.130+0000 7f0a1db39b80 1 client.27328 shutdown 2023-10-06T21:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.130+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 msgr2=0x556b938eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.130+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09ec01a8f0 tx=0x7f09ec005320 comp rx=0 tx=0).stop 2023-10-06T21:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.130+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 shutdown_connections 2023-10-06T21:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b938933d0 0x556b938eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556b938f7920 0x556b938f9d40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556b93887c00 0x556b938eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 --2- 172.21.15.26:0/2528754767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b93888520 0x556b938eb310 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 >> 172.21.15.26:0/2528754767 conn(0x556b937fd200 msgr2=0x556b93889a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f09f2ffd700 0 client.27328 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 shutdown_connections 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 1 -- 172.21.15.26:0/2528754767 wait complete. 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.131+0000 7f0a1db39b80 20 client.27328 trim_cache size 0 max 16384 2023-10-06T21:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 msgr2=0x7f09e40a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 0x7f09e40a3be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f09fc013560 tx=0x7f09fc00c040 comp rx=0 tx=0).stop 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 msgr2=0x7f0a04115920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09f400c9b0 tx=0x7f09f400ccc0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 shutdown_connections 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a0411a690 0x7f0a041a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a0410f620 0x7f0a04115920 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f09e40a1720 0x7f09e40a3be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 --2- 172.21.15.26:0/659000491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a0410fa90 0x7f0a04115e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.132+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 >> 172.21.15.26:0/659000491 conn(0x7f0a0406cfa0 msgr2=0x7f0a0411bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.133+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 shutdown_connections 2023-10-06T21:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:05.133+0000 7f0a0a1ff700 1 -- 172.21.15.26:0/659000491 wait complete. 2023-10-06T21:25:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:06 smithi167 ceph-mon[134494]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:25:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:06 smithi167 ceph-mon[134494]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.5 MiB/s wr, 867 op/s 2023-10-06T21:25:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:06 smithi026 ceph-mon[98490]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:25:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:06 smithi026 ceph-mon[98490]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.5 MiB/s wr, 867 op/s 2023-10-06T21:25:06.875 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:06 smithi078 ceph-mon[134896]: osdmap e106: 12 total, 12 up, 12 in 2023-10-06T21:25:06.876 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:06 smithi078 ceph-mon[134896]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.5 MiB/s wr, 867 op/s 2023-10-06T21:25:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 865.0 s, 576.0 tps, lat 166.739 ms stddev 237.002 2023-10-06T21:25:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:08 smithi167 ceph-mon[134494]: pgmap v465: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2023-10-06T21:25:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:08 smithi026 ceph-mon[98490]: pgmap v465: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2023-10-06T21:25:09.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:08 smithi078 ceph-mon[134896]: pgmap v465: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 5.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2023-10-06T21:25:10.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:09 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:10.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:09 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:10.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:09 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:11.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:10 smithi167 ceph-mon[134494]: pgmap v466: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 4.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:25:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:10 smithi026 ceph-mon[98490]: pgmap v466: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 4.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:25:11.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:10 smithi078 ceph-mon[134896]: pgmap v466: 97 pgs: 10 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 73 active+clean; 4.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.57k op/s 2023-10-06T21:25:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 870.0 s, 739.8 tps, lat 141.006 ms stddev 209.295 2023-10-06T21:25:13.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:12 smithi167 ceph-mon[134494]: pgmap v467: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2023-10-06T21:25:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:12 smithi026 ceph-mon[98490]: pgmap v467: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2023-10-06T21:25:13.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:12 smithi078 ceph-mon[134896]: pgmap v467: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2023-10-06T21:25:14.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:14 smithi167 ceph-mon[134494]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:25:14.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:14 smithi167 ceph-mon[134494]: pgmap v469: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-10-06T21:25:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:14 smithi026 ceph-mon[98490]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:25:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:14 smithi026 ceph-mon[98490]: pgmap v469: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-10-06T21:25:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:14 smithi078 ceph-mon[134896]: osdmap e107: 12 total, 12 up, 12 in 2023-10-06T21:25:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:14 smithi078 ceph-mon[134896]: pgmap v469: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-10-06T21:25:17.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:16 smithi167 ceph-mon[134494]: pgmap v470: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-06T21:25:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:16 smithi026 ceph-mon[98490]: pgmap v470: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-06T21:25:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:16 smithi078 ceph-mon[134896]: pgmap v470: 97 pgs: 97 active+clean; 4.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-06T21:25:17.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 875.0 s, 816.4 tps, lat 121.961 ms stddev 167.109 2023-10-06T21:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.358+0000 7fa020b9d700 1 -- 172.21.15.26:0/2888045110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c10d600 msgr2=0x7fa01c10f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.358+0000 7fa020b9d700 1 --2- 172.21.15.26:0/2888045110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c10d600 0x7fa01c10f9f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa014009f90 tx=0x7fa01400b330 comp rx=0 tx=0).stop 2023-10-06T21:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 -- 172.21.15.26:0/2888045110 shutdown_connections 2023-10-06T21:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 --2- 172.21.15.26:0/2888045110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c10d600 0x7fa01c10f9f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 --2- 172.21.15.26:0/2888045110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c0ffa50 0x7fa01c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 --2- 172.21.15.26:0/2888045110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c100400 0x7fa01c10d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 -- 172.21.15.26:0/2888045110 >> 172.21.15.26:0/2888045110 conn(0x7fa01c0fb690 msgr2=0x7fa01c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.361+0000 7fa020b9d700 1 -- 172.21.15.26:0/2888045110 shutdown_connections 2023-10-06T21:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.362+0000 7fa020b9d700 1 -- 172.21.15.26:0/2888045110 wait complete. 2023-10-06T21:25:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.363+0000 7fa020b9d700 1 Processor -- start 2023-10-06T21:25:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 -- start start 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 0x7fa01c102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c10d600 0x7fa01c1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1a97d0 con 0x7fa01c10d600 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1a9940 con 0x7fa01c0ffa50 2023-10-06T21:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.364+0000 7fa020b9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c1a9b00 con 0x7fa01c100400 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55842/0 (socket says 172.21.15.26:55842) 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1167547428 learned_addr learned my addr 172.21.15.26:0/1167547428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01affd700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 0x7fa01c102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01bfff700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c10d600 0x7fa01c1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1167547428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 msgr2=0x7fa01c102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 0x7fa01c102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c10d600 msgr2=0x7fa01c1027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c10d600 0x7fa01c1027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1167547428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0095f0 con 0x7fa01c0ffa50 2023-10-06T21:25:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.365+0000 7fa01affd700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 0x7fa01c102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00ec10 tx=0x7fa00c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa018ff9700 1 -- 172.21.15.26:0/1167547428 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00cc90 con 0x7fa01c0ffa50 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa014009c40 con 0x7fa01c0ffa50 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa018ff9700 1 -- 172.21.15.26:0/1167547428 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c0044e0 con 0x7fa01c0ffa50 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa018ff9700 1 -- 172.21.15.26:0/1167547428 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c010430 con 0x7fa01c0ffa50 2023-10-06T21:25:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.366+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1aa160 con 0x7fa01c0ffa50 2023-10-06T21:25:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.371+0000 7fa018ff9700 1 -- 172.21.15.26:0/1167547428 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa00c003fb0 con 0x7fa01c0ffa50 2023-10-06T21:25:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.372+0000 7fa03565cb80 1 Processor -- start 2023-10-06T21:25:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 -- start start 2023-10-06T21:25:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0778140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0783530 0x55abf0785920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf06e1400 con 0x55abf0777d60 2023-10-06T21:25:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf06d2bc0 con 0x55abf0778680 2023-10-06T21:25:18.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.373+0000 7fa03565cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf06d04e0 con 0x55abf0783530 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0783530 0x55abf0785920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55850/0 (socket says 172.21.15.26:55850) 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1008864903 learned_addr learned my addr 172.21.15.26:0/1008864903 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01bfff700 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0778140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa018ff9700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 0x7fa0080a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1008864903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0783530 msgr2=0x55abf0785920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0783530 0x55abf0785920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1008864903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 msgr2=0x55abf0778140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0778140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01b7fe700 1 -- 172.21.15.26:0/1008864903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf04f6d80 con 0x55abf0778680 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.374+0000 7fa01bfff700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 0x7fa0080a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.375+0000 7fa018ff9700 1 -- 172.21.15.26:0/1167547428 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa00c014070 con 0x7fa01c0ffa50 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.375+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa00c09a330 tx=0x7fa00c09a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fc65f545248c31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.375+0000 7fa011ffb700 1 -- 172.21.15.26:0/1008864903 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c009a10 con 0x55abf0778680 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.375+0000 7fa011ffb700 1 -- 172.21.15.26:0/1008864903 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c00cdf0 con 0x55abf0778680 2023-10-06T21:25:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.375+0000 7fa011ffb700 1 -- 172.21.15.26:0/1008864903 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0163a0 con 0x55abf0778680 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa01bfff700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 0x7fa0080a3ab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0140060b0 tx=0x7fa014009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 -- 172.21.15.26:0/1008864903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 msgr2=0x55abf0782ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa00c09a330 tx=0x7fa00c09a260 comp rx=0 tx=0).stop 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 -- 172.21.15.26:0/1008864903 shutdown_connections 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0783530 0x55abf0785920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0778140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 --2- 172.21.15.26:0/1008864903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf0782ff0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 -- 172.21.15.26:0/1008864903 >> 172.21.15.26:0/1008864903 conn(0x55abf06ed200 msgr2=0x55abf07741b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 -- 172.21.15.26:0/1008864903 shutdown_connections 2023-10-06T21:25:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.376+0000 7fa03565cb80 1 -- 172.21.15.26:0/1008864903 wait complete. 2023-10-06T21:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.377+0000 7fa03565cb80 1 -- start start 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.377+0000 7fa03565cb80 1 Processor -- start 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0782f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf07801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf06cbdd0 con 0x55abf0777d60 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf06686b0 con 0x55abf0783530 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.378+0000 7fa03565cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf0668350 con 0x55abf0778680 2023-10-06T21:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa03565cb80 10 client.0 _ll_register_callbacks cb 0x55abf04c57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55858/0 (socket says 172.21.15.26:55858) 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 -- 172.21.15.26:0/3230033238 learned_addr learned my addr 172.21.15.26:0/3230033238 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01affd700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf07801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 -- 172.21.15.26:0/3230033238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 msgr2=0x55abf07801a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf07801a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 -- 172.21.15.26:0/3230033238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 msgr2=0x55abf0782f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.379+0000 7fa01bfff700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0782f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7fa01bfff700 1 -- 172.21.15.26:0/3230033238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1aa160 con 0x55abf0783530 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7fa01bfff700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa014015000 tx=0x7fa014009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22de50dfd7e2a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7f9ffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400e040 con 0x55abf0783530 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa01400be00 con 0x55abf0783530 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.380+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abf076beb0 con 0x55abf0783530 2023-10-06T21:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.381+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01404b450 con 0x55abf0783530 2023-10-06T21:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 10 client.27503 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55abf04dfc00 con 0x55abf0783530 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abf076beb0 con 0x55abf0783530 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa014026070 con 0x55abf0783530 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 10 client.27503 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 10 client.27503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 20 client.27503 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf0791a60 0x55abf0793e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 4 client.27503 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.383+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abf06c4db0 con 0x55abf0791a60 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.384+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf0791a60 0x55abf0793e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.384+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf0791a60 0x55abf0793e80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9be97c4d8cf4f5 server_cookie=5f303061acb18a91 in_seq=0 out_seq=0 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.384+0000 7f9ffaffd700 10 client.27503 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7f9ffaffd700 1 -- 172.21.15.26:0/3230033238 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abf06c4db0 con 0x55abf0791a60 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7f9ffaffd700 10 client.27503 handle_command_reply: tid=0 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 client.27503 shutdown 2023-10-06T21:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 msgr2=0x55abf0780820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa014015000 tx=0x7fa014009700 comp rx=0 tx=0).stop 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 shutdown_connections 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf0791a60 0x55abf0793e80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0780820 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0782f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7f9ffaffd700 0 client.27503 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.385+0000 7fa03565cb80 1 --2- 172.21.15.26:0/3230033238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf07801a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.386+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 >> 172.21.15.26:0/3230033238 conn(0x55abf06ed200 msgr2=0x55abf0773da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.386+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 shutdown_connections 2023-10-06T21:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.386+0000 7fa03565cb80 1 -- 172.21.15.26:0/3230033238 wait complete. 2023-10-06T21:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.386+0000 7fa03565cb80 20 client.27503 trim_cache size 0 max 16384 2023-10-06T21:25:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.409+0000 7fa03565cb80 1 Processor -- start 2023-10-06T21:25:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.409+0000 7fa03565cb80 1 -- start start 2023-10-06T21:25:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf078a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0783530 0x55abf077f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf054c8c0 con 0x55abf0783530 2023-10-06T21:25:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf078a660 con 0x55abf0778680 2023-10-06T21:25:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa03565cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf0775660 con 0x55abf0777d60 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49748/0 (socket says 172.21.15.26:49748) 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 -- 172.21.15.26:0/451871776 learned_addr learned my addr 172.21.15.26:0/451871776 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01bfff700 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0783530 0x55abf077f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01affd700 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf078a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 -- 172.21.15.26:0/451871776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 msgr2=0x55abf078a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf078a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 -- 172.21.15.26:0/451871776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0783530 msgr2=0x55abf077f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0783530 0x55abf077f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.410+0000 7fa01b7fe700 1 -- 172.21.15.26:0/451871776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf0785af0 con 0x55abf0777d60 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.411+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa00c09a5c0 tx=0x7fa00c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ed9a301c9559310 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.411+0000 7fa0127fc700 1 -- 172.21.15.26:0/451871776 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0850d0 con 0x55abf0777d60 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.411+0000 7fa0127fc700 1 -- 172.21.15.26:0/451871776 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c016330 con 0x55abf0777d60 2023-10-06T21:25:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.411+0000 7fa0127fc700 1 -- 172.21.15.26:0/451871776 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c004260 con 0x55abf0777d60 2023-10-06T21:25:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- 172.21.15.26:0/451871776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 msgr2=0x55abf077ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa00c09a5c0 tx=0x7fa00c00ef90 comp rx=0 tx=0).stop 2023-10-06T21:25:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- 172.21.15.26:0/451871776 shutdown_connections 2023-10-06T21:25:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0783530 0x55abf077f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0777d60 0x55abf077ac00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 --2- 172.21.15.26:0/451871776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0778680 0x55abf078a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- 172.21.15.26:0/451871776 >> 172.21.15.26:0/451871776 conn(0x55abf06ed200 msgr2=0x55abf078e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- 172.21.15.26:0/451871776 shutdown_connections 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- 172.21.15.26:0/451871776 wait complete. 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.412+0000 7fa03565cb80 1 -- start start 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 Processor -- start 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0786050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf0786590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0786ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf078a400 con 0x55abf0777d60 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf07e6890 con 0x55abf0783530 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf07e69d0 con 0x55abf0778680 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.413+0000 7fa03565cb80 10 client.0 _ll_register_callbacks cb 0x7fa014007b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0786050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf0786590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0786050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38134/0 (socket says 172.21.15.26:38134) 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01b7fe700 1 -- 172.21.15.26:0/2303074633 learned_addr learned my addr 172.21.15.26:0/2303074633 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01bfff700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0786ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 -- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 msgr2=0x55abf0786ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0786ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 -- 172.21.15.26:0/2303074633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 msgr2=0x55abf0786050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0786050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 -- 172.21.15.26:0/2303074633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abf04f6d80 con 0x55abf0778680 2023-10-06T21:25:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7fa01affd700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf0786590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55abf076ba50 tx=0x7fa004003be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8155fda4e3a8cf42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.414+0000 7f9ffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.415+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004017070 con 0x55abf0778680 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.415+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa004004a30 con 0x55abf0778680 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.416+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa00c00cdf0 con 0x55abf0778680 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.417+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa004004bf0 con 0x55abf0778680 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.417+0000 7fa03565cb80 10 client.27346 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.417+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55abf051d1f0 con 0x55abf0778680 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa00c00cdf0 con 0x55abf0778680 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa004024050 con 0x55abf0778680 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 10 client.27346 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 10 client.27346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 20 client.27346 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf07e8180 0x55abf07ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 4 client.27346 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.418+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abf06c4ad0 con 0x55abf07e8180 2023-10-06T21:25:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.419+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf07e8180 0x55abf07ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.419+0000 7fa01b7fe700 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf07e8180 0x55abf07ea5a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6ce83ea854c397 server_cookie=2fc4f07d95fe7113 in_seq=0 out_seq=0 2023-10-06T21:25:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.419+0000 7f9ffaffd700 10 client.27346 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7f9ffaffd700 1 -- 172.21.15.26:0/2303074633 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abf06c4ad0 con 0x55abf07e8180 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7f9ffaffd700 10 client.27346 handle_command_reply: tid=0 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 client.27346 shutdown 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 msgr2=0x55abf0786590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf0786590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55abf076ba50 tx=0x7fa004003be0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 shutdown_connections 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abf07e8180 0x55abf07ea5a0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abf0783530 0x55abf0786ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abf0777d60 0x55abf0786050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 --2- 172.21.15.26:0/2303074633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abf0778680 0x55abf0786590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 >> 172.21.15.26:0/2303074633 conn(0x55abf06ed200 msgr2=0x55abf077b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.497+0000 7f9ffaffd700 0 client.27346 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.498+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 shutdown_connections 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.498+0000 7fa03565cb80 1 -- 172.21.15.26:0/2303074633 wait complete. 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.498+0000 7fa03565cb80 20 client.27346 trim_cache size 0 max 16384 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:25:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.498+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 msgr2=0x7fa0080a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.498+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 0x7fa0080a3ab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0140060b0 tx=0x7fa014009920 comp rx=0 tx=0).stop 2023-10-06T21:25:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 msgr2=0x7fa01c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00ec10 tx=0x7fa00c00ef20 comp rx=0 tx=0).stop 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 shutdown_connections 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa0080a15f0 0x7fa0080a3ab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01c10d600 0x7fa01c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa01c0ffa50 0x7fa01c101d30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 --2- 172.21.15.26:0/1167547428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01c100400 0x7fa01c102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 >> 172.21.15.26:0/1167547428 conn(0x7fa01c0fb690 msgr2=0x7fa01c10e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 shutdown_connections 2023-10-06T21:25:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:18.499+0000 7fa020b9d700 1 -- 172.21.15.26:0/1167547428 wait complete. 2023-10-06T21:25:19.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:18 smithi167 ceph-mon[134494]: pgmap v471: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 796 op/s 2023-10-06T21:25:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:18 smithi026 ceph-mon[98490]: pgmap v471: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 796 op/s 2023-10-06T21:25:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:18 smithi078 ceph-mon[134896]: pgmap v471: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 796 op/s 2023-10-06T21:25:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:20 smithi167 ceph-mon[134494]: pgmap v472: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 743 op/s 2023-10-06T21:25:21.080 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:20 smithi026 ceph-mon[98490]: pgmap v472: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 743 op/s 2023-10-06T21:25:21.124 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:20 smithi078 ceph-mon[134896]: pgmap v472: 97 pgs: 97 active+clean; 4.8 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 743 op/s 2023-10-06T21:25:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 880.0 s, 1027.9 tps, lat 97.205 ms stddev 109.791 2023-10-06T21:25:23.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:22 smithi167 ceph-mon[134494]: pgmap v473: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 722 op/s 2023-10-06T21:25:23.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:25:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:22 smithi026 ceph-mon[98490]: pgmap v473: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 722 op/s 2023-10-06T21:25:23.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:22 smithi078 ceph-mon[134896]: pgmap v473: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 722 op/s 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 -- 172.21.15.26:0/1135936502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc111bc0 msgr2=0x7f0cdc113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/1135936502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc113fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009f90 tx=0x7f0cd400b330 comp rx=0 tx=0).stop 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 -- 172.21.15.26:0/1135936502 shutdown_connections 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/1135936502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc113fb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/1135936502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0fef30 0x7f0cdc0ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/1135936502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 -- 172.21.15.26:0/1135936502 >> 172.21.15.26:0/1135936502 conn(0x7f0cdc076ed0 msgr2=0x7f0cdc0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 -- 172.21.15.26:0/1135936502 shutdown_connections 2023-10-06T21:25:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.417+0000 7f0ce2bff700 1 -- 172.21.15.26:0/1135936502 wait complete. 2023-10-06T21:25:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.418+0000 7f0ce2bff700 1 Processor -- start 2023-10-06T21:25:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 -- start start 2023-10-06T21:25:23.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc0fef30 0x7f0cdc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a25a0 con 0x7f0cdc0fef30 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a2710 con 0x7f0cdc0ff850 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0ce2bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a28d0 con 0x7f0cdc111bc0 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.419+0000 7f0cdbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60382/0 (socket says 172.21.15.26:60382) 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 -- 172.21.15.26:0/17898193 learned_addr learned my addr 172.21.15.26:0/17898193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0ce099b700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc0fef30 0x7f0cdc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0ce119c700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 -- 172.21.15.26:0/17898193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc111bc0 msgr2=0x7f0cdc1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 -- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc0fef30 msgr2=0x7f0cdc19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc0fef30 0x7f0cdc19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 -- 172.21.15.26:0/17898193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4009c40 con 0x7f0cdc0ff850 2023-10-06T21:25:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.420+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c920 tx=0x7f0cc800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.421+0000 7f0cd9ffb700 1 -- 172.21.15.26:0/17898193 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8004a30 con 0x7f0cdc0ff850 2023-10-06T21:25:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.421+0000 7f0cd9ffb700 1 -- 172.21.15.26:0/17898193 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cc80054e0 con 0x7f0cdc0ff850 2023-10-06T21:25:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.421+0000 7f0cd9ffb700 1 -- 172.21.15.26:0/17898193 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8007b30 con 0x7f0cdc0ff850 2023-10-06T21:25:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.421+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc1a2bb0 con 0x7f0cdc0ff850 2023-10-06T21:25:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.421+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1aa3f0 con 0x7f0cdc0ff850 2023-10-06T21:25:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.424+0000 7f0cd9ffb700 1 -- 172.21.15.26:0/17898193 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f0cc8007c90 con 0x7f0cdc0ff850 2023-10-06T21:25:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.425+0000 7f0cf655bb80 1 Processor -- start 2023-10-06T21:25:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 -- start start 2023-10-06T21:25:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af78c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8aed4460 con 0x563f8af6b9e0 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8aec5c20 con 0x563f8af6b0c0 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8aec3540 con 0x563f8af76890 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0ce119c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0ce119c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60398/0 (socket says 172.21.15.26:60398) 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0ce119c700 1 -- 172.21.15.26:0/2875926703 learned_addr learned my addr 172.21.15.26:0/2875926703 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.426+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0cd9ffb700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 0x7f0ccc0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af78c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0cd9ffb700 1 -- 172.21.15.26:0/17898193 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0cc80ccb30 con 0x7f0cdc0ff850 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 -- 172.21.15.26:0/2875926703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 msgr2=0x563f8af78c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af78c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 -- 172.21.15.26:0/2875926703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 msgr2=0x563f8af76350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af76350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 -- 172.21.15.26:0/2875926703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8ace9de0 con 0x563f8af6b0c0 2023-10-06T21:25:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 0x7f0ccc0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.427+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4003d20 tx=0x7f0cd4003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f10e244fec885a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0cc6ffd700 1 -- 172.21.15.26:0/2875926703 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400e040 con 0x563f8af6b0c0 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0cc6ffd700 1 -- 172.21.15.26:0/2875926703 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd4007800 con 0x563f8af6b0c0 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0cc6ffd700 1 -- 172.21.15.26:0/2875926703 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4027660 con 0x563f8af6b0c0 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2875926703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 msgr2=0x563f8af6b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4003d20 tx=0x7f0cd4003d50 comp rx=0 tx=0).stop 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.428+0000 7f0ce119c700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 0x7f0ccc0a3bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4029000 tx=0x7f0cd4023000 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2875926703 shutdown_connections 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af78c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af6b4a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2875926703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af76350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2875926703 >> 172.21.15.26:0/2875926703 conn(0x563f8aee0260 msgr2=0x563f8af67450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2875926703 shutdown_connections 2023-10-06T21:25:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2875926703 wait complete. 2023-10-06T21:25:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.429+0000 7f0cf655bb80 1 -- start start 2023-10-06T21:25:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 Processor -- start 2023-10-06T21:25:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af76890 0x563f8af73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8aebee30 con 0x563f8af6b9e0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ae5b710 con 0x563f8af76890 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.430+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ae5b3b0 con 0x563f8af6b0c0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0cf655bb80 10 client.0 _ll_register_callbacks cb 0x563f8acb8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:46714/0 (socket says 172.21.15.26:46714) 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 -- 172.21.15.26:0/3737431967 learned_addr learned my addr 172.21.15.26:0/3737431967 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce119c700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af76890 0x563f8af73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 -- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af76890 msgr2=0x563f8af73bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af76890 0x563f8af73bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 -- 172.21.15.26:0/3737431967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 msgr2=0x563f8af73520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af73520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 -- 172.21.15.26:0/3737431967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc114bb0 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0ce099b700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0012720 tx=0x7f0cd0012a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ceda95b9a5e7eed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.431+0000 7f0cbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0017070 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0cd0004030 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f8af5ef10 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd00122e0 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cf655bb80 10 client.27355 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.432+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x563f8acd2c60 con 0x563f8af6b0c0 2023-10-06T21:25:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f8af5ef10 con 0x563f8af6b0c0 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cd000f4e0 con 0x563f8af6b0c0 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 10 client.27355 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 10 client.27355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 20 client.27355 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8af863d0 0x563f8af887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 4 client.27355 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.433+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f8aeb7e10 con 0x563f8af863d0 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.434+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8af863d0 0x563f8af887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.434+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8af863d0 0x563f8af887f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49cdf12fb81faf2 server_cookie=4e0a99a87bb2fadb in_seq=0 out_seq=0 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.434+0000 7f0cbbfff700 10 client.27355 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.435+0000 7f0cbbfff700 1 -- 172.21.15.26:0/3737431967 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f8aeb7e10 con 0x563f8af863d0 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.435+0000 7f0cbbfff700 10 client.27355 handle_command_reply: tid=0 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.435+0000 7f0cf655bb80 1 client.27355 shutdown 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 msgr2=0x563f8af762a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0012720 tx=0x7f0cd0012a30 comp rx=0 tx=0).stop 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 shutdown_connections 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8af863d0 0x563f8af887f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af76890 0x563f8af73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af6b0c0 0x563f8af762a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/3737431967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cbbfff700 0 client.27355 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 >> 172.21.15.26:0/3737431967 conn(0x563f8aee0260 msgr2=0x563f8af66fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 shutdown_connections 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 1 -- 172.21.15.26:0/3737431967 wait complete. 2023-10-06T21:25:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.436+0000 7f0cf655bb80 20 client.27355 trim_cache size 0 max 16384 2023-10-06T21:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 Processor -- start 2023-10-06T21:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 -- start start 2023-10-06T21:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8ad3f920 con 0x563f8af6b9e0 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8af744d0 con 0x563f8af6b0c0 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.463+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8af7d920 con 0x563f8af76890 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60412/0 (socket says 172.21.15.26:60412) 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 -- 172.21.15.26:0/2781835823 learned_addr learned my addr 172.21.15.26:0/2781835823 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 -- 172.21.15.26:0/2781835823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 msgr2=0x563f8af6dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af6dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 -- 172.21.15.26:0/2781835823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 msgr2=0x563f8af6d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce099b700 1 -- 172.21.15.26:0/2781835823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8ad10250 con 0x563f8af6b0c0 2023-10-06T21:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.464+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.465+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005d10 tx=0x7f0cd0009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f42fccb90d14f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.465+0000 7f0cc77fe700 1 -- 172.21.15.26:0/2781835823 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0017030 con 0x563f8af6b0c0 2023-10-06T21:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.465+0000 7f0cc77fe700 1 -- 172.21.15.26:0/2781835823 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd0018040 con 0x563f8af6b0c0 2023-10-06T21:25:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.465+0000 7f0cc77fe700 1 -- 172.21.15.26:0/2781835823 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000fcc0 con 0x563f8af6b0c0 2023-10-06T21:25:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2781835823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 msgr2=0x563f8af7d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005d10 tx=0x7f0cd0009990 comp rx=0 tx=0).stop 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2781835823 shutdown_connections 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b0c0 0x563f8af7d3e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2781835823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b9e0 0x563f8af6d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2781835823 >> 172.21.15.26:0/2781835823 conn(0x563f8aee0260 msgr2=0x563f8af6cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.466+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2781835823 shutdown_connections 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.467+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2781835823 wait complete. 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.467+0000 7f0cf655bb80 1 -- start start 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.467+0000 7f0cf655bb80 1 Processor -- start 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b0c0 0x563f8af7d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af7baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8af7c190 con 0x563f8af6b0c0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8af7c300 con 0x563f8af6b9e0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8afda200 con 0x563f8af76890 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.468+0000 7f0cf655bb80 10 client.0 _ll_register_callbacks cb 0x7f0cd0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:60416/0 (socket says 172.21.15.26:60416) 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 -- 172.21.15.26:0/2238457874 learned_addr learned my addr 172.21.15.26:0/2238457874 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b0c0 0x563f8af7d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0ce119c700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af7baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 -- 172.21.15.26:0/2238457874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 msgr2=0x563f8af7baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af7baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 -- 172.21.15.26:0/2238457874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b0c0 msgr2=0x563f8af7d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b0c0 0x563f8af7d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 -- 172.21.15.26:0/2238457874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f8acd2c60 con 0x563f8af6b9e0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cdbfff700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc80ca270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169ddcd451a2ebc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.469+0000 7f0cbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8099870 con 0x563f8af6b9e0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd4007800 con 0x563f8af6b9e0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0cc809a1a0 con 0x563f8af6b9e0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc8005650 con 0x563f8af6b9e0 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cf655bb80 10 client.27527 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x563f8ace82b0 con 0x563f8af6b9e0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.470+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd4007800 con 0x563f8af6b9e0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0cc80ca490 con 0x563f8af6b9e0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 10 client.27527 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 10 client.27527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 20 client.27527 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8afdb7d0 0x563f8afddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 4 client.27527 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f8aeb7b30 con 0x563f8afdb7d0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.471+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8afdb7d0 0x563f8afddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0ce099b700 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8afdb7d0 0x563f8afddbf0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691effe9bfec2a11 server_cookie=a143f50c640cac2 in_seq=0 out_seq=0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cbbfff700 10 client.27527 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cbbfff700 1 -- 172.21.15.26:0/2238457874 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563f8aeb7b30 con 0x563f8afdb7d0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cbbfff700 10 client.27527 handle_command_reply: tid=0 2023-10-06T21:25:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 client.27527 shutdown 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 msgr2=0x563f8af7b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc80ca270 comp rx=0 tx=0).stop 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 shutdown_connections 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f8af76890 0x563f8af7baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563f8af6b0c0 0x563f8af7d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x563f8afdb7d0 0x563f8afddbf0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 --2- 172.21.15.26:0/2238457874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f8af6b9e0 0x563f8af7b560 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 >> 172.21.15.26:0/2238457874 conn(0x563f8aee0260 msgr2=0x563f8af6d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.472+0000 7f0cbbfff700 0 client.27527 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 shutdown_connections 2023-10-06T21:25:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0cf655bb80 1 -- 172.21.15.26:0/2238457874 wait complete. 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0cf655bb80 20 client.27527 trim_cache size 0 max 16384 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:25:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:25:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:25:23.480 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:25:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 msgr2=0x7f0ccc0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 0x7f0ccc0a3bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4029000 tx=0x7f0cd4023000 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 msgr2=0x7f0cdc19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.473+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c920 tx=0x7f0cc800cc30 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 shutdown_connections 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cdc111bc0 0x7f0cdc1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f0ccc0a1730 0x7f0ccc0a3bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cdc0fef30 0x7f0cdc19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 --2- 172.21.15.26:0/17898193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cdc0ff850 0x7f0cdc19db00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 >> 172.21.15.26:0/17898193 conn(0x7f0cdc076ed0 msgr2=0x7f0cdc110610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 shutdown_connections 2023-10-06T21:25:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:23.474+0000 7f0ce2bff700 1 -- 172.21.15.26:0/17898193 wait complete. 2023-10-06T21:25:23.504 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:25:24.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:24 smithi167 ceph-mon[134494]: pgmap v474: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 666 op/s 2023-10-06T21:25:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:24 smithi026 ceph-mon[98490]: pgmap v474: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 666 op/s 2023-10-06T21:25:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:24 smithi078 ceph-mon[134896]: pgmap v474: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 12 MiB/s wr, 666 op/s 2023-10-06T21:25:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:26 smithi167 ceph-mon[134494]: pgmap v475: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 566 op/s 2023-10-06T21:25:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:26 smithi026 ceph-mon[98490]: pgmap v475: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 566 op/s 2023-10-06T21:25:27.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:26 smithi078 ceph-mon[134896]: pgmap v475: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 566 op/s 2023-10-06T21:25:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 885.0 s, 1073.0 tps, lat 93.369 ms stddev 102.369 2023-10-06T21:25:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:28 smithi167 ceph-mon[134494]: pgmap v476: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 791 op/s 2023-10-06T21:25:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:28 smithi026 ceph-mon[98490]: pgmap v476: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 791 op/s 2023-10-06T21:25:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:28 smithi078 ceph-mon[134896]: pgmap v476: 97 pgs: 97 active+clean; 4.8 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 14 MiB/s wr, 791 op/s 2023-10-06T21:25:31.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:30 smithi167 ceph-mon[134494]: pgmap v477: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:25:31.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:30 smithi026 ceph-mon[98490]: pgmap v477: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:25:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:30 smithi078 ceph-mon[134896]: pgmap v477: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.743+0000 7f33ae3ff700 1 -- 172.21.15.26:0/3392226738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80fef60 msgr2=0x7f33a80ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.743+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/3392226738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80fef60 0x7f33a80ff3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f339c0098f0 tx=0x7f339c009c00 comp rx=0 tx=0).stop 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 -- 172.21.15.26:0/3392226738 shutdown_connections 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/3392226738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a80ff920 0x7f33a810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/3392226738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a8106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/3392226738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80fef60 0x7f33a80ff3e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 -- 172.21.15.26:0/3392226738 >> 172.21.15.26:0/3392226738 conn(0x7f33a8076e70 msgr2=0x7f33a80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 -- 172.21.15.26:0/3392226738 shutdown_connections 2023-10-06T21:25:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.744+0000 7f33ae3ff700 1 -- 172.21.15.26:0/3392226738 wait complete. 2023-10-06T21:25:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.746+0000 7f33ae3ff700 1 Processor -- start 2023-10-06T21:25:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.746+0000 7f33ae3ff700 1 -- start start 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a80fef60 0x7f33a8072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80ff920 0x7f33a80729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806ba40 con 0x7f33a80fef60 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806bbb0 con 0x7f33a80ff920 2023-10-06T21:25:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ae3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a806bd70 con 0x7f33a8105fb0 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36420/0 (socket says 172.21.15.26:36420) 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.747+0000 7f33ac99c700 1 -- 172.21.15.26:0/505542157 learned_addr learned my addr 172.21.15.26:0/505542157 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33a77fe700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80ff920 0x7f33a80729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33ac99c700 1 -- 172.21.15.26:0/505542157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80ff920 msgr2=0x7f33a80729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33ac99c700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80ff920 0x7f33a80729b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33ac99c700 1 -- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a80fef60 msgr2=0x7f33a8072430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33ac99c700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a80fef60 0x7f33a8072430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.748+0000 7f33ac99c700 1 -- 172.21.15.26:0/505542157 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0009d00 con 0x7f33a8105fb0 2023-10-06T21:25:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33ac99c700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c340 tx=0x7f33a000c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33a57fa700 1 -- 172.21.15.26:0/505542157 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000e070 con 0x7f33a8105fb0 2023-10-06T21:25:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33a57fa700 1 -- 172.21.15.26:0/505542157 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a000f040 con 0x7f33a8105fb0 2023-10-06T21:25:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33a57fa700 1 -- 172.21.15.26:0/505542157 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0007b60 con 0x7f33a8105fb0 2023-10-06T21:25:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c009520 con 0x7f33a8105fb0 2023-10-06T21:25:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.749+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a80701a0 con 0x7f33a8105fb0 2023-10-06T21:25:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.758+0000 7f33a57fa700 1 -- 172.21.15.26:0/505542157 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f33a00092d0 con 0x7f33a8105fb0 2023-10-06T21:25:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.758+0000 7f33c1d50b80 1 Processor -- start 2023-10-06T21:25:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.758+0000 7f33c1d50b80 1 -- start start 2023-10-06T21:25:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b51c400 con 0x55908b5be3d0 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b50dbc0 con 0x55908b5b2c00 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b50b4e0 con 0x55908b5b3520 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36430/0 (socket says 172.21.15.26:36430) 2023-10-06T21:25:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33a7fff700 1 -- 172.21.15.26:0/1957355088 learned_addr learned my addr 172.21.15.26:0/1957355088 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.759+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 -- 172.21.15.26:0/1957355088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 msgr2=0x55908b5b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 -- 172.21.15.26:0/1957355088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 msgr2=0x55908b5c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 -- 172.21.15.26:0/1957355088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55908b331d80 con 0x55908b5b3520 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a57fa700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 0x7f33900a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a57fa700 1 -- 172.21.15.26:0/505542157 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f33a00ccd70 con 0x7f33a8105fb0 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.760+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3398009b90 tx=0x7f3398009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e797fecfb957962c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.761+0000 7f338e7fc700 1 -- 172.21.15.26:0/1957355088 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800eb00 con 0x55908b5b3520 2023-10-06T21:25:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.761+0000 7f338e7fc700 1 -- 172.21.15.26:0/1957355088 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3398004cf0 con 0x55908b5b3520 2023-10-06T21:25:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.761+0000 7f338e7fc700 1 -- 172.21.15.26:0/1957355088 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398005580 con 0x55908b5b3520 2023-10-06T21:25:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.762+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1957355088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 msgr2=0x55908b5bde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.762+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3398009b90 tx=0x7f3398009ea0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.763+0000 7f33a77fe700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 0x7f33900a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1957355088 shutdown_connections 2023-10-06T21:25:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1957355088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5bde90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1957355088 >> 172.21.15.26:0/1957355088 conn(0x55908b528200 msgr2=0x55908b5af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.764+0000 7f33a77fe700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 0x7f33900a3c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f339c009fd0 tx=0x7f339c000bc0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.765+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1957355088 shutdown_connections 2023-10-06T21:25:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.765+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1957355088 wait complete. 2023-10-06T21:25:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.765+0000 7f33c1d50b80 1 -- start start 2023-10-06T21:25:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 Processor -- start 2023-10-06T21:25:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5b3520 0x55908b5bb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b506dd0 con 0x55908b5b3520 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b4a36b0 con 0x55908b5be3d0 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b4a3350 con 0x55908b5b2c00 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.766+0000 7f33c1d50b80 10 client.0 _ll_register_callbacks cb 0x55908b3007a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42148/0 (socket says 172.21.15.26:42148) 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 -- 172.21.15.26:0/1399110615 learned_addr learned my addr 172.21.15.26:0/1399110615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5b3520 0x55908b5bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 -- 172.21.15.26:0/1399110615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 msgr2=0x55908b5bdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5bdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 -- 172.21.15.26:0/1399110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5b3520 msgr2=0x55908b5bb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5b3520 0x55908b5bb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 -- 172.21.15.26:0/1399110615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3398009870 con 0x55908b5be3d0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33ac99c700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f33a00961a0 tx=0x7f33a009a3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c856331a06635b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.767+0000 7f33877fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.768+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a009ddf0 con 0x55908b5be3d0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.768+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f33a00a2030 con 0x55908b5be3d0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.768+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55908b5a6d50 con 0x55908b5be3d0 2023-10-06T21:25:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.768+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a0099190 con 0x55908b5be3d0 2023-10-06T21:25:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.770+0000 7f33c1d50b80 10 client.27536 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.770+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55908b31ac00 con 0x55908b5be3d0 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55908b5a6d50 con 0x55908b5be3d0 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33a0095a90 con 0x55908b5be3d0 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 10 client.27536 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 10 client.27536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 20 client.27536 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b5cdee0 0x55908b5d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 4 client.27536 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.774+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55908b4ffdb0 con 0x55908b5cdee0 2023-10-06T21:25:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.775+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b5cdee0 0x55908b5d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.775+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b5cdee0 0x55908b5d0300 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55741f1d1f7e33a2 server_cookie=5038bdb2397a4456 in_seq=0 out_seq=0 2023-10-06T21:25:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.775+0000 7f33877fe700 10 client.27536 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33877fe700 1 -- 172.21.15.26:0/1399110615 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55908b4ffdb0 con 0x55908b5cdee0 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33877fe700 10 client.27536 handle_command_reply: tid=0 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33c1d50b80 1 client.27536 shutdown 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 msgr2=0x55908b5bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f33a00961a0 tx=0x7f33a009a3a0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 shutdown_connections 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.776+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b5cdee0 0x55908b5d0300 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5be3d0 0x55908b5bb700 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5bdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33877fe700 0 client.27536 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1399110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5b3520 0x55908b5bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 >> 172.21.15.26:0/1399110615 conn(0x55908b528200 msgr2=0x55908b5aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 shutdown_connections 2023-10-06T21:25:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1399110615 wait complete. 2023-10-06T21:25:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.777+0000 7f33c1d50b80 20 client.27536 trim_cache size 0 max 16384 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 Processor -- start 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 -- start start 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 0x55908b5b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b3878c0 con 0x55908b5be3d0 2023-10-06T21:25:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b5afdc0 con 0x55908b5b3520 2023-10-06T21:25:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b5b6ed0 con 0x55908b5b2c00 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 0x55908b5b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 0x55908b5b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42160/0 (socket says 172.21.15.26:42160) 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:36452/0 (socket says 172.21.15.26:36452) 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.799+0000 7f33a77fe700 1 -- 172.21.15.26:0/3412972949 learned_addr learned my addr 172.21.15.26:0/3412972949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 -- 172.21.15.26:0/3412972949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 msgr2=0x55908b5b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 0x55908b5b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 -- 172.21.15.26:0/3412972949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 msgr2=0x55908b5aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 -- 172.21.15.26:0/3412972949 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55908b3581f0 con 0x55908b5b2c00 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.800+0000 7f33a7fff700 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3398005950 tx=0x7f3398009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a8b086df1874eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.801+0000 7f338effd700 1 -- 172.21.15.26:0/3412972949 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339802c030 con 0x55908b5b2c00 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.801+0000 7f338effd700 1 -- 172.21.15.26:0/3412972949 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3398028de0 con 0x55908b5b2c00 2023-10-06T21:25:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.801+0000 7f338effd700 1 -- 172.21.15.26:0/3412972949 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800ebf0 con 0x55908b5b2c00 2023-10-06T21:25:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 -- 172.21.15.26:0/3412972949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 msgr2=0x55908b5b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3398005950 tx=0x7f3398009400 comp rx=0 tx=0).stop 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 -- 172.21.15.26:0/3412972949 shutdown_connections 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b2c00 0x55908b5b6390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/3412972949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b3520 0x55908b5b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 -- 172.21.15.26:0/3412972949 >> 172.21.15.26:0/3412972949 conn(0x55908b528200 msgr2=0x55908b5c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 -- 172.21.15.26:0/3412972949 shutdown_connections 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.802+0000 7f33c1d50b80 1 -- 172.21.15.26:0/3412972949 wait complete. 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.803+0000 7f33c1d50b80 1 -- start start 2023-10-06T21:25:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.803+0000 7f33c1d50b80 1 Processor -- start 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b5b8750 con 0x55908b5be3d0 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b621d10 con 0x55908b5b2c00 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908b621e50 con 0x55908b5b3520 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33c1d50b80 10 client.0 _ll_register_callbacks cb 0x7f33a0099ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33ac99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:42168/0 (socket says 172.21.15.26:42168) 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.804+0000 7f33a7fff700 1 -- 172.21.15.26:0/1442730314 learned_addr learned my addr 172.21.15.26:0/1442730314 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 -- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 msgr2=0x55908b5c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 -- 172.21.15.26:0/1442730314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 msgr2=0x55908b5b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 -- 172.21.15.26:0/1442730314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339801c040 con 0x55908b5b3520 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33a77fe700 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5b7900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55908b5bc8a0 tx=0x7f339c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ffe7558b59b3bab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33877fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c02e050 con 0x55908b5b3520 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3398004cf0 con 0x55908b5b3520 2023-10-06T21:25:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.805+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f339c005450 con 0x55908b5b3520 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c005610 con 0x55908b5b3520 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 10 client.27379 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55908b330250 con 0x55908b5b3520 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3398004cf0 con 0x55908b5b3520 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f339c03c070 con 0x55908b5b3520 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 10 client.27379 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 10 client.27379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 20 client.27379 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b623560 0x55908b625980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 4 client.27379 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55908b4ffad0 con 0x55908b623560 2023-10-06T21:25:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.806+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b623560 0x55908b625980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.807+0000 7f33a7fff700 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b623560 0x55908b625980 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4406a15d0eb5691 server_cookie=969d4ee688db282a in_seq=0 out_seq=0 2023-10-06T21:25:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.807+0000 7f33877fe700 10 client.27379 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.819+0000 7f33877fe700 1 -- 172.21.15.26:0/1442730314 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55908b4ffad0 con 0x55908b623560 2023-10-06T21:25:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.819+0000 7f33877fe700 10 client.27379 handle_command_reply: tid=0 2023-10-06T21:25:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.819+0000 7f33c1d50b80 1 client.27379 shutdown 2023-10-06T21:25:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 msgr2=0x55908b5b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5b7900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55908b5bc8a0 tx=0x7f339c005e00 comp rx=0 tx=0).stop 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 shutdown_connections 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55908b5be3d0 0x55908b5b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55908b623560 0x55908b625980 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55908b5b2c00 0x55908b5c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33877fe700 0 client.27379 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 --2- 172.21.15.26:0/1442730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55908b5b3520 0x55908b5b7900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 >> 172.21.15.26:0/1442730314 conn(0x55908b528200 msgr2=0x55908b5aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 shutdown_connections 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 1 -- 172.21.15.26:0/1442730314 wait complete. 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.820+0000 7f33c1d50b80 20 client.27379 trim_cache size 0 max 16384 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:25:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:25:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 msgr2=0x7f33900a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 0x7f33900a3c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f339c009fd0 tx=0x7f339c000bc0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 msgr2=0x7f33a806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c340 tx=0x7f33a000c700 comp rx=0 tx=0).stop 2023-10-06T21:25:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 shutdown_connections 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a8105fb0 0x7f33a806b270 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f33900a1740 0x7f33900a3c00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a80fef60 0x7f33a8072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 --2- 172.21.15.26:0/505542157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a80ff920 0x7f33a80729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 >> 172.21.15.26:0/505542157 conn(0x7f33a8076e70 msgr2=0x7f33a80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 shutdown_connections 2023-10-06T21:25:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:31.821+0000 7f33ae3ff700 1 -- 172.21.15.26:0/505542157 wait complete. 2023-10-06T21:25:32.124 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 890.0 s, 1146.4 tps, lat 87.657 ms stddev 101.050 2023-10-06T21:25:33.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:32 smithi167 ceph-mon[134494]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 743 op/s 2023-10-06T21:25:33.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:32 smithi026 ceph-mon[98490]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 743 op/s 2023-10-06T21:25:33.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:32 smithi078 ceph-mon[134896]: pgmap v478: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 743 op/s 2023-10-06T21:25:34.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:34 smithi167 ceph-mon[134494]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-10-06T21:25:34.826 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:34 smithi026 ceph-mon[98490]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-10-06T21:25:34.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:34 smithi078 ceph-mon[134896]: pgmap v479: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-10-06T21:25:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:36 smithi167 ceph-mon[134494]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 580 op/s 2023-10-06T21:25:37.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:36 smithi026 ceph-mon[98490]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 580 op/s 2023-10-06T21:25:37.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:36 smithi078 ceph-mon[134896]: pgmap v480: 97 pgs: 97 active+clean; 4.9 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 580 op/s 2023-10-06T21:25:37.128 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 895.0 s, 875.0 tps, lat 114.066 ms stddev 146.900 2023-10-06T21:25:39.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:38 smithi167 ceph-mon[134494]: pgmap v481: 97 pgs: 97 active+clean; 5.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-10-06T21:25:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:38 smithi026 ceph-mon[98490]: pgmap v481: 97 pgs: 97 active+clean; 5.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-10-06T21:25:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:38 smithi078 ceph-mon[134896]: pgmap v481: 97 pgs: 97 active+clean; 5.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 17 MiB/s wr, 1.19k op/s 2023-10-06T21:25:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:40 smithi167 ceph-mon[134494]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-10-06T21:25:41.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:40 smithi026 ceph-mon[98490]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-10-06T21:25:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:40 smithi078 ceph-mon[134896]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-10-06T21:25:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:progress: 900.0 s, 1168.4 tps, lat 85.681 ms stddev 108.595 2023-10-06T21:25:42.384 INFO:teuthology.orchestra.run.smithi026.stdout:transaction type: 2023-10-06T21:25:42.384 INFO:teuthology.orchestra.run.smithi026.stdout:scaling factor: 32 2023-10-06T21:25:42.384 INFO:teuthology.orchestra.run.smithi026.stdout:query mode: simple 2023-10-06T21:25:42.384 INFO:teuthology.orchestra.run.smithi026.stdout:number of clients: 100 2023-10-06T21:25:42.384 INFO:teuthology.orchestra.run.smithi026.stdout:number of threads: 4 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:duration: 900 s 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:number of transactions actually processed: 1452052 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:latency average = 61.977 ms 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:latency stddev = 104.637 ms 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:tps = 1612.925516 (including connections establishing) 2023-10-06T21:25:42.385 INFO:teuthology.orchestra.run.smithi026.stdout:tps = 1612.936295 (excluding connections establishing) 2023-10-06T21:25:42.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-10-06T21:25:42.799 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:42 smithi026 ceph-mon[98490]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-10-06T21:25:43.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:42 smithi078 ceph-mon[134896]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-10-06T21:25:43.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:42 smithi167 ceph-mon[134494]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-10-06T21:25:44.882 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:44 smithi078 ceph-mon[134896]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-10-06T21:25:45.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:44 smithi167 ceph-mon[134494]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-10-06T21:25:45.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:44 smithi026 ceph-mon[98490]: pgmap v484: 97 pgs: 97 active+clean; 5.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-10-06T21:25:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 -- 172.21.15.26:0/1622285900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10fa90 msgr2=0x7fe53c1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 --2- 172.21.15.26:0/1622285900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10fa90 0x7fe53c1083f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe53801a9a0 tx=0x7fe53801acb0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 -- 172.21.15.26:0/1622285900 shutdown_connections 2023-10-06T21:25:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 --2- 172.21.15.26:0/1622285900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c11a690 0x7fe53c11ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 --2- 172.21.15.26:0/1622285900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c10f620 0x7fe53c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 --2- 172.21.15.26:0/1622285900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10fa90 0x7fe53c1083f0 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe53801a9a0 tx=0x7fe53801acb0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.114+0000 7fe543dff700 1 -- 172.21.15.26:0/1622285900 >> 172.21.15.26:0/1622285900 conn(0x7fe53c06cfa0 msgr2=0x7fe53c06d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.115+0000 7fe543dff700 1 -- 172.21.15.26:0/1622285900 shutdown_connections 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.115+0000 7fe543dff700 1 -- 172.21.15.26:0/1622285900 wait complete. 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.117+0000 7fe543dff700 1 Processor -- start 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.117+0000 7fe543dff700 1 -- start start 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c11a690 0x7fe53c115a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c115f90 0x7fe53c116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c11a460 con 0x7fe53c10f620 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1aa360 con 0x7fe53c11a690 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.118+0000 7fe543dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c1aa4a0 con 0x7fe53c115f90 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.119+0000 7fe541b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.119+0000 7fe541b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58160/0 (socket says 172.21.15.26:58160) 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.119+0000 7fe541b9b700 1 -- 172.21.15.26:0/872638863 learned_addr learned my addr 172.21.15.26:0/872638863 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.119+0000 7fe54239c700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c115f90 0x7fe53c116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.119+0000 7fe54139a700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c11a690 0x7fe53c115a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.120+0000 7fe541b9b700 1 -- 172.21.15.26:0/872638863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c115f90 msgr2=0x7fe53c116440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.120+0000 7fe541b9b700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c115f90 0x7fe53c116440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.120+0000 7fe541b9b700 1 -- 172.21.15.26:0/872638863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c11a690 msgr2=0x7fe53c115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.120+0000 7fe541b9b700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c11a690 0x7fe53c115a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.120+0000 7fe541b9b700 1 -- 172.21.15.26:0/872638863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53801a600 con 0x7fe53c10f620 2023-10-06T21:25:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.121+0000 7fe541b9b700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c950 tx=0x7fe52c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.125+0000 7fe532ffd700 1 -- 172.21.15.26:0/872638863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c007ac0 con 0x7fe53c10f620 2023-10-06T21:25:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.125+0000 7fe532ffd700 1 -- 172.21.15.26:0/872638863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe52c004cf0 con 0x7fe53c10f620 2023-10-06T21:25:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.125+0000 7fe532ffd700 1 -- 172.21.15.26:0/872638863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c005550 con 0x7fe53c10f620 2023-10-06T21:25:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.127+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c1aa750 con 0x7fe53c10f620 2023-10-06T21:25:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.127+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c1aabe0 con 0x7fe53c10f620 2023-10-06T21:25:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.134+0000 7fe55770db80 1 Processor -- start 2023-10-06T21:25:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.134+0000 7fe55770db80 1 -- start start 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad16fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1727c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad0ce400 con 0x5556ad1703d0 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad0bfbc0 con 0x5556ad164c00 2023-10-06T21:25:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.135+0000 7fe55770db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad0bd4e0 con 0x5556ad165520 2023-10-06T21:25:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.136+0000 7fe54239c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.136+0000 7fe54239c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:48786/0 (socket says 172.21.15.26:48786) 2023-10-06T21:25:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.136+0000 7fe54239c700 1 -- 172.21.15.26:0/1261914985 learned_addr learned my addr 172.21.15.26:0/1261914985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.136+0000 7fe541b9b700 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad16fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.139+0000 7fe54139a700 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.152+0000 7fe532ffd700 1 -- 172.21.15.26:0/872638863 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe52c004750 con 0x7fe53c10f620 2023-10-06T21:25:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe54239c700 1 -- 172.21.15.26:0/1261914985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 msgr2=0x5556ad16fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe54239c700 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad16fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe54239c700 1 -- 172.21.15.26:0/1261914985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 msgr2=0x5556ad1727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe54239c700 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe54239c700 1 -- 172.21.15.26:0/1261914985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556acee3d80 con 0x5556ad164c00 2023-10-06T21:25:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe532ffd700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 0x7fe5280a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.154+0000 7fe532ffd700 1 -- 172.21.15.26:0/872638863 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe52c0d29e0 con 0x7fe53c10f620 2023-10-06T21:25:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.164+0000 7fe54239c700 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5340084c0 tx=0x7fe5340087d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e750a2b523cee2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.186+0000 7fe541b9b700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 0x7fe5280a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.188+0000 7fe527fff700 1 -- 172.21.15.26:0/1261914985 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534003c00 con 0x5556ad164c00 2023-10-06T21:25:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.188+0000 7fe527fff700 1 -- 172.21.15.26:0/1261914985 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe534010040 con 0x5556ad164c00 2023-10-06T21:25:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.188+0000 7fe527fff700 1 -- 172.21.15.26:0/1261914985 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534012760 con 0x5556ad164c00 2023-10-06T21:25:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.194+0000 7fe55770db80 1 -- 172.21.15.26:0/1261914985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 msgr2=0x5556ad164fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.194+0000 7fe55770db80 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5340084c0 tx=0x7fe5340087d0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.195+0000 7fe55770db80 1 -- 172.21.15.26:0/1261914985 shutdown_connections 2023-10-06T21:25:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.195+0000 7fe55770db80 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.195+0000 7fe55770db80 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad164fe0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.195+0000 7fe55770db80 1 --2- 172.21.15.26:0/1261914985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad16fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.195+0000 7fe55770db80 1 -- 172.21.15.26:0/1261914985 >> 172.21.15.26:0/1261914985 conn(0x5556ad0da200 msgr2=0x5556ad161080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.211+0000 7fe55770db80 1 -- 172.21.15.26:0/1261914985 shutdown_connections 2023-10-06T21:25:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.235+0000 7fe55770db80 1 -- 172.21.15.26:0/1261914985 wait complete. 2023-10-06T21:25:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.235+0000 7fe55770db80 1 -- start start 2023-10-06T21:25:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.236+0000 7fe55770db80 1 Processor -- start 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad16d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad16dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad0b8dd0 con 0x5556ad165520 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad0556b0 con 0x5556ad1703d0 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad055350 con 0x5556ad164c00 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.250+0000 7fe55770db80 10 client.0 _ll_register_callbacks cb 0x5556aceb27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.261+0000 7fe54139a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.261+0000 7fe541b9b700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 0x7fe5280a3cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0d4e60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.261+0000 7fe54139a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58178/0 (socket says 172.21.15.26:58178) 2023-10-06T21:25:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.261+0000 7fe54139a700 1 -- 172.21.15.26:0/2197343125 learned_addr learned my addr 172.21.15.26:0/2197343125 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.264+0000 7fe54239c700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad16dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.264+0000 7fe54239c700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad16d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.277+0000 7fe54139a700 1 -- 172.21.15.26:0/2197343125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 msgr2=0x5556ad16d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.277+0000 7fe54139a700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad16d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.277+0000 7fe54139a700 1 -- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 msgr2=0x5556ad16dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.277+0000 7fe54139a700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad16dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.277+0000 7fe54139a700 1 -- 172.21.15.26:0/2197343125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe534007d10 con 0x5556ad165520 2023-10-06T21:25:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.287+0000 7fe54139a700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe53c06dd20 tx=0x7fe5380037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af4200cf6f9b11f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.297+0000 7fe5257fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:25:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.297+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53801a090 con 0x5556ad165520 2023-10-06T21:25:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.297+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe538005240 con 0x5556ad165520 2023-10-06T21:25:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.297+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556ad158d50 con 0x5556ad165520 2023-10-06T21:25:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.331+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe538005400 con 0x5556ad165520 2023-10-06T21:25:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.340+0000 7fe55770db80 10 client.17163 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.340+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5556aceccc00 con 0x5556ad165520 2023-10-06T21:25:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.350+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556ad158d50 con 0x5556ad165520 2023-10-06T21:25:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.350+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe538003960 con 0x5556ad165520 2023-10-06T21:25:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 10 client.17163 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 20 client.17163 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad17ffb0 0x5556ad1823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 4 client.17163 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556ad0b1db0 con 0x5556ad17ffb0 2023-10-06T21:25:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.352+0000 7fe54239c700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad17ffb0 0x5556ad1823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.392+0000 7fe54239c700 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad17ffb0 0x5556ad1823d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfffaf7416b4cb6f server_cookie=777137d0764b9688 in_seq=0 out_seq=0 2023-10-06T21:25:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.394+0000 7fe5257fa700 10 client.17163 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.416+0000 7fe5257fa700 1 -- 172.21.15.26:0/2197343125 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556ad0b1db0 con 0x5556ad17ffb0 2023-10-06T21:25:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.416+0000 7fe5257fa700 10 client.17163 handle_command_reply: tid=0 2023-10-06T21:25:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.416+0000 7fe55770db80 1 client.17163 shutdown 2023-10-06T21:25:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.420+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 msgr2=0x5556ad16d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.420+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe53c06dd20 tx=0x7fe5380037c0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 shutdown_connections 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad17ffb0 0x5556ad1823d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad16dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad16d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 --2- 172.21.15.26:0/2197343125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad16d6a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 >> 172.21.15.26:0/2197343125 conn(0x5556ad0da200 msgr2=0x5556ad160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.433+0000 7fe5257fa700 0 client.17163 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.437+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 shutdown_connections 2023-10-06T21:25:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.442+0000 7fe55770db80 1 -- 172.21.15.26:0/2197343125 wait complete. 2023-10-06T21:25:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.442+0000 7fe55770db80 20 client.17163 trim_cache size 0 max 16384 2023-10-06T21:25:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.485+0000 7fe55770db80 1 Processor -- start 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.485+0000 7fe55770db80 1 -- start start 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad1823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad176f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556acf398c0 con 0x5556ad1703d0 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad161dc0 con 0x5556ad164c00 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe55770db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad177460 con 0x5556ad165520 2023-10-06T21:25:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe54239c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe54239c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58192/0 (socket says 172.21.15.26:58192) 2023-10-06T21:25:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.486+0000 7fe54239c700 1 -- 172.21.15.26:0/773505710 learned_addr learned my addr 172.21.15.26:0/773505710 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.492+0000 7fe54239c700 1 -- 172.21.15.26:0/773505710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 msgr2=0x5556ad176f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T21:25:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.492+0000 7fe54239c700 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad176f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.492+0000 7fe54239c700 1 -- 172.21.15.26:0/773505710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 msgr2=0x5556ad1823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.492+0000 7fe54239c700 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad1823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.492+0000 7fe54239c700 1 -- 172.21.15.26:0/773505710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556acf0a1f0 con 0x5556ad1703d0 2023-10-06T21:25:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.515+0000 7fe54239c700 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe534008b20 tx=0x7fe5340087d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93137b23fba766d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.516+0000 7fe530bff700 1 -- 172.21.15.26:0/773505710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534012780 con 0x5556ad1703d0 2023-10-06T21:25:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.516+0000 7fe530bff700 1 -- 172.21.15.26:0/773505710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe534007610 con 0x5556ad1703d0 2023-10-06T21:25:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.516+0000 7fe530bff700 1 -- 172.21.15.26:0/773505710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534023b90 con 0x5556ad1703d0 2023-10-06T21:25:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.517+0000 7fe55770db80 1 -- 172.21.15.26:0/773505710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 msgr2=0x5556ad1684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.517+0000 7fe55770db80 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe534008b20 tx=0x7fe5340087d0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.522+0000 7fe55770db80 1 -- 172.21.15.26:0/773505710 shutdown_connections 2023-10-06T21:25:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.522+0000 7fe55770db80 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad1703d0 0x5556ad1684d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.522+0000 7fe55770db80 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad164c00 0x5556ad1823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.522+0000 7fe55770db80 1 --2- 172.21.15.26:0/773505710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad165520 0x5556ad176f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.522+0000 7fe55770db80 1 -- 172.21.15.26:0/773505710 >> 172.21.15.26:0/773505710 conn(0x5556ad0da200 msgr2=0x5556ad17b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.536+0000 7fe55770db80 1 -- 172.21.15.26:0/773505710 shutdown_connections 2023-10-06T21:25:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.536+0000 7fe55770db80 1 -- 172.21.15.26:0/773505710 wait complete. 2023-10-06T21:25:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.536+0000 7fe55770db80 1 -- start start 2023-10-06T21:25:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.537+0000 7fe55770db80 1 Processor -- start 2023-10-06T21:25:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.537+0000 7fe55770db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.537+0000 7fe55770db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad1cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe55770db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad1c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe55770db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad1c8350 con 0x5556ad165520 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe55770db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad1c84c0 con 0x5556ad1703d0 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe55770db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ad1c8680 con 0x5556ad164c00 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe55770db80 10 client.0 _ll_register_callbacks cb 0x7fe53801a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe54239c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe54239c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:49998/0 (socket says 172.21.15.26:49998) 2023-10-06T21:25:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.538+0000 7fe54239c700 1 -- 172.21.15.26:0/1434937233 learned_addr learned my addr 172.21.15.26:0/1434937233 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.550+0000 7fe54139a700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad1cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.561+0000 7fe54239c700 1 -- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 msgr2=0x5556ad1c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.561+0000 7fe54239c700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad1c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.561+0000 7fe54239c700 1 -- 172.21.15.26:0/1434937233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 msgr2=0x5556ad1cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.561+0000 7fe54239c700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad1cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.561+0000 7fe54239c700 1 -- 172.21.15.26:0/1434937233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556acee3d80 con 0x5556ad164c00 2023-10-06T21:25:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.573+0000 7fe54239c700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe534004940 tx=0x7fe534007c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd657c482dfa0ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.583+0000 7fe5257fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.593+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534004c90 con 0x5556ad164c00 2023-10-06T21:25:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.593+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe534004df0 con 0x5556ad164c00 2023-10-06T21:25:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.602+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556ad0b1ad0 con 0x5556ad164c00 2023-10-06T21:25:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.613+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe534023830 con 0x5556ad164c00 2023-10-06T21:25:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.613+0000 7fe55770db80 10 client.27391 fetch_fsmap learned FSMap version 57 2023-10-06T21:25:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.613+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5556acee2250 con 0x5556ad164c00 2023-10-06T21:25:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.621+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe53403b030 con 0x5556ad164c00 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.621+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 57) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe534005710 con 0x5556ad164c00 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 10 client.27391 fetch_fsmap finished waiting for FSMap version 57 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 10 client.27391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 20 client.27391 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad1d4d80 0x5556ad1d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 4 client.27391 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-06T21:25:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.622+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556acf04f10 con 0x5556ad1d4d80 2023-10-06T21:25:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.627+0000 7fe54139a700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad1d4d80 0x5556ad1d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.689+0000 7fe54139a700 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad1d4d80 0x5556ad1d71a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96759e392df98f00 server_cookie=328f64e0fe130333 in_seq=0 out_seq=0 2023-10-06T21:25:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:45.690+0000 7fe5257fa700 10 client.27391 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:47.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:46 smithi026 ceph-mon[98490]: pgmap v485: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:25:47.138 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:46 smithi078 ceph-mon[134896]: pgmap v485: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:25:47.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:46 smithi167 ceph-mon[134494]: pgmap v485: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 15 MiB/s wr, 1.32k op/s 2023-10-06T21:25:49.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:48 smithi026 ceph-mon[98490]: pgmap v486: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-10-06T21:25:49.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:48 smithi078 ceph-mon[134896]: pgmap v486: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-10-06T21:25:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.263+0000 7fe5257fa700 1 -- 172.21.15.26:0/1434937233 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556acf04f10 con 0x5556ad1d4d80 2023-10-06T21:25:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.263+0000 7fe5257fa700 10 client.27391 handle_command_reply: tid=0 2023-10-06T21:25:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.263+0000 7fe55770db80 1 client.27391 shutdown 2023-10-06T21:25:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.263+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 msgr2=0x5556ad1cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.263+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe534004940 tx=0x7fe534007c60 comp rx=0 tx=0).stop 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 shutdown_connections 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5556ad1d4d80 0x5556ad1d71a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556ad1703d0 0x5556ad1c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556ad164c00 0x5556ad1cd300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 --2- 172.21.15.26:0/1434937233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556ad165520 0x5556ad1cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 >> 172.21.15.26:0/1434937233 conn(0x5556ad0da200 msgr2=0x5556ad1684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe5257fa700 0 client.27391 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 shutdown_connections 2023-10-06T21:25:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.264+0000 7fe55770db80 1 -- 172.21.15.26:0/1434937233 wait complete. 2023-10-06T21:25:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.265+0000 7fe55770db80 20 client.27391 trim_cache size 0 max 16384 2023-10-06T21:25:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2023-10-06T21:25:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.266+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 msgr2=0x7fe5280a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.266+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 0x7fe5280a3cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0d4e60 comp rx=0 tx=0).stop 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.266+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 msgr2=0x7fe53c119c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.266+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c950 tx=0x7fe52c00cd10 comp rx=0 tx=0).stop 2023-10-06T21:25:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 shutdown_connections 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe53c115f90 0x7fe53c116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe5280a1800 0x7fe5280a3cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53c10f620 0x7fe53c119c60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 --2- 172.21.15.26:0/872638863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe53c11a690 0x7fe53c115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 >> 172.21.15.26:0/872638863 conn(0x7fe53c06cfa0 msgr2=0x7fe53c11bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 shutdown_connections 2023-10-06T21:25:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2023-10-06T21:25:49.274+0000 7fe543dff700 1 -- 172.21.15.26:0/872638863 wait complete. 2023-10-06T21:25:49.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:48 smithi167 ceph-mon[134494]: pgmap v486: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 19 MiB/s wr, 1.63k op/s 2023-10-06T21:25:50.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:49 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:50.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:49 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:50.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:49 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:50 smithi078 ceph-mon[134896]: pgmap v487: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-10-06T21:25:51.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:50 smithi167 ceph-mon[134494]: pgmap v487: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-10-06T21:25:51.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:50 smithi026 ceph-mon[98490]: pgmap v487: 97 pgs: 97 active+clean; 5.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-10-06T21:25:51.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/: 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 data 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data: 2023-10-06T21:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:total 31 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 . 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 .. 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 base 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Oct 6 21:10 current_logfiles 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 6 21:10 global 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 log 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_commit_ts 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_dynshmem 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 6 21:10 pg_hba.conf 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 6 21:10 pg_ident.conf 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 pg_logical 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 pg_multixact 2023-10-06T21:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 pg_notify 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_replslot 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_serial 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_snapshots 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 pg_stat 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:25 pg_stat_tmp 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:25 pg_subtrans 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_tblspc 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pg_twophase 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Oct 6 21:25 pg_wal 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:20 pg_xact 2023-10-06T21:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 6 21:10 postgresql.auto.conf 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 6 21:10 postgresql.conf 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Oct 6 21:10 postmaster.opts 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base: 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 . 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 1 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 12346 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 6 21:10 12347 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 pgsql_tmp 2023-10-06T21:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:total 7201 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 . 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 .. 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:10 1259 2023-10-06T21:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 6 21:10 2608 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610 2023-10-06T21:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:10 2619 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2619_fsm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:10 2658 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2696 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:total 7201 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 6 21:10 . 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 .. 2023-10-06T21:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:10 1259 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 6 21:10 2608 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:10 2619 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2619_fsm 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:10 2658 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2696 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:total 608264 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 6 21:10 . 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 .. 2023-10-06T21:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 112 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 113 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 6 21:10 12182 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12182_fsm 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12182_vm 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12184 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12186 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12187_fsm 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12187_vm 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12189 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12191 2023-10-06T21:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12192_fsm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12192_vm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12194 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12196 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12197_fsm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12197_vm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12199 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12201 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12202_fsm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12202_vm 2023-10-06T21:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12204 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12206 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 12207_fsm 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12207_vm 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12209 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12211 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12212 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 12214 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 12216 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 6 21:10 1247 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1247_fsm 2023-10-06T21:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1247_vm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 6 21:10 1249 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1249_fsm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1249_vm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 6 21:10 1255 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1255_fsm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1255_vm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 6 21:25 1259 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1259_fsm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1259_vm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1417_vm 2023-10-06T21:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 1418_vm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 7938048 Oct 6 21:25 16387 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:25 16387_fsm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:25 16387_vm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 688128 Oct 6 21:25 16393 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:25 16393_fsm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:25 16393_vm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 457662464 Oct 6 21:25 16396 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 6 21:25 16396_fsm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:14 16396_vm 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:25 16397 2023-10-06T21:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 524288 Oct 6 21:25 16399 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Oct 6 21:25 16401 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 76234752 Oct 6 21:25 16403 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:25 16403_fsm 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 174 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 175 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2187 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2224_vm 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2328_vm 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336 2023-10-06T21:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2336_vm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2337 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2600 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2600_fsm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2600_vm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2601_fsm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2601_vm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2602 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2602_fsm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2602_vm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2603 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2603_fsm 2023-10-06T21:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2603_vm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2604_vm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2605 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2605_fsm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2605_vm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2606_fsm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2606_vm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2607_fsm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2607_vm 2023-10-06T21:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Oct 6 21:10 2608 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2608_fsm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2608_vm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 6 21:10 2609 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2609_fsm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2609_vm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2610 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2610_fsm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2610_vm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2611_vm 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612 2023-10-06T21:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2612_fsm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2612_vm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2613_vm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2615_fsm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2615_vm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2616_fsm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2616_vm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 6 21:10 2617 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2617_fsm 2023-10-06T21:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2617_vm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 6 21:10 2618 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2618_fsm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2618_vm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Oct 6 21:25 2619 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:20 2619_fsm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2619_vm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2620_vm 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2650 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2651 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2652 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2653 2023-10-06T21:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2654 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2655 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2656 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2657 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Oct 6 21:10 2658 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2659 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2660 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2661 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2662 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2663 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2664 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2665 2023-10-06T21:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2666 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2667 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2668 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2669 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2670 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Oct 6 21:10 2673 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 6 21:10 2674 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 6 21:10 2675 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2678 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2679 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2680 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2681 2023-10-06T21:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2682 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2683 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2684 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2685 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2686 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2687 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2688 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:10 2689 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 6 21:10 2690 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 6 21:10 2691 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2692 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2693 2023-10-06T21:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:20 2696 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2699 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2701 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2702 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2703 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 2704 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2753 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2753_fsm 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2753_vm 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2754 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2755 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2756 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 2757 2023-10-06T21:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2830_vm 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2831 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2832_vm 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2833 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2834_vm 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2835 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2836_vm 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2837 2023-10-06T21:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 6 21:10 2838 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2838_fsm 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2838_vm 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2839 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2840_fsm 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2840_vm 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2841 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2995_vm 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2996 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079 2023-10-06T21:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3079_fsm 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3079_vm 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3080 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3081 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3085 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3118_vm 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3119 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3164 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3256_vm 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3257 2023-10-06T21:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3258 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3350_vm 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3351 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3379 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3380 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3381_vm 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3394 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3394_fsm 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3394_vm 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3395 2023-10-06T21:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3439_vm 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3440 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 6 21:10 3455 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3456 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3456_fsm 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3456_vm 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3466_vm 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3467 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3468 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3501_vm 2023-10-06T21:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3502 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3503 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3534 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3541_fsm 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3541_vm 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3542 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3574 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3575 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3576_vm 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596 2023-10-06T21:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3596_vm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3597 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3598_vm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3599 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3600_fsm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3600_vm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3601_fsm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3601_vm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3602_fsm 2023-10-06T21:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3602_vm 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3603 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3603_fsm 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3603_vm 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3604 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3605 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3606 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3607 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3608 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 6 21:10 3609 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3712 2023-10-06T21:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 3764_fsm 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3764_vm 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3766 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 3767 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3997 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 5002 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 548 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 549 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6102_vm 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6104_vm 2023-10-06T21:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6106_vm 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6110 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6111 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6112 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6113 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6117 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 826_vm 2023-10-06T21:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 827 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 828 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Oct 6 21:10 pg_internal.init 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:10 PG_VERSION 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 6 21:10 .. 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/global: 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:total 574 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 6 21:10 . 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1136 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1136_fsm 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1136_vm 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1137 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1213 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1213_fsm 2023-10-06T21:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1213_vm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1214 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1214_fsm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1214_vm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1232 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 1233 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1260 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1260_fsm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1260_vm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1261 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1261_fsm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1261_vm 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1262 2023-10-06T21:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 1262_fsm 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 1262_vm 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2396 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 6 21:10 2396_fsm 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2396_vm 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2397 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2671 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2672 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2676 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2677 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2694 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2695 2023-10-06T21:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2697 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 6 21:10 2698 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2846 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2846_vm 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2847 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2964 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2964_vm 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2965 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2966 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 2966_vm 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 2967 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3592 2023-10-06T21:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 3592_vm 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 3593 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 4060 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 4060_vm 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 4061 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6000 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6000_vm 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6001 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6002 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6100 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 6100_vm 2023-10-06T21:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6114 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 6115 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:25 pg_control 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 6 21:10 pg_filenode.map 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Oct 6 21:10 pg_internal.init 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/log: 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:total 2 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1121 Oct 6 21:25 postgresql-Fri.log 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:total 1 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 . 2023-10-06T21:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 mappings 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 6 21:25 replorigin_checkpoint 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 snapshots 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 .. 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:52 smithi078 ceph-mon[134896]: pgmap v488: 97 pgs: 97 active+clean; 5.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-10-06T21:25:53.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:52 smithi167 ceph-mon[134494]: pgmap v488: 97 pgs: 97 active+clean; 5.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-10-06T21:25:53.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:52 smithi026 ceph-mon[98490]: pgmap v488: 97 pgs: 97 active+clean; 5.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-10-06T21:25:53.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:25:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/2639361418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb74103300 msgr2=0x7ffb741036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/2639361418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb74103300 0x7ffb741036e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009990 tx=0x7ffb70009ca0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/2639361418 shutdown_connections 2023-10-06T21:25:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/2639361418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb7410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/2639361418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb74103300 0x7ffb741036e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/2639361418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103cb0 0x7ffb74107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/2639361418 >> 172.21.15.26:0/2639361418 conn(0x7ffb740feb40 msgr2=0x7ffb74100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/2639361418 shutdown_connections 2023-10-06T21:25:53.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.748+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/2639361418 wait complete. 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.749+0000 7ffb7b5ff700 1 Processor -- start 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.749+0000 7ffb7b5ff700 1 -- start start 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 0x7ffb741991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb741996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 0x7ffb7419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7419e5c0 con 0x7ffb74108420 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7419e730 con 0x7ffb7419db40 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7b5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7419e8f0 con 0x7ffb74103300 2023-10-06T21:25:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 0x7ffb741991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb79b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 0x7ffb7419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 0x7ffb741991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:55470/0 (socket says 172.21.15.26:55470) 2023-10-06T21:25:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb741996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.750+0000 7ffb7939b700 1 -- 172.21.15.26:0/289032260 learned_addr learned my addr 172.21.15.26:0/289032260 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 -- 172.21.15.26:0/289032260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 msgr2=0x7ffb741991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 0x7ffb741991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 -- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 msgr2=0x7ffb741996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb741996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 -- 172.21.15.26:0/289032260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009c40 con 0x7ffb7419db40 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb741996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:25:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb79b9c700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 0x7ffb7419dfc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00efb0 tx=0x7ffb6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb6a7fc700 1 -- 172.21.15.26:0/289032260 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c003a00 con 0x7ffb7419db40 2023-10-06T21:25:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb6a7fc700 1 -- 172.21.15.26:0/289032260 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb6c00f040 con 0x7ffb7419db40 2023-10-06T21:25:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb6a7fc700 1 -- 172.21.15.26:0/289032260 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004540 con 0x7ffb7419db40 2023-10-06T21:25:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.751+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb700095f0 con 0x7ffb7419db40 2023-10-06T21:25:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.752+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb741a6150 con 0x7ffb7419db40 2023-10-06T21:25:53.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.756+0000 7ffb6a7fc700 1 -- 172.21.15.26:0/289032260 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffb6c003bf0 con 0x7ffb7419db40 2023-10-06T21:25:53.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.757+0000 7ffb8ef34b80 1 Processor -- start 2023-10-06T21:25:53.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.757+0000 7ffb8ef34b80 1 -- start start 2023-10-06T21:25:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269340c0 0x55b8269344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82693f890 0x55b826941c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82689d460 con 0x55b82693f890 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82688ec20 con 0x55b8269349e0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82688c540 con 0x55b8269340c0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb6a7fc700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 0x7ffb600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb6a7fc700 1 -- 172.21.15.26:0/289032260 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffb6c0cccc0 con 0x7ffb7419db40 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57292/0 (socket says 172.21.15.26:57292) 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb7939b700 1 -- 172.21.15.26:0/548483458 learned_addr learned my addr 172.21.15.26:0/548483458 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.758+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269340c0 0x55b8269344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82693f890 0x55b826941c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb79b9c700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 0x7ffb600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 -- 172.21.15.26:0/548483458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269340c0 msgr2=0x55b8269344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269340c0 0x55b8269344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 -- 172.21.15.26:0/548483458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82693f890 msgr2=0x55b826941c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82693f890 0x55b826941c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 -- 172.21.15.26:0/548483458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8266b2de0 con 0x55b8269349e0 2023-10-06T21:25:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb7939b700 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b640 tx=0x7ffb7000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9cf4eff6b8ffbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb5f7fe700 1 -- 172.21.15.26:0/548483458 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001d030 con 0x55b8269349e0 2023-10-06T21:25:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb79b9c700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 0x7ffb600a3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0a1a90 tx=0x7ffb6c003f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb5f7fe700 1 -- 172.21.15.26:0/548483458 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7001e040 con 0x55b8269349e0 2023-10-06T21:25:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.759+0000 7ffb5f7fe700 1 -- 172.21.15.26:0/548483458 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70004aa0 con 0x55b8269349e0 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/548483458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 msgr2=0x55b82693f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b640 tx=0x7ffb7000b670 comp rx=0 tx=0).stop 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/548483458 shutdown_connections 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b82693f890 0x55b826941c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269340c0 0x55b8269344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/548483458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693f350 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/548483458 >> 172.21.15.26:0/548483458 conn(0x55b8268a9260 msgr2=0x55b826930450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/548483458 shutdown_connections 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.760+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/548483458 wait complete. 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.761+0000 7ffb8ef34b80 1 -- start start 2023-10-06T21:25:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.761+0000 7ffb8ef34b80 1 Processor -- start 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b82693f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82693f890 0x55b82693cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826887e30 con 0x55b8269340c0 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826824710 con 0x55b8269349e0 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8268243b0 con 0x55b82693f890 2023-10-06T21:25:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb8ef34b80 10 client.0 _ll_register_callbacks cb 0x55b826681800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b82693f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb7939b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57306/0 (socket says 172.21.15.26:57306) 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.762+0000 7ffb7939b700 1 -- 172.21.15.26:0/1003098297 learned_addr learned my addr 172.21.15.26:0/1003098297 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82693f890 0x55b82693cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 -- 172.21.15.26:0/1003098297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82693f890 msgr2=0x55b82693cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82693f890 0x55b82693cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 -- 172.21.15.26:0/1003098297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 msgr2=0x55b82693f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b82693f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 -- 172.21.15.26:0/1003098297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb74110810 con 0x55b8269349e0 2023-10-06T21:25:53.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000c00 tx=0x7ffb7000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683d72f5939caa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001d070 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b826927f10 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffb70004850 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70004a10 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb8ef34b80 10 client.27560 fetch_fsmap learned FSMap version 58 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.763+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55b82669bc60 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b826927f10 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ffb70034070 con 0x55b8269349e0 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 10 client.27560 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T21:25:53.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 10 client.27560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 20 client.27560 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b82694edf0 0x55b8269511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 4 client.27560 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b826880e10 con 0x55b82694edf0 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.764+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b82694edf0 0x55b8269511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.765+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b82694edf0 0x55b8269511e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba67326630a1d186 server_cookie=77167480ea1d72b0 in_seq=0 out_seq=0 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.765+0000 7ffb5cff9700 10 client.27560 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.765+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1003098297 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b826880e10 con 0x55b82694edf0 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.765+0000 7ffb5cff9700 10 client.27560 handle_command_reply: tid=0 2023-10-06T21:25:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 client.27560 shutdown 2023-10-06T21:25:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 msgr2=0x55b82693c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000c00 tx=0x7ffb7000bf10 comp rx=0 tx=0).stop 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 shutdown_connections 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82693f890 0x55b82693cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b82693f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b82694edf0 0x55b8269511e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1003098297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8269349e0 0x55b82693c560 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 >> 172.21.15.26:0/1003098297 conn(0x55b8268a9260 msgr2=0x55b82692ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb5cff9700 0 client.27560 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 shutdown_connections 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1003098297 wait complete. 2023-10-06T21:25:53.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.766+0000 7ffb8ef34b80 20 client.27560 trim_cache size 0 max 16384 2023-10-06T21:25:53.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 Processor -- start 2023-10-06T21:25:53.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 -- start start 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b82693ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b8269378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826708920 con 0x55b8269340c0 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82693d280 con 0x55b82693f890 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82692a550 con 0x55b8269349e0 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.787+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36980/0 (socket says 172.21.15.26:36980) 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 -- 172.21.15.26:0/782418928 learned_addr learned my addr 172.21.15.26:0/782418928 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b8269378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb7939b700 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b82693ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 -- 172.21.15.26:0/782418928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 msgr2=0x55b82693ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b82693ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 -- 172.21.15.26:0/782418928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 msgr2=0x55b8269378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b8269378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 -- 172.21.15.26:0/782418928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8266d9250 con 0x55b8269340c0 2023-10-06T21:25:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.788+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb64005950 tx=0x7ffb64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edda2ab164f8bdc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.789+0000 7ffb5ffff700 1 -- 172.21.15.26:0/782418928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64023a40 con 0x55b8269340c0 2023-10-06T21:25:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.789+0000 7ffb5ffff700 1 -- 172.21.15.26:0/782418928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb64004cf0 con 0x55b8269340c0 2023-10-06T21:25:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.789+0000 7ffb5ffff700 1 -- 172.21.15.26:0/782418928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400fc10 con 0x55b8269340c0 2023-10-06T21:25:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.789+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/782418928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 msgr2=0x55b826936f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.789+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb64005950 tx=0x7ffb64009260 comp rx=0 tx=0).stop 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/782418928 shutdown_connections 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b8269378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826936f60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/782418928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b82693ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/782418928 >> 172.21.15.26:0/782418928 conn(0x55b8268a9260 msgr2=0x55b82692ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/782418928 shutdown_connections 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/782418928 wait complete. 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.790+0000 7ffb8ef34b80 1 -- start start 2023-10-06T21:25:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 Processor -- start 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826946050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b826944400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b826944940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826946960 con 0x55b8269340c0 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826944e80 con 0x55b82693f890 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b826944ff0 con 0x55b8269349e0 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.791+0000 7ffb8ef34b80 10 client.0 _ll_register_callbacks cb 0x7ffb70009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:25:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826946050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb78b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826946050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36986/0 (socket says 172.21.15.26:36986) 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb78b9a700 1 -- 172.21.15.26:0/1298063660 learned_addr learned my addr 172.21.15.26:0/1298063660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b826944400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b826944940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 -- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 msgr2=0x55b826944940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b826944940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 -- 172.21.15.26:0/1298063660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 msgr2=0x55b826946050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826946050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 -- 172.21.15.26:0/1298063660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb64019040 con 0x55b8269349e0 2023-10-06T21:25:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb7939b700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b826944400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffb70037040 tx=0x7ffb70004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5000165d2bb66ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.792+0000 7ffb5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001d070 con 0x55b8269349e0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb7001e040 con 0x55b8269349e0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ffb70000e40 con 0x55b8269349e0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb700056d0 con 0x55b8269349e0 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb8ef34b80 10 client.27409 fetch_fsmap learned FSMap version 58 2023-10-06T21:25:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55b8266b12b0 con 0x55b8269349e0 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.793+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7001e040 con 0x55b8269349e0 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ffb7000b850 con 0x55b8269349e0 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 10 client.27409 fetch_fsmap finished waiting for FSMap version 58 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 10 client.27409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 20 client.27409 populate_metadata read hostname 'smithi026' 2023-10-06T21:25:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b8269a3b20 0x55b8269a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 4 client.27409 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b826880b30 con 0x55b8269a3b20 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b8269a3b20 0x55b8269a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb78b9a700 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b8269a3b20 0x55b8269a5f10 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5117e9ceae16c0 server_cookie=bc4d57c002456bd7 in_seq=0 out_seq=0 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.794+0000 7ffb5cff9700 10 client.27409 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb5cff9700 1 -- 172.21.15.26:0/1298063660 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b826880b30 con 0x55b8269a3b20 2023-10-06T21:25:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb5cff9700 10 client.27409 handle_command_reply: tid=0 2023-10-06T21:25:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 client.27409 shutdown 2023-10-06T21:25:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 msgr2=0x55b826944400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b826944400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffb70037040 tx=0x7ffb70004f60 comp rx=0 tx=0).stop 2023-10-06T21:25:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 shutdown_connections 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55b8269a3b20 0x55b8269a5f10 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b82693f890 0x55b826944940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b8269340c0 0x55b826946050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 --2- 172.21.15.26:0/1298063660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8269349e0 0x55b826944400 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb5cff9700 0 client.27409 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:25:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 >> 172.21.15.26:0/1298063660 conn(0x55b8268a9260 msgr2=0x55b8269378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 shutdown_connections 2023-10-06T21:25:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 1 -- 172.21.15.26:0/1298063660 wait complete. 2023-10-06T21:25:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.795+0000 7ffb8ef34b80 20 client.27409 trim_cache size 0 max 16384 2023-10-06T21:25:53.806 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:25:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-06T21:25:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:25:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:25:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:25:53.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:25:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:25:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:25:53.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:25:53.808 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:25:53.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 msgr2=0x7ffb600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 0x7ffb600a3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0a1a90 tx=0x7ffb6c003f90 comp rx=0 tx=0).stop 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 msgr2=0x7ffb7419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 0x7ffb7419dfc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00efb0 tx=0x7ffb6c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 shutdown_connections 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb7419db40 0x7ffb7419dfc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb74103300 0x7ffb741991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb600a1670 0x7ffb600a3b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 --2- 172.21.15.26:0/289032260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb74108420 0x7ffb741996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:25:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 >> 172.21.15.26:0/289032260 conn(0x7ffb740feb40 msgr2=0x7ffb740ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:25:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 shutdown_connections 2023-10-06T21:25:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:25:53.796+0000 7ffb7b5ff700 1 -- 172.21.15.26:0/289032260 wait complete. 2023-10-06T21:25:53.820 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:25:55.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:54 smithi078 ceph-mon[134896]: pgmap v489: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-10-06T21:25:55.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:54 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:54 smithi167 ceph-mon[134494]: pgmap v489: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-10-06T21:25:55.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:54 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:55.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:54 smithi026 ceph-mon[98490]: pgmap v489: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-10-06T21:25:55.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:54 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:25:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:56 smithi078 ceph-mon[134896]: pgmap v490: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-10-06T21:25:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:56 smithi167 ceph-mon[134494]: pgmap v490: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-10-06T21:25:57.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:56 smithi026 ceph-mon[98490]: pgmap v490: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-10-06T21:25:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:58 smithi078 ceph-mon[134896]: pgmap v491: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.2 MiB/s wr, 874 op/s 2023-10-06T21:25:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:58 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:58 smithi167 ceph-mon[134494]: pgmap v491: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.2 MiB/s wr, 874 op/s 2023-10-06T21:25:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:58 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:59.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:58 smithi026 ceph-mon[98490]: pgmap v491: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.2 MiB/s wr, 874 op/s 2023-10-06T21:25:59.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:58 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:25:59.577 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 .. 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 . 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 members 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 offsets 2023-10-06T21:25:59.578 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 .. 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:10 .. 2023-10-06T21:25:59.579 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:total 8 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:10 . 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 6 21:10 0000 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.581 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:total 13 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 6 21:25 . 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Oct 6 21:25 db_0.stat 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Oct 6 21:25 db_12347.stat 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Oct 6 21:25 global.stat 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.582 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:25 . 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:total 48 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 6 21:25 . 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 6 21:25 0016 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.583 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:total 1048576 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Oct 6 21:25 . 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AA 2023-10-06T21:25:59.584 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AB 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AC 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AD 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AE 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000AF 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B0 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B1 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B2 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B3 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B4 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B5 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B6 2023-10-06T21:25:59.585 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000B7 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000B8 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000B9 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000BA 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000BB 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000BC 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000BD 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000BE 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000BF 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C0 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C1 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C2 2023-10-06T21:25:59.586 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C3 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C4 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C5 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C6 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C7 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C8 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000C9 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000CA 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000CB 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000CC 2023-10-06T21:25:59.587 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000CD 2023-10-06T21:25:59.588 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:22 0000000100000001000000CE 2023-10-06T21:25:59.588 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:21 0000000100000001000000CF 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D0 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D1 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D2 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D3 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D4 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000D5 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000D6 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000D7 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D8 2023-10-06T21:25:59.589 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000D9 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000DA 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000DB 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000DC 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000DD 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000DE 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000DF 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000E0 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000E1 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000E2 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000E3 2023-10-06T21:25:59.590 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000E4 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000E5 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000E6 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:25 0000000100000001000000E7 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:23 0000000100000001000000E8 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 6 21:24 0000000100000001000000E9 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 archive_status 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:total 0 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 6 21:10 . 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Oct 6 21:25 .. 2023-10-06T21:25:59.591 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:total 360 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 6 21:20 . 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 6 21:25 .. 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Oct 6 21:20 0000 2023-10-06T21:25:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 6 21:25 0001 2023-10-06T21:25:59.592 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-06T21:25:59.603 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:25:59.616 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-10-06T21:25:59.626 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-10-06T21:25:59.626 INFO:teuthology.task.exec:Executing custom commands... 2023-10-06T21:25:59.626 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:25:59.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-10-06T21:25:59.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-10-06T21:25:59.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.050+0000 7fdb869ff700 1 -- 172.21.15.26:0/3323838769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801061f0 msgr2=0x7fdb8010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.050+0000 7fdb869ff700 1 --2- 172.21.15.26:0/3323838769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801061f0 0x7fdb8010dd70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009f90 tx=0x7fdb7800b330 comp rx=0 tx=0).stop 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 -- 172.21.15.26:0/3323838769 shutdown_connections 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 --2- 172.21.15.26:0/3323838769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801061f0 0x7fdb8010dd70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 --2- 172.21.15.26:0/3323838769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb801010d0 0x7fdb801014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 --2- 172.21.15.26:0/3323838769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb80101a80 0x7fdb80105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 -- 172.21.15.26:0/3323838769 >> 172.21.15.26:0/3323838769 conn(0x7fdb800fc950 msgr2=0x7fdb800fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 -- 172.21.15.26:0/3323838769 shutdown_connections 2023-10-06T21:26:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.051+0000 7fdb869ff700 1 -- 172.21.15.26:0/3323838769 wait complete. 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 Processor -- start 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 -- start start 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb801061f0 0x7fdb801996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb8019dcc0 0x7fdb8019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019e740 con 0x7fdb8019dcc0 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019e8b0 con 0x7fdb801010d0 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.052+0000 7fdb869ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019ea70 con 0x7fdb801061f0 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57330/0 (socket says 172.21.15.26:57330) 2023-10-06T21:26:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 -- 172.21.15.26:0/995094482 learned_addr learned my addr 172.21.15.26:0/995094482 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7f7fe700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb801061f0 0x7fdb801996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb84f9c700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb8019dcc0 0x7fdb8019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 -- 172.21.15.26:0/995094482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb801061f0 msgr2=0x7fdb801996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb801061f0 0x7fdb801996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 -- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb8019dcc0 msgr2=0x7fdb8019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb8019dcc0 0x7fdb8019e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 -- 172.21.15.26:0/995094482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700095f0 con 0x7fdb801010d0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.053+0000 7fdb7ffff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000ec10 tx=0x7fdb7000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.054+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000cc90 con 0x7fdb801010d0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.054+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb700044e0 con 0x7fdb801010d0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.054+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb78009c40 con 0x7fdb801010d0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.054+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70010430 con 0x7fdb801010d0 2023-10-06T21:26:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.054+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb801a62c0 con 0x7fdb801010d0 2023-10-06T21:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.055+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb801005a0 con 0x7fdb801010d0 2023-10-06T21:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.057+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdb70003fb0 con 0x7fdb801010d0 2023-10-06T21:26:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.059+0000 7fdb7d7fa700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 0x7fdb680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.059+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdb70014070 con 0x7fdb801010d0 2023-10-06T21:26:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.059+0000 7fdb7f7fe700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 0x7fdb680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.060+0000 7fdb7f7fe700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 0x7fdb680a3ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb8019a930 tx=0x7fdb74009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:00.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.061+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb70096950 con 0x7fdb801010d0 2023-10-06T21:26:00.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:59 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:00.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:59 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:00.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:25:59 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:26:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:59 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:59 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:00.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:25:59 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:26:00.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.323+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb80101a80 con 0x7fdb680a15f0 2023-10-06T21:26:00.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.332+0000 7fdb7d7fa700 1 -- 172.21.15.26:0/995094482 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7fdb80101a80 con 0x7fdb680a15f0 2023-10-06T21:26:00.335 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T21:10:20", "first": "2023-10-06T21:11:00", "last": "2023-10-06T21:25:00", "last_pruned": "2023-10-06T21:25:00", "created_count": 15, "pruned_count": 9, "active": true} 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 msgr2=0x7fdb680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 0x7fdb680a3ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb8019a930 tx=0x7fdb74009260 comp rx=0 tx=0).stop 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 msgr2=0x7fdb80199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000ec10 tx=0x7fdb7000ef20 comp rx=0 tx=0).stop 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 shutdown_connections 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdb680a15f0 0x7fdb680a3ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb8019dcc0 0x7fdb8019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb801010d0 0x7fdb80199180 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 --2- 172.21.15.26:0/995094482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb801061f0 0x7fdb801996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 >> 172.21.15.26:0/995094482 conn(0x7fdb800fc950 msgr2=0x7fdb800fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:00.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 shutdown_connections 2023-10-06T21:26:00.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.335+0000 7fdb869ff700 1 -- 172.21.15.26:0/995094482 wait complete. 2023-10-06T21:26:00.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-10-06T21:26:00.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:59 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:26:00.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:59 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:26:00.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:25:59 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 -- 172.21.15.26:0/1394275960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d810b4c0 msgr2=0x7f72d810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/1394275960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d810b4c0 0x7f72d810d960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f72d000b0a0 tx=0x7f72d000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 -- 172.21.15.26:0/1394275960 shutdown_connections 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/1394275960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d810b4c0 0x7f72d810d960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/1394275960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d81020f0 0x7f72d81024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/1394275960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d8102a10 0x7f72d810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 -- 172.21.15.26:0/1394275960 >> 172.21.15.26:0/1394275960 conn(0x7f72d80fb870 msgr2=0x7f72d80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:00.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 -- 172.21.15.26:0/1394275960 shutdown_connections 2023-10-06T21:26:00.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.583+0000 7f72dc9ff700 1 -- 172.21.15.26:0/1394275960 wait complete. 2023-10-06T21:26:00.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.584+0000 7f72dc9ff700 1 Processor -- start 2023-10-06T21:26:00.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.584+0000 7f72dc9ff700 1 -- start start 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d8102a10 0x7f72d8100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d810b4c0 0x7f72d81a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d81a57e0 con 0x7f72d810b4c0 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d81a5920 con 0x7f72d81020f0 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72dc9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d81a5a90 con 0x7f72d8102a10 2023-10-06T21:26:00.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72d659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72d659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57346/0 (socket says 172.21.15.26:57346) 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72d659c700 1 -- 172.21.15.26:0/2591073560 learned_addr learned my addr 172.21.15.26:0/2591073560 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72d6d9d700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d810b4c0 0x7f72d81a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.585+0000 7f72d5d9b700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d8102a10 0x7f72d8100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 -- 172.21.15.26:0/2591073560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d8102a10 msgr2=0x7f72d8100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d8102a10 0x7f72d8100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 -- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d810b4c0 msgr2=0x7f72d81a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d810b4c0 0x7f72d81a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 -- 172.21.15.26:0/2591073560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009520 con 0x7f72d81020f0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72d659c700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72c800e9a0 tx=0x7f72c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8009790 con 0x7f72d81020f0 2023-10-06T21:26:00.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c8010430 con 0x7f72d81020f0 2023-10-06T21:26:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8018680 con 0x7f72d81020f0 2023-10-06T21:26:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.586+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d0009d00 con 0x7f72d81020f0 2023-10-06T21:26:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.587+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d81a60a0 con 0x7f72d81020f0 2023-10-06T21:26:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.588+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d8105d50 con 0x7f72d81020f0 2023-10-06T21:26:00.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.590+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f72c80105a0 con 0x7f72d81020f0 2023-10-06T21:26:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.591+0000 7f72c77fe700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 0x7f72c00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.591+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f72c8014070 con 0x7f72d81020f0 2023-10-06T21:26:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.591+0000 7f72d5d9b700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 0x7f72c00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.592+0000 7f72d5d9b700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 0x7f72c00a3b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72d8101740 tx=0x7f72cc009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:00.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.596+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f72c80965d0 con 0x7f72d81020f0 2023-10-06T21:26:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.931+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 --> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f72d8110e50 con 0x7f72c00a1670 2023-10-06T21:26:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.938+0000 7f72c77fe700 1 -- 172.21.15.26:0/2591073560 <== mgr.15306 v2:172.21.15.26:6800/3248613920 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f72d8110e50 con 0x7f72c00a1670 2023-10-06T21:26:00.941 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 msgr2=0x7f72c00a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 0x7f72c00a3b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72d8101740 tx=0x7f72cc009260 comp rx=0 tx=0).stop 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 msgr2=0x7f72d80fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72c800e9a0 tx=0x7f72c800ecb0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 shutdown_connections 2023-10-06T21:26:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f72c00a1670 0x7f72c00a3b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72d810b4c0 0x7f72d81a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72d81020f0 0x7f72d80fff20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 --2- 172.21.15.26:0/2591073560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72d8102a10 0x7f72d8100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 >> 172.21.15.26:0/2591073560 conn(0x7f72d80fb870 msgr2=0x7f72d8102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 shutdown_connections 2023-10-06T21:26:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:00.941+0000 7f72dc9ff700 1 -- 172.21.15.26:0/2591073560 wait complete. 2023-10-06T21:26:00.964 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-06T21:26:00.975 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-06T21:26:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:00 smithi167 ceph-mon[134494]: pgmap v492: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2023-10-06T21:26:01.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:00 smithi026 ceph-mon[98490]: pgmap v492: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2023-10-06T21:26:01.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:00 smithi078 ceph-mon[134896]: pgmap v492: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.0 MiB/s wr, 565 op/s 2023-10-06T21:26:02.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:01 smithi167 ceph-mon[134494]: from='client.27572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:02.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:01 smithi167 ceph-mon[134494]: from='client.27578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:02.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:01 smithi026 ceph-mon[98490]: from='client.27572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:02.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:01 smithi026 ceph-mon[98490]: from='client.27578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:02.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:01 smithi078 ceph-mon[134896]: from='client.27572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:02.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:01 smithi078 ceph-mon[134896]: from='client.27578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-06T21:26:03.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:02 smithi167 ceph-mon[134494]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.1 MiB/s wr, 376 op/s 2023-10-06T21:26:03.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:02 smithi026 ceph-mon[98490]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.1 MiB/s wr, 376 op/s 2023-10-06T21:26:03.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:02 smithi078 ceph-mon[134896]: pgmap v493: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.1 MiB/s wr, 376 op/s 2023-10-06T21:26:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:04 smithi167 ceph-mon[134494]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-10-06T21:26:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:04 smithi026 ceph-mon[98490]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-10-06T21:26:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:04 smithi078 ceph-mon[134896]: pgmap v494: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.0 MiB/s wr, 250 op/s 2023-10-06T21:26:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:06 smithi167 ceph-mon[134494]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:26:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:06 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:06 smithi167 ceph-mon[134494]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 39 op/s 2023-10-06T21:26:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:06 smithi026 ceph-mon[98490]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:26:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:06 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:06 smithi026 ceph-mon[98490]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 39 op/s 2023-10-06T21:26:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:06 smithi078 ceph-mon[134896]: osdmap e108: 12 total, 12 up, 12 in 2023-10-06T21:26:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:06 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:06 smithi078 ceph-mon[134896]: pgmap v496: 97 pgs: 97 active+clean; 5.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 150 KiB/s wr, 39 op/s 2023-10-06T21:26:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:08 smithi167 ceph-mon[134494]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:08 smithi026 ceph-mon[98490]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:08 smithi078 ceph-mon[134896]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: pgmap v498: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: scrub summary: active paths [/] 2023-10-06T21:26:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:10 smithi167 ceph-mon[134494]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: pgmap v498: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:11.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:26:11.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:11.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: scrub summary: active paths [/] 2023-10-06T21:26:11.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:10 smithi026 ceph-mon[98490]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: pgmap v498: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 96 KiB/s wr, 33 op/s 2023-10-06T21:26:11.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:11.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:26:11.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:11.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: scrub summary: active paths [/] 2023-10-06T21:26:11.123 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:10 smithi078 ceph-mon[134896]: scrub summary: idle+waiting paths [/] 2023-10-06T21:26:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:12 smithi167 ceph-mon[134494]: pgmap v499: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-10-06T21:26:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:12 smithi026 ceph-mon[98490]: pgmap v499: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-10-06T21:26:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:12 smithi078 ceph-mon[134896]: pgmap v499: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-10-06T21:26:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:14 smithi167 ceph-mon[134494]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:26:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:14 smithi167 ceph-mon[134494]: pgmap v501: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-10-06T21:26:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:14 smithi026 ceph-mon[98490]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:26:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:14 smithi026 ceph-mon[98490]: pgmap v501: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-10-06T21:26:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:14 smithi078 ceph-mon[134896]: osdmap e109: 12 total, 12 up, 12 in 2023-10-06T21:26:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:14 smithi078 ceph-mon[134896]: pgmap v501: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-10-06T21:26:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:16 smithi167 ceph-mon[134494]: pgmap v502: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-10-06T21:26:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:16 smithi026 ceph-mon[98490]: pgmap v502: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-10-06T21:26:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:16 smithi078 ceph-mon[134896]: pgmap v502: 97 pgs: 97 active+clean; 4.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-10-06T21:26:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:18 smithi167 ceph-mon[134494]: pgmap v503: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:18 smithi026 ceph-mon[98490]: pgmap v503: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:18 smithi078 ceph-mon[134896]: pgmap v503: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:20 smithi167 ceph-mon[134494]: pgmap v504: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:21.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:20 smithi026 ceph-mon[98490]: pgmap v504: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:20 smithi078 ceph-mon[134896]: pgmap v504: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-10-06T21:26:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:22 smithi167 ceph-mon[134494]: pgmap v505: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-10-06T21:26:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:22 smithi026 ceph-mon[98490]: pgmap v505: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-10-06T21:26:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:22 smithi078 ceph-mon[134896]: pgmap v505: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-10-06T21:26:23.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:26:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 -- 172.21.15.26:0/843256453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 msgr2=0x7fa48010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 --2- 172.21.15.26:0/843256453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48010d8d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa47800b0a0 tx=0x7fa47800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 -- 172.21.15.26:0/843256453 shutdown_connections 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 --2- 172.21.15.26:0/843256453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48010d8d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 --2- 172.21.15.26:0/843256453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480101ff0 0x7fa4801023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 --2- 172.21.15.26:0/843256453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480102910 0x7fa48010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 -- 172.21.15.26:0/843256453 >> 172.21.15.26:0/843256453 conn(0x7fa4800fb830 msgr2=0x7fa4800fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 -- 172.21.15.26:0/843256453 shutdown_connections 2023-10-06T21:26:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.059+0000 7fa4865ff700 1 -- 172.21.15.26:0/843256453 wait complete. 2023-10-06T21:26:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 Processor -- start 2023-10-06T21:26:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 -- start start 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480101ff0 0x7fa4801992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480102910 0x7fa4801997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019e290 con 0x7fa480102910 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019e400 con 0x7fa48010b410 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa4865ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019e5c0 con 0x7fa480101ff0 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.061+0000 7fa484b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51820/0 (socket says 172.21.15.26:51820) 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 -- 172.21.15.26:0/705392469 learned_addr learned my addr 172.21.15.26:0/705392469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa47ffff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480101ff0 0x7fa4801992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480102910 0x7fa4801997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 -- 172.21.15.26:0/705392469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480101ff0 msgr2=0x7fa4801992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480101ff0 0x7fa4801992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 -- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480102910 msgr2=0x7fa4801997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480102910 0x7fa4801997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 -- 172.21.15.26:0/705392469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa478009d00 con 0x7fa48010b410 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa484b9c700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa478009fd0 tx=0x7fa47800bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa47d7fa700 1 -- 172.21.15.26:0/705392469 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800e040 con 0x7fa48010b410 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa47d7fa700 1 -- 172.21.15.26:0/705392469 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa478004e60 con 0x7fa48010b410 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48019e840 con 0x7fa48010b410 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.062+0000 7fa47d7fa700 1 -- 172.21.15.26:0/705392469 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780053b0 con 0x7fa48010b410 2023-10-06T21:26:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.063+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4801a60e0 con 0x7fa48010b410 2023-10-06T21:26:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.067+0000 7fa47d7fa700 1 -- 172.21.15.26:0/705392469 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fa47801d9c0 con 0x7fa48010b410 2023-10-06T21:26:24.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.067+0000 7fa499e67b80 1 Processor -- start 2023-10-06T21:26:24.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 -- start start 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 0x56414d7064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d711890 0x56414d713c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d66f460 con 0x56414d7060c0 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d660c20 con 0x56414d7069e0 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa499e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d65e540 con 0x56414d711890 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa47ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 0x56414d7064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa47ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51830/0 (socket says 172.21.15.26:51830) 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.068+0000 7fa47ffff700 1 -- 172.21.15.26:0/2591230576 learned_addr learned my addr 172.21.15.26:0/2591230576 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d711890 0x56414d713c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47d7fa700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 0x7fa4680a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 -- 172.21.15.26:0/2591230576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d711890 msgr2=0x56414d713c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d711890 0x56414d713c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 -- 172.21.15.26:0/2591230576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 msgr2=0x56414d7064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 0x56414d7064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 -- 172.21.15.26:0/2591230576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414d484de0 con 0x56414d7069e0 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 0x56414d7064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47d7fa700 1 -- 172.21.15.26:0/705392469 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa4780ce380 con 0x7fa48010b410 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa47ffff700 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa470009d10 tx=0x7fa4700116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85e4567a8c7646f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa484b9c700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 0x7fa4680a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa4667fc700 1 -- 172.21.15.26:0/2591230576 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4700097e0 con 0x56414d7069e0 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa4667fc700 1 -- 172.21.15.26:0/2591230576 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa470011ed0 con 0x56414d7069e0 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.069+0000 7fa4667fc700 1 -- 172.21.15.26:0/2591230576 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000f8e0 con 0x56414d7069e0 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 -- 172.21.15.26:0/2591230576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 msgr2=0x56414d711350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa484b9c700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 0x7fa4680a3b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4780cbac0 tx=0x7fa478097040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa470009d10 tx=0x7fa4700116c0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 -- 172.21.15.26:0/2591230576 shutdown_connections 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d711890 0x56414d713c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7060c0 0x56414d7064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 --2- 172.21.15.26:0/2591230576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d7069e0 0x56414d711350 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 -- 172.21.15.26:0/2591230576 >> 172.21.15.26:0/2591230576 conn(0x56414d67b260 msgr2=0x56414d702450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 -- 172.21.15.26:0/2591230576 shutdown_connections 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.070+0000 7fa499e67b80 1 -- 172.21.15.26:0/2591230576 wait complete. 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.071+0000 7fa499e67b80 1 -- start start 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.071+0000 7fa499e67b80 1 Processor -- start 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d711300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d659e30 con 0x56414d7069e0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d5f6710 con 0x56414d711890 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d5f63b0 con 0x56414d7060c0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa499e67b80 10 client.0 _ll_register_callbacks cb 0x56414d453800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51838/0 (socket says 172.21.15.26:51838) 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 -- 172.21.15.26:0/313085144 learned_addr learned my addr 172.21.15.26:0/313085144 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d711300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 -- 172.21.15.26:0/313085144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 msgr2=0x56414d711300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d711300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 -- 172.21.15.26:0/313085144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 msgr2=0x56414d70e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.072+0000 7fa47f7fe700 1 -- 172.21.15.26:0/313085144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa470009940 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa474012e70 tx=0x7fa474014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896be4146ee8c3ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa45f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740109c0 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414d6f9f10 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa474010b20 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474010ce0 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa499e67b80 10 client.27590 fetch_fsmap learned FSMap version 61 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.073+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56414d46dc60 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47402d080 con 0x56414d711890 2023-10-06T21:26:24.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa474003980 con 0x56414d711890 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 10 client.27590 fetch_fsmap finished waiting for FSMap version 61 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 20 client.27590 populate_metadata read hostname 'smithi026' 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7213e0 0x56414d723800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 4 client.27590 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56414d652e10 con 0x56414d7213e0 2023-10-06T21:26:24.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.074+0000 7fa47ffff700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7213e0 0x56414d723800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.075+0000 7fa47ffff700 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7213e0 0x56414d723800 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2efbd12674d2d9 server_cookie=1feb4bd1a1d0c319 in_seq=0 out_seq=0 2023-10-06T21:26:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.075+0000 7fa45f7fe700 10 client.27590 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.075+0000 7fa45f7fe700 1 -- 172.21.15.26:0/313085144 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56414d652e10 con 0x56414d7213e0 2023-10-06T21:26:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.075+0000 7fa45f7fe700 10 client.27590 handle_command_reply: tid=0 2023-10-06T21:26:24.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.075+0000 7fa499e67b80 1 client.27590 shutdown 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 msgr2=0x56414d70ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa474012e70 tx=0x7fa474014040 comp rx=0 tx=0).stop 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 shutdown_connections 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7213e0 0x56414d723800 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d70ece0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d711300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 --2- 172.21.15.26:0/313085144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa45f7fe700 0 client.27590 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 >> 172.21.15.26:0/313085144 conn(0x56414d67b260 msgr2=0x56414d701fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 shutdown_connections 2023-10-06T21:26:24.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 1 -- 172.21.15.26:0/313085144 wait complete. 2023-10-06T21:26:24.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.076+0000 7fa499e67b80 20 client.27590 trim_cache size 0 max 16384 2023-10-06T21:26:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.099+0000 7fa499e67b80 1 Processor -- start 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 -- start start 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d709030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d4da920 con 0x56414d7069e0 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d70f700 con 0x56414d711890 2023-10-06T21:26:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa499e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d70a440 con 0x56414d7060c0 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51854/0 (socket says 172.21.15.26:51854) 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa47f7fe700 1 -- 172.21.15.26:0/1134137370 learned_addr learned my addr 172.21.15.26:0/1134137370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa47ffff700 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.100+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d709030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 -- 172.21.15.26:0/1134137370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 msgr2=0x56414d709030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d709030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 -- 172.21.15.26:0/1134137370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 msgr2=0x56414d70ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 -- 172.21.15.26:0/1134137370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414d4ab250 con 0x56414d711890 2023-10-06T21:26:24.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa474009410 tx=0x7fa474012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e413ded818dc605 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa466ffd700 1 -- 172.21.15.26:0/1134137370 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474003680 con 0x56414d711890 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa466ffd700 1 -- 172.21.15.26:0/1134137370 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56414d6f9f10 con 0x56414d711890 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.101+0000 7fa466ffd700 1 -- 172.21.15.26:0/1134137370 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740041b0 con 0x56414d711890 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- 172.21.15.26:0/1134137370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 msgr2=0x56414d7098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa474009410 tx=0x7fa474012b50 comp rx=0 tx=0).stop 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- 172.21.15.26:0/1134137370 shutdown_connections 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d7098b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d709030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 --2- 172.21.15.26:0/1134137370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d70ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- 172.21.15.26:0/1134137370 >> 172.21.15.26:0/1134137370 conn(0x56414d67b260 msgr2=0x56414d71d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- 172.21.15.26:0/1134137370 shutdown_connections 2023-10-06T21:26:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- 172.21.15.26:0/1134137370 wait complete. 2023-10-06T21:26:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.102+0000 7fa499e67b80 1 -- start start 2023-10-06T21:26:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 Processor -- start 2023-10-06T21:26:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d716bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d7170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d775310 con 0x56414d7069e0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d775450 con 0x56414d711890 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414d775610 con 0x56414d7060c0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.103+0000 7fa499e67b80 10 client.0 _ll_register_callbacks cb 0x7fa474012c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d7170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51856/0 (socket says 172.21.15.26:51856) 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 -- 172.21.15.26:0/285896294 learned_addr learned my addr 172.21.15.26:0/285896294 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d716bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 -- 172.21.15.26:0/285896294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 msgr2=0x56414d716bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d716bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 -- 172.21.15.26:0/285896294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 msgr2=0x56414d7170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d7170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47f7fe700 1 -- 172.21.15.26:0/285896294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414d46dc60 con 0x56414d711890 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.104+0000 7fa47ffff700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d7170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa47f7fe700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa474009a40 tx=0x7fa474000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41463b90cec972d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa45f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474010dd0 con 0x56414d711890 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa470011ed0 con 0x56414d711890 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fa474033070 con 0x56414d711890 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474005070 con 0x56414d711890 2023-10-06T21:26:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa499e67b80 10 client.27596 fetch_fsmap learned FSMap version 61 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56414d4832b0 con 0x56414d711890 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.105+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa470011ed0 con 0x56414d711890 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 61) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa474036020 con 0x56414d711890 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 10 client.27596 fetch_fsmap finished waiting for FSMap version 61 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 10 client.27596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 20 client.27596 populate_metadata read hostname 'smithi026' 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7768f0 0x56414d778ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 4 client.27596 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56414d652b30 con 0x56414d7768f0 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa47ffff700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7768f0 0x56414d778ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa47ffff700 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7768f0 0x56414d778ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3b64a82baf1cf6 server_cookie=e0fc79c63b4f3e26 in_seq=0 out_seq=0 2023-10-06T21:26:24.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.106+0000 7fa45f7fe700 10 client.27596 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:24.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa45f7fe700 1 -- 172.21.15.26:0/285896294 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56414d652b30 con 0x56414d7768f0 2023-10-06T21:26:24.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa45f7fe700 10 client.27596 handle_command_reply: tid=0 2023-10-06T21:26:24.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 client.27596 shutdown 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 msgr2=0x56414d717630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa474009a40 tx=0x7fa474000bc0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 shutdown_connections 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56414d7768f0 0x56414d778ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56414d711890 0x56414d717630 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56414d7060c0 0x56414d716bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 --2- 172.21.15.26:0/285896294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56414d7069e0 0x56414d7170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 >> 172.21.15.26:0/285896294 conn(0x56414d67b260 msgr2=0x56414d7098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa45f7fe700 0 client.27596 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 shutdown_connections 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 1 -- 172.21.15.26:0/285896294 wait complete. 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.107+0000 7fa499e67b80 20 client.27596 trim_cache size 0 max 16384 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:26:24.115 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:26:24.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:26:24.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:26:24.116 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:26:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 msgr2=0x7fa4680a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 0x7fa4680a3b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa4780cbac0 tx=0x7fa478097040 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 msgr2=0x7fa48019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa478009fd0 tx=0x7fa47800bd90 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 shutdown_connections 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa48010b410 0x7fa48019dc50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa480101ff0 0x7fa4801992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fa4680a1670 0x7fa4680a3b30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 --2- 172.21.15.26:0/705392469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa480102910 0x7fa4801997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 >> 172.21.15.26:0/705392469 conn(0x7fa4800fb830 msgr2=0x7fa48010d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 shutdown_connections 2023-10-06T21:26:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:24.108+0000 7fa4865ff700 1 -- 172.21.15.26:0/705392469 wait complete. 2023-10-06T21:26:24.132 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:26:24.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:24 smithi167 ceph-mon[134494]: pgmap v506: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-10-06T21:26:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:24 smithi026 ceph-mon[98490]: pgmap v506: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-10-06T21:26:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:24 smithi078 ceph-mon[134896]: pgmap v506: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-10-06T21:26:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:26 smithi167 ceph-mon[134494]: pgmap v507: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-10-06T21:26:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:26 smithi026 ceph-mon[98490]: pgmap v507: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-10-06T21:26:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:26 smithi078 ceph-mon[134896]: pgmap v507: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-10-06T21:26:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:28 smithi167 ceph-mon[134494]: pgmap v508: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-10-06T21:26:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:28 smithi026 ceph-mon[98490]: pgmap v508: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-10-06T21:26:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:28 smithi078 ceph-mon[134896]: pgmap v508: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-10-06T21:26:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:30 smithi167 ceph-mon[134494]: pgmap v509: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:30 smithi026 ceph-mon[98490]: pgmap v509: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:30 smithi078 ceph-mon[134896]: pgmap v509: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:32 smithi167 ceph-mon[134494]: pgmap v510: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-10-06T21:26:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:32 smithi026 ceph-mon[98490]: pgmap v510: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-10-06T21:26:33.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:32 smithi078 ceph-mon[134896]: pgmap v510: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-10-06T21:26:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:34 smithi167 ceph-mon[134494]: pgmap v511: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:26:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:34 smithi026 ceph-mon[98490]: pgmap v511: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:26:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:34 smithi078 ceph-mon[134896]: pgmap v511: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:26:36.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:36 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:36.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:36 smithi167 ceph-mon[134494]: pgmap v512: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:36.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:36 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:36.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:36 smithi026 ceph-mon[98490]: pgmap v512: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:36.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:36 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:36.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:36 smithi078 ceph-mon[134896]: pgmap v512: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-10-06T21:26:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:38 smithi167 ceph-mon[134494]: pgmap v513: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 9 op/s 2023-10-06T21:26:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:38 smithi026 ceph-mon[98490]: pgmap v513: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 9 op/s 2023-10-06T21:26:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:38 smithi078 ceph-mon[134896]: pgmap v513: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 9 op/s 2023-10-06T21:26:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:40 smithi167 ceph-mon[134494]: pgmap v514: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2023-10-06T21:26:41.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:40 smithi026 ceph-mon[98490]: pgmap v514: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2023-10-06T21:26:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:40 smithi078 ceph-mon[134896]: pgmap v514: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 KiB/s wr, 6 op/s 2023-10-06T21:26:43.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:42 smithi167 ceph-mon[134494]: pgmap v515: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-10-06T21:26:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:42 smithi026 ceph-mon[98490]: pgmap v515: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-10-06T21:26:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:42 smithi078 ceph-mon[134896]: pgmap v515: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-10-06T21:26:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:44 smithi167 ceph-mon[134494]: pgmap v516: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:44 smithi026 ceph-mon[98490]: pgmap v516: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:44 smithi078 ceph-mon[134896]: pgmap v516: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:46 smithi167 ceph-mon[134494]: pgmap v517: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:46 smithi026 ceph-mon[98490]: pgmap v517: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:46 smithi078 ceph-mon[134896]: pgmap v517: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 7 op/s 2023-10-06T21:26:49.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:48 smithi167 ceph-mon[134494]: pgmap v518: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-10-06T21:26:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:48 smithi026 ceph-mon[98490]: pgmap v518: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-10-06T21:26:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:48 smithi078 ceph-mon[134896]: pgmap v518: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-10-06T21:26:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:50 smithi167 ceph-mon[134494]: pgmap v519: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 7 op/s 2023-10-06T21:26:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:50 smithi026 ceph-mon[98490]: pgmap v519: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 7 op/s 2023-10-06T21:26:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:50 smithi078 ceph-mon[134896]: pgmap v519: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 7 op/s 2023-10-06T21:26:52.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:51 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:52.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:51 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:52.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:51 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:26:53.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:52 smithi167 ceph-mon[134494]: pgmap v520: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-10-06T21:26:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:52 smithi026 ceph-mon[98490]: pgmap v520: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-10-06T21:26:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:52 smithi078 ceph-mon[134896]: pgmap v520: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-10-06T21:26:54.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 -- 172.21.15.26:0/2716269149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04105ca0 msgr2=0x7f2a04068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 --2- 172.21.15.26:0/2716269149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04105ca0 0x7f2a04068040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009990 tx=0x7f29f4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 -- 172.21.15.26:0/2716269149 shutdown_connections 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 --2- 172.21.15.26:0/2716269149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a041036f0 0x7f2a04103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 --2- 172.21.15.26:0/2716269149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04105ca0 0x7f2a04068040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 --2- 172.21.15.26:0/2716269149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a04068580 0x7f2a040689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 -- 172.21.15.26:0/2716269149 >> 172.21.15.26:0/2716269149 conn(0x7f2a040769d0 msgr2=0x7f2a04076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 -- 172.21.15.26:0/2716269149 shutdown_connections 2023-10-06T21:26:54.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.360+0000 7f2a03fff700 1 -- 172.21.15.26:0/2716269149 wait complete. 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.361+0000 7f2a03fff700 1 Processor -- start 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.361+0000 7f2a03fff700 1 -- start start 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.361+0000 7f2a03fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.361+0000 7f2a03fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a041036f0 0x7f2a0419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a03fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a04105ca0 0x7f2a041a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a03fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041a2240 con 0x7f2a041036f0 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a03fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041a23b0 con 0x7f2a04068580 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a03fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041a2570 con 0x7f2a04105ca0 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54866/0 (socket says 172.21.15.26:54866) 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 -- 172.21.15.26:0/401397277 learned_addr learned my addr 172.21.15.26:0/401397277 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f29fbfff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a041036f0 0x7f2a0419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a037fe700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a04105ca0 0x7f2a041a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 -- 172.21.15.26:0/401397277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a04105ca0 msgr2=0x7f2a041a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a04105ca0 0x7f2a041a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 -- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a041036f0 msgr2=0x7f2a0419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a041036f0 0x7f2a0419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.362+0000 7f2a02ffd700 1 -- 172.21.15.26:0/401397277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40095f0 con 0x7f2a04068580 2023-10-06T21:26:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29f4005910 tx=0x7f29f4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a00ff9700 1 -- 172.21.15.26:0/401397277 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f401d070 con 0x7f2a04068580 2023-10-06T21:26:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a00ff9700 1 -- 172.21.15.26:0/401397277 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f4022470 con 0x7f2a04068580 2023-10-06T21:26:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a041a27f0 con 0x7f2a04068580 2023-10-06T21:26:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a00ff9700 1 -- 172.21.15.26:0/401397277 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4017650 con 0x7f2a04068580 2023-10-06T21:26:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.363+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a041aa090 con 0x7f2a04068580 2023-10-06T21:26:54.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.366+0000 7f2a00ff9700 1 -- 172.21.15.26:0/401397277 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f29f40225e0 con 0x7f2a04068580 2023-10-06T21:26:54.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.368+0000 7f2a00ff9700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 0x7f29f00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.368+0000 7f2a00ff9700 1 -- 172.21.15.26:0/401397277 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f29f40ce2d0 con 0x7f2a04068580 2023-10-06T21:26:54.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.368+0000 7f29fbfff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 0x7f29f00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f2a1d166b80 1 Processor -- start 2023-10-06T21:26:54.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f29fbfff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 0x7f29f00a3bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a0419e870 tx=0x7f29ec009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f2a1d166b80 1 -- start start 2023-10-06T21:26:54.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.369+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b0ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561a6a460 con 0x564561b0ca70 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561a5bc20 con 0x564561b01bc0 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561a59540 con 0x564561b012a0 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54868/0 (socket says 172.21.15.26:54868) 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 -- 172.21.15.26:0/769001382 learned_addr learned my addr 172.21.15.26:0/769001382 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a037fe700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b0ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 -- 172.21.15.26:0/769001382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 msgr2=0x564561b01680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b01680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 -- 172.21.15.26:0/769001382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 msgr2=0x564561b0ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b0ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f29fbfff700 1 -- 172.21.15.26:0/769001382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456187fde0 con 0x564561b01bc0 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.370+0000 7f2a037fe700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b0ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.371+0000 7f29fbfff700 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f29ec011d50 tx=0x7f29ec014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=236cf2b57db1b09a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.371+0000 7f29f97fa700 1 -- 172.21.15.26:0/769001382 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00eb20 con 0x564561b01bc0 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.371+0000 7f29f97fa700 1 -- 172.21.15.26:0/769001382 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29ec021070 con 0x564561b01bc0 2023-10-06T21:26:54.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.371+0000 7f29f97fa700 1 -- 172.21.15.26:0/769001382 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec01d670 con 0x564561b01bc0 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 -- 172.21.15.26:0/769001382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 msgr2=0x564561b0c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f29ec011d50 tx=0x7f29ec014000 comp rx=0 tx=0).stop 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 -- 172.21.15.26:0/769001382 shutdown_connections 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b0ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b01680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/769001382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b01bc0 0x564561b0c530 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 -- 172.21.15.26:0/769001382 >> 172.21.15.26:0/769001382 conn(0x564561a76260 msgr2=0x564561afd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 -- 172.21.15.26:0/769001382 shutdown_connections 2023-10-06T21:26:54.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.372+0000 7f2a1d166b80 1 -- 172.21.15.26:0/769001382 wait complete. 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.373+0000 7f2a1d166b80 1 -- start start 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.373+0000 7f2a1d166b80 1 Processor -- start 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561a54e30 con 0x564561b01bc0 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645619f1710 con 0x564561b0ca70 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645619f13b0 con 0x564561b012a0 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a1d166b80 10 client.0 _ll_register_callbacks cb 0x56456184e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54876/0 (socket says 172.21.15.26:54876) 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 -- 172.21.15.26:0/2607233973 learned_addr learned my addr 172.21.15.26:0/2607233973 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 -- 172.21.15.26:0/2607233973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 msgr2=0x564561b0c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b0c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 -- 172.21.15.26:0/2607233973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 msgr2=0x564561b09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.374+0000 7f2a037fe700 1 -- 172.21.15.26:0/2607233973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec017070 con 0x564561b0ca70 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f2a037fe700 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007c60 tx=0x7f29fc007c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c3ada1ca6a71e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f29ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:26:54.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00e040 con 0x564561b0ca70 2023-10-06T21:26:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564561af5160 con 0x564561b0ca70 2023-10-06T21:26:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f29fc013970 con 0x564561b0ca70 2023-10-06T21:26:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29fc013b30 con 0x564561b0ca70 2023-10-06T21:26:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f2a1d166b80 10 client.27608 fetch_fsmap learned FSMap version 63 2023-10-06T21:26:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x564561868c60 con 0x564561b0ca70 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.375+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564561af5160 con 0x564561b0ca70 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 63) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f29fc01e3b0 con 0x564561b0ca70 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 10 client.27608 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 10 client.27608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 20 client.27608 populate_metadata read hostname 'smithi026' 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b1c600 0x564561b1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 4 client.27608 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564561a4de10 con 0x564561b1c600 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b1c600 0x564561b1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b1c600 0x564561b1ea20 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b66453130f8584c5 server_cookie=e250ab9933249f81 in_seq=0 out_seq=0 2023-10-06T21:26:54.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.376+0000 7f29ea7fc700 10 client.27608 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.377+0000 7f29ea7fc700 1 -- 172.21.15.26:0/2607233973 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564561a4de10 con 0x564561b1c600 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.377+0000 7f29ea7fc700 10 client.27608 handle_command_reply: tid=0 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.377+0000 7f2a1d166b80 1 client.27608 shutdown 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.377+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 msgr2=0x564561b09e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.377+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007c60 tx=0x7f29fc007c90 comp rx=0 tx=0).stop 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 shutdown_connections 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b1c600 0x564561b1ea20 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b09e60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b0c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/2607233973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f29ea7fc700 0 client.27608 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 >> 172.21.15.26:0/2607233973 conn(0x564561a76260 msgr2=0x564561afd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 shutdown_connections 2023-10-06T21:26:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 1 -- 172.21.15.26:0/2607233973 wait complete. 2023-10-06T21:26:54.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.378+0000 7f2a1d166b80 20 client.27608 trim_cache size 0 max 16384 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 Processor -- start 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 -- start start 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b01bc0 0x564561b08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645618d5920 con 0x564561b0ca70 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561b04690 con 0x564561b012a0 2023-10-06T21:26:54.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561af9c70 con 0x564561b01bc0 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.408+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54890/0 (socket says 172.21.15.26:54890) 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 -- 172.21.15.26:0/4117293752 learned_addr learned my addr 172.21.15.26:0/4117293752 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a037fe700 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f29fbfff700 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b01bc0 0x564561b08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 -- 172.21.15.26:0/4117293752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b01bc0 msgr2=0x564561b08f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b01bc0 0x564561b08f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 -- 172.21.15.26:0/4117293752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 msgr2=0x564561b17d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b17d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 -- 172.21.15.26:0/4117293752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645618a6250 con 0x564561b012a0 2023-10-06T21:26:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f29f4097d40 tx=0x7f29f4022cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea9bc9d389892ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f29f9ffb700 1 -- 172.21.15.26:0/4117293752 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40b2030 con 0x564561b012a0 2023-10-06T21:26:54.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f29f9ffb700 1 -- 172.21.15.26:0/4117293752 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f409b9d0 con 0x564561b012a0 2023-10-06T21:26:54.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.409+0000 7f29f9ffb700 1 -- 172.21.15.26:0/4117293752 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4099670 con 0x564561b012a0 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 -- 172.21.15.26:0/4117293752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 msgr2=0x564561b09730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f29f4097d40 tx=0x7f29f4022cd0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 -- 172.21.15.26:0/4117293752 shutdown_connections 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b0ca70 0x564561b17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b012a0 0x564561b09730 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/4117293752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b01bc0 0x564561b08f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 -- 172.21.15.26:0/4117293752 >> 172.21.15.26:0/4117293752 conn(0x564561a76260 msgr2=0x564561af90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 -- 172.21.15.26:0/4117293752 shutdown_connections 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.410+0000 7f2a1d166b80 1 -- 172.21.15.26:0/4117293752 wait complete. 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 -- start start 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 Processor -- start 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561b123f0 con 0x564561b01bc0 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.411+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561b6f930 con 0x564561b0ca70 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a1d166b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x564561b6faa0 con 0x564561b012a0 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a1d166b80 10 client.0 _ll_register_callbacks cb 0x7f29fc0078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f29fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54894/0 (socket says 172.21.15.26:54894) 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 -- 172.21.15.26:0/1637069686 learned_addr learned my addr 172.21.15.26:0/1637069686 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 -- 172.21.15.26:0/1637069686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 msgr2=0x564561b11790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b11790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 -- 172.21.15.26:0/1637069686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 msgr2=0x564561b11cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b11cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 -- 172.21.15.26:0/1637069686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f40a1040 con 0x564561b0ca70 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f2a037fe700 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007c30 tx=0x7f29fc009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33ddc167783e70d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.412+0000 7f29ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:26:54.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00e070 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29ec021070 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f29fc01eb30 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29fc01ee60 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 10 client.27614 fetch_fsmap learned FSMap version 63 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x56456187e2b0 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29fc037020 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 63) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f29fc013d40 con 0x564561b0ca70 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 10 client.27614 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 20 client.27614 populate_metadata read hostname 'smithi026' 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b710d0 0x564561b734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:26:54.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 4 client.27614 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.413+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564561a4db30 con 0x564561b710d0 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b710d0 0x564561b734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f29ea7fc700 10 client.27614 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f2a02ffd700 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b710d0 0x564561b734f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cdcdbc95c7a126 server_cookie=52feb901564e9aaa in_seq=0 out_seq=0 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f29ea7fc700 1 -- 172.21.15.26:0/1637069686 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564561a4db30 con 0x564561b710d0 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f29ea7fc700 10 client.27614 handle_command_reply: tid=0 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.414+0000 7f2a1d166b80 1 client.27614 shutdown 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 msgr2=0x564561b6f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007c30 tx=0x7f29fc009a60 comp rx=0 tx=0).stop 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 shutdown_connections 2023-10-06T21:26:54.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x564561b710d0 0x564561b734f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564561b0ca70 0x564561b6f3f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564561b012a0 0x564561b11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 --2- 172.21.15.26:0/1637069686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564561b01bc0 0x564561b11cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 >> 172.21.15.26:0/1637069686 conn(0x564561a76260 msgr2=0x564561b17d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f29ea7fc700 0 client.27614 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 shutdown_connections 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 1 -- 172.21.15.26:0/1637069686 wait complete. 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.415+0000 7f2a1d166b80 20 client.27614 trim_cache size 0 max 16384 2023-10-06T21:26:54.423 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:26:54.424 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 msgr2=0x7f29f00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 0x7f29f00a3bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a0419e870 tx=0x7f29ec009260 comp rx=0 tx=0).stop 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 msgr2=0x7f2a0419d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29f4005910 tx=0x7f29f4004710 comp rx=0 tx=0).stop 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 shutdown_connections 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a04105ca0 0x7f2a041a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a04068580 0x7f2a0419d260 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f29f00a1710 0x7f29f00a3bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 --2- 172.21.15.26:0/401397277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a041036f0 0x7f2a0419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:26:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 >> 172.21.15.26:0/401397277 conn(0x7f2a040769d0 msgr2=0x7f2a040fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:26:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 shutdown_connections 2023-10-06T21:26:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:26:54.416+0000 7f2a03fff700 1 -- 172.21.15.26:0/401397277 wait complete. 2023-10-06T21:26:54.440 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:26:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:54 smithi167 ceph-mon[134494]: pgmap v521: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:54 smithi026 ceph-mon[98490]: pgmap v521: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:54 smithi078 ceph-mon[134896]: pgmap v521: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:56 smithi167 ceph-mon[134494]: pgmap v522: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:56 smithi026 ceph-mon[98490]: pgmap v522: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:57.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:56 smithi078 ceph-mon[134896]: pgmap v522: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-10-06T21:26:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:26:58 smithi167 ceph-mon[134494]: pgmap v523: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 9 op/s 2023-10-06T21:26:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:26:58 smithi026 ceph-mon[98490]: pgmap v523: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 9 op/s 2023-10-06T21:26:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:26:58 smithi078 ceph-mon[134896]: pgmap v523: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 9 op/s 2023-10-06T21:27:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:00 smithi167 ceph-mon[134494]: pgmap v524: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-06T21:27:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:00 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:00 smithi026 ceph-mon[98490]: pgmap v524: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-06T21:27:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:00 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:00 smithi078 ceph-mon[134896]: pgmap v524: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-10-06T21:27:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:00 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:27:02.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:01 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:02.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:01 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:02.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:01 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:27:02.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:01 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:02.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:01 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:02.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:01 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:27:02.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:01 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:27:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:01 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:27:02.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:01 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:27:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:02 smithi026 ceph-mon[98490]: pgmap v525: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 21 op/s 2023-10-06T21:27:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:02 smithi078 ceph-mon[134896]: pgmap v525: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 21 op/s 2023-10-06T21:27:03.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:02 smithi167 ceph-mon[134494]: pgmap v525: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 21 op/s 2023-10-06T21:27:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:04 smithi167 ceph-mon[134494]: pgmap v526: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 20 op/s 2023-10-06T21:27:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:04 smithi026 ceph-mon[98490]: pgmap v526: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 20 op/s 2023-10-06T21:27:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:04 smithi078 ceph-mon[134896]: pgmap v526: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 20 op/s 2023-10-06T21:27:06.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:06 smithi167 ceph-mon[134494]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:27:06.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:06 smithi167 ceph-mon[134494]: pgmap v528: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 25 op/s 2023-10-06T21:27:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:06 smithi026 ceph-mon[98490]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:27:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:06 smithi026 ceph-mon[98490]: pgmap v528: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 25 op/s 2023-10-06T21:27:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:06 smithi078 ceph-mon[134896]: osdmap e110: 12 total, 12 up, 12 in 2023-10-06T21:27:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:06 smithi078 ceph-mon[134896]: pgmap v528: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 104 KiB/s wr, 25 op/s 2023-10-06T21:27:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:08 smithi167 ceph-mon[134494]: pgmap v529: 97 pgs: 97 active+clean; 4.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:08 smithi026 ceph-mon[98490]: pgmap v529: 97 pgs: 97 active+clean; 4.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:08 smithi078 ceph-mon[134896]: pgmap v529: 97 pgs: 97 active+clean; 4.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:11.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:10 smithi167 ceph-mon[134494]: pgmap v530: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:10 smithi026 ceph-mon[98490]: pgmap v530: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:11.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:10 smithi078 ceph-mon[134896]: pgmap v530: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-10-06T21:27:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:12 smithi167 ceph-mon[134494]: pgmap v531: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-10-06T21:27:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:12 smithi026 ceph-mon[98490]: pgmap v531: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-10-06T21:27:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:12 smithi078 ceph-mon[134896]: pgmap v531: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-10-06T21:27:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:14 smithi167 ceph-mon[134494]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:27:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:14 smithi167 ceph-mon[134494]: pgmap v533: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-10-06T21:27:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:14 smithi026 ceph-mon[98490]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:27:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:14 smithi026 ceph-mon[98490]: pgmap v533: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-10-06T21:27:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:14 smithi078 ceph-mon[134896]: osdmap e111: 12 total, 12 up, 12 in 2023-10-06T21:27:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:14 smithi078 ceph-mon[134896]: pgmap v533: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-10-06T21:27:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:16 smithi167 ceph-mon[134494]: pgmap v534: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-10-06T21:27:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:16 smithi026 ceph-mon[98490]: pgmap v534: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-10-06T21:27:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:16 smithi078 ceph-mon[134896]: pgmap v534: 97 pgs: 97 active+clean; 3.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-10-06T21:27:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:18 smithi167 ceph-mon[134494]: pgmap v535: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2023-10-06T21:27:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:18 smithi026 ceph-mon[98490]: pgmap v535: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2023-10-06T21:27:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:18 smithi078 ceph-mon[134896]: pgmap v535: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2023-10-06T21:27:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:20 smithi167 ceph-mon[134494]: pgmap v536: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-10-06T21:27:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:20 smithi026 ceph-mon[98490]: pgmap v536: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-10-06T21:27:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:20 smithi078 ceph-mon[134896]: pgmap v536: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-10-06T21:27:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:22 smithi167 ceph-mon[134494]: pgmap v537: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-10-06T21:27:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:22 smithi026 ceph-mon[98490]: pgmap v537: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-10-06T21:27:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:22 smithi078 ceph-mon[134896]: pgmap v537: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-10-06T21:27:24.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:27:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 -- 172.21.15.26:0/179138712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c073260 msgr2=0x7f067c073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 --2- 172.21.15.26:0/179138712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c073260 0x7f067c073640 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f066c009990 tx=0x7f066c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 -- 172.21.15.26:0/179138712 shutdown_connections 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 --2- 172.21.15.26:0/179138712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c10d730 0x7f067c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 --2- 172.21.15.26:0/179138712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c073260 0x7f067c073640 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.702+0000 7f0681bff700 1 --2- 172.21.15.26:0/179138712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.703+0000 7f0681bff700 1 -- 172.21.15.26:0/179138712 >> 172.21.15.26:0/179138712 conn(0x7f067c0fc960 msgr2=0x7f067c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.703+0000 7f0681bff700 1 -- 172.21.15.26:0/179138712 shutdown_connections 2023-10-06T21:27:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.703+0000 7f0681bff700 1 -- 172.21.15.26:0/179138712 wait complete. 2023-10-06T21:27:24.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.704+0000 7f0681bff700 1 Processor -- start 2023-10-06T21:27:24.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.704+0000 7f0681bff700 1 -- start start 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c10d730 0x7f067c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e220 con 0x7f067c10d730 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e390 con 0x7f067c073260 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f0681bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e550 con 0x7f067c073b80 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33428/0 (socket says 172.21.15.26:33428) 2023-10-06T21:27:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 -- 172.21.15.26:0/2237630769 learned_addr learned my addr 172.21.15.26:0/2237630769 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067affd700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067bfff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c10d730 0x7f067c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 -- 172.21.15.26:0/2237630769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 msgr2=0x7f067c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 -- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c10d730 msgr2=0x7f067c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c10d730 0x7f067c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067b7fe700 1 -- 172.21.15.26:0/2237630769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0095f0 con 0x7f067c073260 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.705+0000 7f067affd700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f0678ff9700 1 -- 172.21.15.26:0/2237630769 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c01d070 con 0x7f067c073260 2023-10-06T21:27:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f0678ff9700 1 -- 172.21.15.26:0/2237630769 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066c00fb80 con 0x7f067c073260 2023-10-06T21:27:24.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c19e7d0 con 0x7f067c073260 2023-10-06T21:27:24.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f0678ff9700 1 -- 172.21.15.26:0/2237630769 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c017650 con 0x7f067c073260 2023-10-06T21:27:24.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.706+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c1a6070 con 0x7f067c073260 2023-10-06T21:27:24.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.711+0000 7f0678ff9700 1 -- 172.21.15.26:0/2237630769 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f066c00fcf0 con 0x7f067c073260 2023-10-06T21:27:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.712+0000 7f0695480b80 1 Processor -- start 2023-10-06T21:27:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.712+0000 7f0695480b80 1 -- start start 2023-10-06T21:27:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0678ff9700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 0x7f06640a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 0x55bf3104d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 0x55bf3105ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30fb6460 con 0x55bf3104d0c0 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30fa7c20 con 0x55bf31058890 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0695480b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30fa5540 con 0x55bf3104d9e0 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f0678ff9700 1 -- 172.21.15.26:0/2237630769 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f066c0ce6f0 con 0x7f067c073260 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067bfff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 0x7f06640a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45954/0 (socket says 172.21.15.26:45954) 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067affd700 1 -- 172.21.15.26:0/351024449 learned_addr learned my addr 172.21.15.26:0/351024449 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067bfff700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 0x55bf3105ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.713+0000 7f067b7fe700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 0x55bf3104d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 -- 172.21.15.26:0/351024449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 msgr2=0x55bf3105ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 0x55bf3105ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 -- 172.21.15.26:0/351024449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 msgr2=0x55bf3104d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 0x55bf3104d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 -- 172.21.15.26:0/351024449 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf30dcbde0 con 0x55bf3104d9e0 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067bfff700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 0x55bf3105ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067b7fe700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 0x55bf3104d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067affd700 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f067000bce0 tx=0x7f067000f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632a48144d1cbfbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f067bfff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 0x7f06640a3b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0674009fd0 tx=0x7f0674009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f0661ffb700 1 -- 172.21.15.26:0/351024449 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000fbe0 con 0x55bf3104d9e0 2023-10-06T21:27:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f0661ffb700 1 -- 172.21.15.26:0/351024449 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067000fd40 con 0x55bf3104d9e0 2023-10-06T21:27:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.714+0000 7f0661ffb700 1 -- 172.21.15.26:0/351024449 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670011760 con 0x55bf3104d9e0 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 -- 172.21.15.26:0/351024449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 msgr2=0x55bf31058350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f067000bce0 tx=0x7f067000f360 comp rx=0 tx=0).stop 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 -- 172.21.15.26:0/351024449 shutdown_connections 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf31058890 0x55bf3105ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d0c0 0x55bf3104d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 --2- 172.21.15.26:0/351024449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d9e0 0x55bf31058350 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 -- 172.21.15.26:0/351024449 >> 172.21.15.26:0/351024449 conn(0x55bf30fc2260 msgr2=0x55bf31049450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 -- 172.21.15.26:0/351024449 shutdown_connections 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.715+0000 7f0695480b80 1 -- 172.21.15.26:0/351024449 wait complete. 2023-10-06T21:27:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.716+0000 7f0695480b80 1 -- start start 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.716+0000 7f0695480b80 1 Processor -- start 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d9e0 0x55bf31055550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 0x55bf31055c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30fa0e30 con 0x55bf3104d9e0 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30f3d710 con 0x55bf3104d0c0 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30f3d3b0 con 0x55bf31058890 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f0695480b80 10 client.0 _ll_register_callbacks cb 0x55bf30d9a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d9e0 0x55bf31055550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 -- 172.21.15.26:0/2805697056 learned_addr learned my addr 172.21.15.26:0/2805697056 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067bfff700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 0x55bf31055c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 -- 172.21.15.26:0/2805697056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 msgr2=0x55bf31055c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 0x55bf31055c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 -- 172.21.15.26:0/2805697056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d9e0 msgr2=0x55bf31055550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d9e0 0x55bf31055550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067b7fe700 1 -- 172.21.15.26:0/2805697056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0674012bf0 con 0x55bf3104d0c0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.717+0000 7f067bfff700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 0x55bf31055c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f067b7fe700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f066c0982e0 tx=0x7f066c09b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ff7b722207cdda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f065affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0661f0 con 0x55bf3104d0c0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f066c09bf70 con 0x55bf3104d0c0 2023-10-06T21:27:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf31040f10 con 0x55bf3104d0c0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f066c064f40 con 0x55bf3104d0c0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f0695480b80 10 client.27626 fetch_fsmap learned FSMap version 63 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f067000b940 con 0x55bf3104d0c0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.718+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf31040f10 con 0x55bf3104d0c0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 63) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f066c09ad60 con 0x55bf3104d0c0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 10 client.27626 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 10 client.27626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 20 client.27626 populate_metadata read hostname 'smithi026' 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf31067fa0 0x55bf3106a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 4 client.27626 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf30f99e10 con 0x55bf31067fa0 2023-10-06T21:27:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.719+0000 7f067affd700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf31067fa0 0x55bf3106a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.720+0000 7f067affd700 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf31067fa0 0x55bf3106a3c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a909cfcce40816 server_cookie=e6fd0f964426515 in_seq=0 out_seq=0 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.720+0000 7f065affd700 10 client.27626 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.720+0000 7f065affd700 1 -- 172.21.15.26:0/2805697056 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bf30f99e10 con 0x55bf31067fa0 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.720+0000 7f065affd700 10 client.27626 handle_command_reply: tid=0 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.720+0000 7f0695480b80 1 client.27626 shutdown 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 msgr2=0x55bf31058340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f066c0982e0 tx=0x7f066c09b1d0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 shutdown_connections 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf31058890 0x55bf31055c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf31067fa0 0x55bf3106a3c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d0c0 0x55bf31058340 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 --2- 172.21.15.26:0/2805697056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf3104d9e0 0x55bf31055550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f065affd700 0 client.27626 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 >> 172.21.15.26:0/2805697056 conn(0x55bf30fc2260 msgr2=0x55bf31049040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 shutdown_connections 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 1 -- 172.21.15.26:0/2805697056 wait complete. 2023-10-06T21:27:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.721+0000 7f0695480b80 20 client.27626 trim_cache size 0 max 16384 2023-10-06T21:27:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.739+0000 7f0695480b80 1 Processor -- start 2023-10-06T21:27:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.739+0000 7f0695480b80 1 -- start start 2023-10-06T21:27:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.739+0000 7f0695480b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf3104ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f0695480b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f0695480b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf3105f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f0695480b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf30e21920 con 0x55bf31058890 2023-10-06T21:27:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f0695480b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf31048100 con 0x55bf3104d9e0 2023-10-06T21:27:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f0695480b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf3105f910 con 0x55bf3104d0c0 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf3104ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33454/0 (socket says 172.21.15.26:33454) 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067affd700 1 -- 172.21.15.26:0/1436403797 learned_addr learned my addr 172.21.15.26:0/1436403797 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067bfff700 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf3105f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 -- 172.21.15.26:0/1436403797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 msgr2=0x55bf31047bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 -- 172.21.15.26:0/1436403797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 msgr2=0x55bf3105f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf3105f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067b7fe700 1 -- 172.21.15.26:0/1436403797 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf30db4c60 con 0x55bf3104d0c0 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.740+0000 7f067affd700 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.741+0000 7f067b7fe700 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf3104ff60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f066c006010 tx=0x7f066c022b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb909fed39a89755 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.741+0000 7f06627fc700 1 -- 172.21.15.26:0/1436403797 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c09bf70 con 0x55bf3104d0c0 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.741+0000 7f06627fc700 1 -- 172.21.15.26:0/1436403797 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066c09b870 con 0x55bf3104d0c0 2023-10-06T21:27:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.741+0000 7f06627fc700 1 -- 172.21.15.26:0/1436403797 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c09bd10 con 0x55bf3104d0c0 2023-10-06T21:27:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 -- 172.21.15.26:0/1436403797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 msgr2=0x55bf3104ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf3104ff60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f066c006010 tx=0x7f066c022b80 comp rx=0 tx=0).stop 2023-10-06T21:27:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 -- 172.21.15.26:0/1436403797 shutdown_connections 2023-10-06T21:27:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf3105f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf3104ff60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 --2- 172.21.15.26:0/1436403797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31047bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 -- 172.21.15.26:0/1436403797 >> 172.21.15.26:0/1436403797 conn(0x55bf30fc2260 msgr2=0x55bf3106ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 -- 172.21.15.26:0/1436403797 shutdown_connections 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.742+0000 7f0695480b80 1 -- 172.21.15.26:0/1436403797 wait complete. 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 -- start start 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 Processor -- start 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf31052b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf310517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf31051d10 con 0x55bf31058890 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf31051e50 con 0x55bf3104d9e0 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.743+0000 7f0695480b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf31052010 con 0x55bf3104d0c0 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f0695480b80 10 client.0 _ll_register_callbacks cb 0x7f066c09b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:24.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:33458/0 (socket says 172.21.15.26:33458) 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 -- 172.21.15.26:0/120962376 learned_addr learned my addr 172.21.15.26:0/120962376 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067bfff700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf310517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067b7fe700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf31052b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 -- 172.21.15.26:0/120962376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 msgr2=0x55bf31052b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf31052b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 -- 172.21.15.26:0/120962376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 msgr2=0x55bf310517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf310517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 -- 172.21.15.26:0/120962376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf30dcbde0 con 0x55bf3104d9e0 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067bfff700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf310517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:24.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f067affd700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0670019000 tx=0x7f067000fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24b793b08ddf650 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.744+0000 7f065affd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670020070 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f067000fd40 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0670027980 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0670004550 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 10 client.27632 fetch_fsmap learned FSMap version 63 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f067000b940 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067000fd40 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 63) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f067002e060 con 0x55bf3104d9e0 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 10 client.27632 fetch_fsmap finished waiting for FSMap version 63 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 10 client.27632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 20 client.27632 populate_metadata read hostname 'smithi026' 2023-10-06T21:27:24.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf310bce00 0x55bf310bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:24.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 4 client.27632 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:27:24.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.745+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf30f99b30 con 0x55bf310bce00 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f067b7fe700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf310bce00 0x55bf310bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f067b7fe700 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf310bce00 0x55bf310bf220 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf8ff94c0c4e77b server_cookie=32b5d777b5d36801 in_seq=0 out_seq=0 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f065affd700 10 client.27632 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f065affd700 1 -- 172.21.15.26:0/120962376 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55bf30f99b30 con 0x55bf310bce00 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f065affd700 10 client.27632 handle_command_reply: tid=0 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f0695480b80 1 client.27632 shutdown 2023-10-06T21:27:24.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.746+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 msgr2=0x55bf31053090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0670019000 tx=0x7f067000fb50 comp rx=0 tx=0).stop 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 shutdown_connections 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bf31058890 0x55bf310517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf3104d0c0 0x55bf31052b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55bf310bce00 0x55bf310bf220 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 --2- 172.21.15.26:0/120962376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3104d9e0 0x55bf31053090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f065affd700 0 client.27632 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 >> 172.21.15.26:0/120962376 conn(0x55bf30fc2260 msgr2=0x55bf31050850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 shutdown_connections 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 1 -- 172.21.15.26:0/120962376 wait complete. 2023-10-06T21:27:24.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.747+0000 7f0695480b80 20 client.27632 trim_cache size 0 max 16384 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:27:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:27:24.757 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:27:24.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 msgr2=0x7f06640a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 0x7f06640a3b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0674009fd0 tx=0x7f0674009920 comp rx=0 tx=0).stop 2023-10-06T21:27:24.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 msgr2=0x7f067c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:24.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c0057f0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 shutdown_connections 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f06640a1670 0x7f06640a3b30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f067c10d730 0x7f067c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c073260 0x7f067c199240 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 --2- 172.21.15.26:0/2237630769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f067c073b80 0x7f067c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 >> 172.21.15.26:0/2237630769 conn(0x7f067c0fc960 msgr2=0x7f067c10ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 shutdown_connections 2023-10-06T21:27:24.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:24.748+0000 7f0681bff700 1 -- 172.21.15.26:0/2237630769 wait complete. 2023-10-06T21:27:24.766 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:24 smithi026 ceph-mon[98490]: pgmap v538: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-10-06T21:27:24.768 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:27:24.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:24 smithi167 ceph-mon[134494]: pgmap v538: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-10-06T21:27:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:24 smithi078 ceph-mon[134896]: pgmap v538: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-10-06T21:27:27.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:26 smithi167 ceph-mon[134494]: pgmap v539: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-10-06T21:27:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:26 smithi026 ceph-mon[98490]: pgmap v539: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-10-06T21:27:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:26 smithi078 ceph-mon[134896]: pgmap v539: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-10-06T21:27:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:28 smithi167 ceph-mon[134494]: pgmap v540: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-10-06T21:27:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:28 smithi026 ceph-mon[98490]: pgmap v540: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-10-06T21:27:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:28 smithi078 ceph-mon[134896]: pgmap v540: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-10-06T21:27:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:30 smithi167 ceph-mon[134494]: pgmap v541: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:27:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:30 smithi026 ceph-mon[98490]: pgmap v541: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:27:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:30 smithi078 ceph-mon[134896]: pgmap v541: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-10-06T21:27:33.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:32 smithi167 ceph-mon[134494]: pgmap v542: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-10-06T21:27:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:32 smithi026 ceph-mon[98490]: pgmap v542: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-10-06T21:27:33.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:32 smithi078 ceph-mon[134896]: pgmap v542: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-10-06T21:27:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:34 smithi167 ceph-mon[134494]: pgmap v543: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2023-10-06T21:27:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:34 smithi026 ceph-mon[98490]: pgmap v543: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2023-10-06T21:27:34.873 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:34 smithi078 ceph-mon[134896]: pgmap v543: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2023-10-06T21:27:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:36 smithi167 ceph-mon[134494]: pgmap v544: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2023-10-06T21:27:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:36 smithi026 ceph-mon[98490]: pgmap v544: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2023-10-06T21:27:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:36 smithi078 ceph-mon[134896]: pgmap v544: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2023-10-06T21:27:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:38 smithi167 ceph-mon[134494]: pgmap v545: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-10-06T21:27:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:38 smithi026 ceph-mon[98490]: pgmap v545: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-10-06T21:27:39.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:38 smithi078 ceph-mon[134896]: pgmap v545: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-10-06T21:27:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:40 smithi167 ceph-mon[134494]: pgmap v546: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2023-10-06T21:27:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:40 smithi026 ceph-mon[98490]: pgmap v546: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2023-10-06T21:27:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:40 smithi078 ceph-mon[134896]: pgmap v546: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.6 KiB/s wr, 5 op/s 2023-10-06T21:27:43.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:42 smithi167 ceph-mon[134494]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2023-10-06T21:27:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:42 smithi026 ceph-mon[98490]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2023-10-06T21:27:43.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:42 smithi078 ceph-mon[134896]: pgmap v547: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2023-10-06T21:27:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:44 smithi167 ceph-mon[134494]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:44 smithi026 ceph-mon[98490]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:44 smithi078 ceph-mon[134896]: pgmap v548: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:46 smithi167 ceph-mon[134494]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:46 smithi026 ceph-mon[98490]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:46 smithi078 ceph-mon[134896]: pgmap v549: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2023-10-06T21:27:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:48 smithi167 ceph-mon[134494]: pgmap v550: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:27:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:48 smithi026 ceph-mon[98490]: pgmap v550: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:27:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:48 smithi078 ceph-mon[134896]: pgmap v550: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 597 B/s wr, 7 op/s 2023-10-06T21:27:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:50 smithi167 ceph-mon[134494]: pgmap v551: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 5 op/s 2023-10-06T21:27:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:50 smithi026 ceph-mon[98490]: pgmap v551: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 5 op/s 2023-10-06T21:27:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:50 smithi078 ceph-mon[134896]: pgmap v551: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 5 op/s 2023-10-06T21:27:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:52 smithi167 ceph-mon[134494]: pgmap v552: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:27:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:52 smithi026 ceph-mon[98490]: pgmap v552: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:27:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:52 smithi078 ceph-mon[134896]: pgmap v552: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 7 op/s 2023-10-06T21:27:54.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:27:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 -- 172.21.15.26:0/2800069992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 msgr2=0x7f63d4073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 --2- 172.21.15.26:0/2800069992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4073660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009990 tx=0x7f63c8009ca0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 -- 172.21.15.26:0/2800069992 shutdown_connections 2023-10-06T21:27:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 --2- 172.21.15.26:0/2800069992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d410d550 0x7f63d410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 --2- 172.21.15.26:0/2800069992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4073660 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 --2- 172.21.15.26:0/2800069992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d4073ba0 0x7f63d410d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 -- 172.21.15.26:0/2800069992 >> 172.21.15.26:0/2800069992 conn(0x7f63d40fc7c0 msgr2=0x7f63d40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 -- 172.21.15.26:0/2800069992 shutdown_connections 2023-10-06T21:27:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:54.999+0000 7f63d999d700 1 -- 172.21.15.26:0/2800069992 wait complete. 2023-10-06T21:27:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.000+0000 7f63d999d700 1 Processor -- start 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.000+0000 7f63d999d700 1 -- start start 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 0x7f63d4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d419e030 con 0x7f63d4073280 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d419e1a0 con 0x7f63d410d550 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d999d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d419e360 con 0x7f63d4073ba0 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d919c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 0x7f63d4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37684/0 (socket says 172.21.15.26:37684) 2023-10-06T21:27:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d919c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37084/0 (socket says 172.21.15.26:37084) 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.001+0000 7f63d899b700 1 -- 172.21.15.26:0/4166488028 learned_addr learned my addr 172.21.15.26:0/4166488028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 -- 172.21.15.26:0/4166488028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 msgr2=0x7f63d4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 0x7f63d4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 -- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 msgr2=0x7f63d4199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 -- 172.21.15.26:0/4166488028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0009520 con 0x7f63d410d550 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d3fff700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 0x7f63d4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d919c700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ef90 tx=0x7f63cc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d1ffb700 1 -- 172.21.15.26:0/4166488028 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc018070 con 0x7f63d410d550 2023-10-06T21:27:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c80095f0 con 0x7f63d410d550 2023-10-06T21:27:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d1ffb700 1 -- 172.21.15.26:0/4166488028 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63cc00f040 con 0x7f63d410d550 2023-10-06T21:27:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d1ffb700 1 -- 172.21.15.26:0/4166488028 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00cc30 con 0x7f63d410d550 2023-10-06T21:27:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.002+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41a5ca0 con 0x7f63d410d550 2023-10-06T21:27:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.007+0000 7f63d1ffb700 1 -- 172.21.15.26:0/4166488028 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f63cc003a00 con 0x7f63d410d550 2023-10-06T21:27:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.008+0000 7f63ee513b80 1 Processor -- start 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 -- start start 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d1ffb700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 0x7f63c40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 0x5640e40c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40c7a70 0x5640e40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e4025460 con 0x5640e40bcbc0 2023-10-06T21:27:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e4016c20 con 0x5640e40bc2a0 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63ee513b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e4014540 con 0x5640e40c7a70 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d1ffb700 1 -- 172.21.15.26:0/4166488028 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63cc0ccc20 con 0x7f63d410d550 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d899b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d899b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37086/0 (socket says 172.21.15.26:37086) 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d899b700 1 -- 172.21.15.26:0/1896153574 learned_addr learned my addr 172.21.15.26:0/1896153574 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d3fff700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 0x7f63c40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.009+0000 7f63d919c700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40c7a70 0x5640e40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d3fff700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 0x5640e40c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 -- 172.21.15.26:0/1896153574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40c7a70 msgr2=0x5640e40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40c7a70 0x5640e40c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 -- 172.21.15.26:0/1896153574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 msgr2=0x5640e40c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 0x5640e40c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 -- 172.21.15.26:0/1896153574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e3e3ade0 con 0x5640e40bc2a0 2023-10-06T21:27:55.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d3fff700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 0x5640e40c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d899b700 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b640 tx=0x7f63c800b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bc187068e1ae80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63d3fff700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 0x7f63c40a3bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63d419a660 tx=0x7f63c0009e50 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63beffd700 1 -- 172.21.15.26:0/1896153574 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c801d030 con 0x5640e40bc2a0 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63beffd700 1 -- 172.21.15.26:0/1896153574 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c801e040 con 0x5640e40bc2a0 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.010+0000 7f63beffd700 1 -- 172.21.15.26:0/1896153574 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004af0 con 0x5640e40bc2a0 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 -- 172.21.15.26:0/1896153574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 msgr2=0x5640e40bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b640 tx=0x7f63c800b670 comp rx=0 tx=0).stop 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 -- 172.21.15.26:0/1896153574 shutdown_connections 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40c7a70 0x5640e40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bc2a0 0x5640e40bc680 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 --2- 172.21.15.26:0/1896153574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bcbc0 0x5640e40c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 -- 172.21.15.26:0/1896153574 >> 172.21.15.26:0/1896153574 conn(0x5640e4031260 msgr2=0x5640e40b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 -- 172.21.15.26:0/1896153574 shutdown_connections 2023-10-06T21:27:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.011+0000 7f63ee513b80 1 -- 172.21.15.26:0/1896153574 wait complete. 2023-10-06T21:27:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.012+0000 7f63ee513b80 1 -- start start 2023-10-06T21:27:55.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.012+0000 7f63ee513b80 1 Processor -- start 2023-10-06T21:27:55.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bc2a0 0x5640e40c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40c7a70 0x5640e40c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e400fe30 con 0x5640e40c7a70 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e3fac710 con 0x5640e40bcbc0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e3fac3b0 con 0x5640e40bc2a0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63ee513b80 10 client.0 _ll_register_callbacks cb 0x5640e3e09800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37098/0 (socket says 172.21.15.26:37098) 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 -- 172.21.15.26:0/3655711096 learned_addr learned my addr 172.21.15.26:0/3655711096 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d899b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bc2a0 0x5640e40c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d919c700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40c7a70 0x5640e40c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 -- 172.21.15.26:0/3655711096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bc2a0 msgr2=0x5640e40c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bc2a0 0x5640e40c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 -- 172.21.15.26:0/3655711096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40c7a70 msgr2=0x5640e40c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.013+0000 7f63d3fff700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40c7a70 0x5640e40c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63d3fff700 1 -- 172.21.15.26:0/3655711096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d4110690 con 0x5640e40bcbc0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63d3fff700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f63c000fa80 tx=0x7f63c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b833f35fd4a3baa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0003d20 con 0x5640e40bcbc0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640e40b0160 con 0x5640e40bcbc0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63c001f430 con 0x5640e40bcbc0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c0003e80 con 0x5640e40bcbc0 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63ee513b80 10 client.27644 fetch_fsmap learned FSMap version 64 2023-10-06T21:27:55.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.014+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5640e3e23c60 con 0x5640e40bcbc0 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640e40b0160 con 0x5640e40bcbc0 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 64) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f63c0025020 con 0x5640e40bcbc0 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 10 client.27644 fetch_fsmap finished waiting for FSMap version 64 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 10 client.27644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 20 client.27644 populate_metadata read hostname 'smithi026' 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e40d7490 0x5640e40d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 4 client.27644 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e4008e10 con 0x5640e40d7490 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.015+0000 7f63d899b700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e40d7490 0x5640e40d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.016+0000 7f63d899b700 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e40d7490 0x5640e40d98b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cd6b7b5562b43f server_cookie=b6b9179b0897ec6c in_seq=0 out_seq=0 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.016+0000 7f63b3fff700 10 client.27644 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.016+0000 7f63b3fff700 1 -- 172.21.15.26:0/3655711096 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5640e4008e10 con 0x5640e40d7490 2023-10-06T21:27:55.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.016+0000 7f63b3fff700 10 client.27644 handle_command_reply: tid=0 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 client.27644 shutdown 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 msgr2=0x5640e40c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f63c000fa80 tx=0x7f63c0004750 comp rx=0 tx=0).stop 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 shutdown_connections 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40c7a70 0x5640e40c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bc2a0 0x5640e40c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e40d7490 0x5640e40d98b0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 --2- 172.21.15.26:0/3655711096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40bcbc0 0x5640e40c4cb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 >> 172.21.15.26:0/3655711096 conn(0x5640e4031260 msgr2=0x5640e40b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63b3fff700 0 client.27644 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:55.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 shutdown_connections 2023-10-06T21:27:55.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 1 -- 172.21.15.26:0/3655711096 wait complete. 2023-10-06T21:27:55.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.017+0000 7f63ee513b80 20 client.27644 trim_cache size 0 max 16384 2023-10-06T21:27:55.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:54 smithi167 ceph-mon[134494]: pgmap v553: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:27:55.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:54 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:27:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.045+0000 7f63ee513b80 1 Processor -- start 2023-10-06T21:27:55.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.045+0000 7f63ee513b80 1 -- start start 2023-10-06T21:27:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e40bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e3e90920 con 0x5640e40bc2a0 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e40b2720 con 0x5640e40c7a70 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63ee513b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e40d3e30 con 0x5640e40bcbc0 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e40bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:37102/0 (socket says 172.21.15.26:37102) 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 -- 172.21.15.26:0/597477369 learned_addr learned my addr 172.21.15.26:0/597477369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d3fff700 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 -- 172.21.15.26:0/597477369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 msgr2=0x5640e40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 -- 172.21.15.26:0/597477369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 msgr2=0x5640e40bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e40bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d919c700 1 -- 172.21.15.26:0/597477369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e3e61250 con 0x5640e40c7a70 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.046+0000 7f63d3fff700 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63d919c700 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f63cc096690 tx=0x7f63cc099b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f532c169d9cf29f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63bf7fe700 1 -- 172.21.15.26:0/597477369 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc09a320 con 0x5640e40c7a70 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63bf7fe700 1 -- 172.21.15.26:0/597477369 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63cc099c20 con 0x5640e40c7a70 2023-10-06T21:27:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63bf7fe700 1 -- 172.21.15.26:0/597477369 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc07db80 con 0x5640e40c7a70 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 -- 172.21.15.26:0/597477369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 msgr2=0x5640e40c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f63cc096690 tx=0x7f63cc099b20 comp rx=0 tx=0).stop 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 -- 172.21.15.26:0/597477369 shutdown_connections 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e40c1a20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e40bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 --2- 172.21.15.26:0/597477369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.047+0000 7f63ee513b80 1 -- 172.21.15.26:0/597477369 >> 172.21.15.26:0/597477369 conn(0x5640e4031260 msgr2=0x5640e40bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.048+0000 7f63ee513b80 1 -- 172.21.15.26:0/597477369 shutdown_connections 2023-10-06T21:27:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.048+0000 7f63ee513b80 1 -- 172.21.15.26:0/597477369 wait complete. 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.048+0000 7f63ee513b80 1 -- start start 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.048+0000 7f63ee513b80 1 Processor -- start 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e41254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e411f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e411f8f0 con 0x5640e40bc2a0 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e411fa60 con 0x5640e40c7a70 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e411fc20 con 0x5640e40bcbc0 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63ee513b80 10 client.0 _ll_register_callbacks cb 0x7f63c0004600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37732/0 (socket says 172.21.15.26:37732) 2023-10-06T21:27:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63d899b700 1 -- 172.21.15.26:0/802844987 learned_addr learned my addr 172.21.15.26:0/802844987 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63d919c700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e411f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.049+0000 7f63d3fff700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e41254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 -- 172.21.15.26:0/802844987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 msgr2=0x5640e41254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e41254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 -- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 msgr2=0x5640e411f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e411f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 -- 172.21.15.26:0/802844987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41a5ca0 con 0x5640e40bc2a0 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d919c700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e411f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63d899b700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b610 tx=0x7f63c800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5575007e1313fcad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.050+0000 7f63b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:27:55.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c801d040 con 0x5640e40bc2a0 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c801e040 con 0x5640e40bc2a0 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f63c80320c0 con 0x5640e40bc2a0 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c8032280 con 0x5640e40bc2a0 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63ee513b80 10 client.17268 fetch_fsmap learned FSMap version 64 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5640e3e392b0 con 0x5640e40bc2a0 2023-10-06T21:27:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c801e040 con 0x5640e40bc2a0 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 64) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f63c800fa60 con 0x5640e40bc2a0 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63ee513b80 10 client.17268 fetch_fsmap finished waiting for FSMap version 64 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.051+0000 7f63ee513b80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63ee513b80 20 client.17268 populate_metadata read hostname 'smithi026' 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e412c9d0 0x5640e412edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63ee513b80 4 client.17268 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640e4008b30 con 0x5640e412c9d0 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63d3fff700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e412c9d0 0x5640e412edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63d3fff700 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e412c9d0 0x5640e412edf0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ccfebb5b9cf769 server_cookie=5fd3e254201212d2 in_seq=0 out_seq=0 2023-10-06T21:27:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.052+0000 7f63b3fff700 10 client.17268 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63b3fff700 1 -- 172.21.15.26:0/802844987 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5640e4008b30 con 0x5640e412c9d0 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63b3fff700 10 client.17268 handle_command_reply: tid=0 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 client.17268 shutdown 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 msgr2=0x5640e4124f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f63c800b610 tx=0x7f63c800b390 comp rx=0 tx=0).stop 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 shutdown_connections 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5640e412c9d0 0x5640e412edf0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640e40c7a70 0x5640e411f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e40bc2a0 0x5640e4124f70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63b3fff700 0 client.17268 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 --2- 172.21.15.26:0/802844987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e40bcbc0 0x5640e41254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 >> 172.21.15.26:0/802844987 conn(0x5640e4031260 msgr2=0x5640e40b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 shutdown_connections 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 1 -- 172.21.15.26:0/802844987 wait complete. 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.053+0000 7f63ee513b80 20 client.17268 trim_cache size 0 max 16384 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:27:55.061 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:27:55.062 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 msgr2=0x7f63c40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 0x7f63c40a3bd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63d419a660 tx=0x7f63c0009e50 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 msgr2=0x7f63d419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ef90 tx=0x7f63cc00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 shutdown_connections 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63d410d550 0x7f63d419d9f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63c40a1710 0x7f63c40a3bd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d4073280 0x7f63d4199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 --2- 172.21.15.26:0/4166488028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d4073ba0 0x7f63d4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:27:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 >> 172.21.15.26:0/4166488028 conn(0x7f63d40fc7c0 msgr2=0x7f63d410f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:27:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 shutdown_connections 2023-10-06T21:27:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:27:55.054+0000 7f63d999d700 1 -- 172.21.15.26:0/4166488028 wait complete. 2023-10-06T21:27:55.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:54 smithi026 ceph-mon[98490]: pgmap v553: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:27:55.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:54 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:27:55.078 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:27:55.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:54 smithi078 ceph-mon[134896]: pgmap v553: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:27:55.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:54 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:27:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:56 smithi167 ceph-mon[134494]: pgmap v554: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:27:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:56 smithi026 ceph-mon[98490]: pgmap v554: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:27:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:56 smithi078 ceph-mon[134896]: pgmap v554: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:27:59.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:27:58 smithi167 ceph-mon[134494]: pgmap v555: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:27:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:27:58 smithi026 ceph-mon[98490]: pgmap v555: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:27:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:27:58 smithi078 ceph-mon[134896]: pgmap v555: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:00 smithi167 ceph-mon[134494]: pgmap v556: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:00 smithi026 ceph-mon[98490]: pgmap v556: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:00 smithi078 ceph-mon[134896]: pgmap v556: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:02.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:01 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:02.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:01 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:02.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:01 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:28:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:02 smithi167 ceph-mon[134494]: pgmap v557: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:28:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:02 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:02 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:02 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:28:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:02 smithi026 ceph-mon[98490]: pgmap v557: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:28:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:02 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:03.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:02 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:03.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:02 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:28:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:02 smithi078 ceph-mon[134896]: pgmap v557: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:28:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:02 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:28:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:02 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:28:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:02 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:28:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:04 smithi167 ceph-mon[134494]: pgmap v558: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:28:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:04 smithi026 ceph-mon[98490]: pgmap v558: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:28:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:04 smithi078 ceph-mon[134896]: pgmap v558: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:28:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:06 smithi167 ceph-mon[134494]: pgmap v559: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:28:07.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:06 smithi026 ceph-mon[98490]: pgmap v559: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:28:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:06 smithi078 ceph-mon[134896]: pgmap v559: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:28:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:08 smithi167 ceph-mon[134494]: pgmap v560: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:28:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:08 smithi026 ceph-mon[98490]: pgmap v560: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:28:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:08 smithi078 ceph-mon[134896]: pgmap v560: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:28:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:10 smithi167 ceph-mon[134494]: pgmap v561: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:28:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:10 smithi026 ceph-mon[98490]: pgmap v561: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:28:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:10 smithi078 ceph-mon[134896]: pgmap v561: 97 pgs: 97 active+clean; 3.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:28:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:11 smithi167 ceph-mon[134494]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:28:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:11 smithi026 ceph-mon[98490]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:28:12.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:11 smithi078 ceph-mon[134896]: osdmap e112: 12 total, 12 up, 12 in 2023-10-06T21:28:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:12 smithi167 ceph-mon[134494]: pgmap v563: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:28:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:12 smithi026 ceph-mon[98490]: pgmap v563: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:28:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:12 smithi078 ceph-mon[134896]: pgmap v563: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:28:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:14 smithi167 ceph-mon[134494]: pgmap v564: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:28:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:14 smithi026 ceph-mon[98490]: pgmap v564: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:28:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:14 smithi078 ceph-mon[134896]: pgmap v564: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-10-06T21:28:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:16 smithi167 ceph-mon[134494]: pgmap v565: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:16 smithi026 ceph-mon[98490]: pgmap v565: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:16 smithi078 ceph-mon[134896]: pgmap v565: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:19.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:18 smithi167 ceph-mon[134494]: pgmap v566: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:18 smithi167 ceph-mon[134494]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:28:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:18 smithi026 ceph-mon[98490]: pgmap v566: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:19.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:18 smithi026 ceph-mon[98490]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:28:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:18 smithi078 ceph-mon[134896]: pgmap v566: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:18 smithi078 ceph-mon[134896]: osdmap e113: 12 total, 12 up, 12 in 2023-10-06T21:28:20.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:19 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:20.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:19 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:20.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:19 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:20 smithi167 ceph-mon[134494]: pgmap v568: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:28:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:20 smithi026 ceph-mon[98490]: pgmap v568: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:28:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:20 smithi078 ceph-mon[134896]: pgmap v568: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:28:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:22 smithi167 ceph-mon[134494]: pgmap v569: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:22 smithi026 ceph-mon[98490]: pgmap v569: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:22 smithi078 ceph-mon[134896]: pgmap v569: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:24.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:24 smithi167 ceph-mon[134494]: pgmap v570: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:24 smithi026 ceph-mon[98490]: pgmap v570: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:24.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:24 smithi078 ceph-mon[134896]: pgmap v570: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:28:25.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:28:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 -- 172.21.15.26:0/2257663583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd581061d0 msgr2=0x7fcd5810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/2257663583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd581061d0 0x7fcd5810dd50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000b0a0 tx=0x7fcd5000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 -- 172.21.15.26:0/2257663583 shutdown_connections 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/2257663583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd581061d0 0x7fcd5810dd50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/2257663583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd58101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/2257663583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd58101a60 0x7fcd58105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 -- 172.21.15.26:0/2257663583 >> 172.21.15.26:0/2257663583 conn(0x7fcd580fc950 msgr2=0x7fcd580fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 -- 172.21.15.26:0/2257663583 shutdown_connections 2023-10-06T21:28:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.334+0000 7fcd5dbff700 1 -- 172.21.15.26:0/2257663583 wait complete. 2023-10-06T21:28:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.336+0000 7fcd5dbff700 1 Processor -- start 2023-10-06T21:28:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.336+0000 7fcd5dbff700 1 -- start start 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 0x7fcd580736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 0x7fcd581a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581a5960 con 0x7fcd581061d0 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581a5ad0 con 0x7fcd581010b0 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd5dbff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581a5c90 con 0x7fcd58073c30 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43884/0 (socket says 172.21.15.26:43884) 2023-10-06T21:28:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 -- 172.21.15.26:0/4132544931 learned_addr learned my addr 172.21.15.26:0/4132544931 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 0x7fcd580736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 0x7fcd581a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 -- 172.21.15.26:0/4132544931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 msgr2=0x7fcd581a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 0x7fcd581a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 -- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 msgr2=0x7fcd580736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 0x7fcd580736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd577fe700 1 -- 172.21.15.26:0/4132544931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009d00 con 0x7fcd581010b0 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 0x7fcd580736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.337+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 0x7fcd581a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd577fe700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9a0 tx=0x7fcd4800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd54ff9700 1 -- 172.21.15.26:0/4132544931 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48009790 con 0x7fcd581010b0 2023-10-06T21:28:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd54ff9700 1 -- 172.21.15.26:0/4132544931 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4800cbd0 con 0x7fcd581010b0 2023-10-06T21:28:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd581a5f70 con 0x7fcd581010b0 2023-10-06T21:28:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd54ff9700 1 -- 172.21.15.26:0/4132544931 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48018680 con 0x7fcd581010b0 2023-10-06T21:28:25.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.338+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd581a63f0 con 0x7fcd581010b0 2023-10-06T21:28:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.342+0000 7fcd54ff9700 1 -- 172.21.15.26:0/4132544931 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fcd4800cd40 con 0x7fcd581010b0 2023-10-06T21:28:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.342+0000 7fcd71521b80 1 Processor -- start 2023-10-06T21:28:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 -- start start 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a59350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3a5bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a39b7460 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a39a8c20 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd71521b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a39a6540 con 0x55e2a3a59890 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43900/0 (socket says 172.21.15.26:43900) 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a59350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.343+0000 7fcd57fff700 1 -- 172.21.15.26:0/3877893205 learned_addr learned my addr 172.21.15.26:0/3877893205 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3a5bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 -- 172.21.15.26:0/3877893205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 msgr2=0x55e2a3a5bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3a5bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 -- 172.21.15.26:0/3877893205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 msgr2=0x55e2a3a59350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a59350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 -- 172.21.15.26:0/3877893205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a37ccde0 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd577fe700 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a59350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:25.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000b780 tx=0x7fcd50009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97347dac117dbe5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.344+0000 7fcd3dffb700 1 -- 172.21.15.26:0/3877893205 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000e040 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd3dffb700 1 -- 172.21.15.26:0/3877893205 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50009460 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd3dffb700 1 -- 172.21.15.26:0/3877893205 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50004cb0 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd54ff9700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 0x7fcd400a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd54ff9700 1 -- 172.21.15.26:0/4132544931 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcd48014070 con 0x7fcd581010b0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd577fe700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 0x7fcd400a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 -- 172.21.15.26:0/3877893205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 msgr2=0x55e2a3a4e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000b780 tx=0x7fcd50009c80 comp rx=0 tx=0).stop 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 -- 172.21.15.26:0/3877893205 shutdown_connections 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3a5bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a4e4a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3877893205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a59350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.345+0000 7fcd71521b80 1 -- 172.21.15.26:0/3877893205 >> 172.21.15.26:0/3877893205 conn(0x55e2a39c3260 msgr2=0x55e2a3a4a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.346+0000 7fcd71521b80 1 -- 172.21.15.26:0/3877893205 shutdown_connections 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.346+0000 7fcd71521b80 1 -- 172.21.15.26:0/3877893205 wait complete. 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.346+0000 7fcd577fe700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 0x7fcd400a3ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd4809a380 tx=0x7fcd48096040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.346+0000 7fcd71521b80 1 -- start start 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.346+0000 7fcd71521b80 1 Processor -- start 2023-10-06T21:28:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a59240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a39a1e30 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a393e710 con 0x55e2a3a59890 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a393e3b0 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd71521b80 10 client.0 _ll_register_callbacks cb 0x55e2a379b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd56ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43906/0 (socket says 172.21.15.26:43906) 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.347+0000 7fcd57fff700 1 -- 172.21.15.26:0/3392022899 learned_addr learned my addr 172.21.15.26:0/3392022899 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a59240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 -- 172.21.15.26:0/3392022899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 msgr2=0x55e2a3a59240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a59240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 -- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 msgr2=0x55e2a3a56af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 -- 172.21.15.26:0/3392022899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5810e5c0 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a564c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806a470 tx=0x7fcd4c012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd482478a1a066e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.348+0000 7fcd36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:28:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c009690 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcd4c004230 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2a3a41f10 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4c0043f0 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd71521b80 10 client.17280 fetch_fsmap learned FSMap version 65 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55e2a37b5c60 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4c028080 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd4c010970 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd71521b80 10 client.17280 fetch_fsmap finished waiting for FSMap version 65 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.349+0000 7fcd71521b80 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd71521b80 20 client.17280 populate_metadata read hostname 'smithi026' 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3a693b0 0x55e2a3a6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd71521b80 4 client.17280 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2a399ae10 con 0x55e2a3a693b0 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3a693b0 0x55e2a3a6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd57fff700 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3a693b0 0x55e2a3a6b7d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1d275569919490 server_cookie=9aeae563fb4ae55b in_seq=0 out_seq=0 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.350+0000 7fcd36ffd700 10 client.17280 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd36ffd700 1 -- 172.21.15.26:0/3392022899 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7fcd50033730 con 0x55e2a3a693b0 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd36ffd700 10 client.17280 handle_command_reply: tid=0 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 client.17280 shutdown 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 msgr2=0x55e2a3a564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a564c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806a470 tx=0x7fcd4c012a30 comp rx=0 tx=0).stop 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 shutdown_connections 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3a693b0 0x55e2a3a6b7d0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a59890 0x55e2a3a56af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a59240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd36ffd700 0 client.17280 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 --2- 172.21.15.26:0/3392022899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a564c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.351+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 >> 172.21.15.26:0/3392022899 conn(0x55e2a39c3260 msgr2=0x55e2a3a49fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.352+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 shutdown_connections 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.352+0000 7fcd71521b80 1 -- 172.21.15.26:0/3392022899 wait complete. 2023-10-06T21:28:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.352+0000 7fcd71521b80 20 client.17280 trim_cache size 0 max 16384 2023-10-06T21:28:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 Processor -- start 2023-10-06T21:28:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 -- start start 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a50f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a59890 0x55e2a3a53750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3822920 con 0x55e2a3a59890 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3a49240 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.375+0000 7fcd71521b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3a52330 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43916/0 (socket says 172.21.15.26:43916) 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 -- 172.21.15.26:0/4171366898 learned_addr learned my addr 172.21.15.26:0/4171366898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a50f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a59890 0x55e2a3a53750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 -- 172.21.15.26:0/4171366898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 msgr2=0x55e2a3a50f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a50f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 -- 172.21.15.26:0/4171366898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a59890 msgr2=0x55e2a3a53750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a59890 0x55e2a3a53750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 -- 172.21.15.26:0/4171366898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a37f3250 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.376+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a010 tx=0x7fcd4c005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d1ebe87a68f266 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd3e7fc700 1 -- 172.21.15.26:0/4171366898 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c009690 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd3e7fc700 1 -- 172.21.15.26:0/4171366898 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2a3a41f10 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd3e7fc700 1 -- 172.21.15.26:0/4171366898 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0038e0 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 -- 172.21.15.26:0/4171366898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 msgr2=0x55e2a3a60380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a010 tx=0x7fcd4c005980 comp rx=0 tx=0).stop 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 -- 172.21.15.26:0/4171366898 shutdown_connections 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a59890 0x55e2a3a53750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3a50f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4171366898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3a60380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.377+0000 7fcd71521b80 1 -- 172.21.15.26:0/4171366898 >> 172.21.15.26:0/4171366898 conn(0x55e2a39c3260 msgr2=0x55e2a3a517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.378+0000 7fcd71521b80 1 -- 172.21.15.26:0/4171366898 shutdown_connections 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.378+0000 7fcd71521b80 1 -- 172.21.15.26:0/4171366898 wait complete. 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.378+0000 7fcd71521b80 1 -- start start 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.378+0000 7fcd71521b80 1 Processor -- start 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3ab5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3ab1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3ab1700 con 0x55e2a3a4e9e0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3ab1840 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a3ab1a00 con 0x55e2a3a59890 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd71521b80 10 client.0 _ll_register_callbacks cb 0x7fcd4c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd56ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3ab5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:43932/0 (socket says 172.21.15.26:43932) 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 -- 172.21.15.26:0/4009998684 learned_addr learned my addr 172.21.15.26:0/4009998684 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3ab1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 -- 172.21.15.26:0/4009998684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 msgr2=0x55e2a3ab1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3ab1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 -- 172.21.15.26:0/4009998684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 msgr2=0x55e2a3ab5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3ab5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd57fff700 1 -- 172.21.15.26:0/4009998684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2a37b5c60 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.379+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3ab5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd57fff700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000bbc0 tx=0x7fcd5000bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a0dcbd93761238 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000e040 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcd500076b0 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd50009460 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd50007870 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd71521b80 10 client.27668 fetch_fsmap learned FSMap version 65 2023-10-06T21:28:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.380+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55e2a37cb2b0 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50045020 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 65) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd50019040 con 0x55e2a3a4e0c0 2023-10-06T21:28:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 10 client.27668 fetch_fsmap finished waiting for FSMap version 65 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 20 client.27668 populate_metadata read hostname 'smithi026' 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3abe710 0x55e2a3ac0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 4 client.27668 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e2a399ab30 con 0x55e2a3abe710 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3abe710 0x55e2a3ac0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd56ffd700 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3abe710 0x55e2a3ac0b30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90476bfe412b73 server_cookie=a548f7cb1de73869 in_seq=0 out_seq=0 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.381+0000 7fcd36ffd700 10 client.27668 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd36ffd700 1 -- 172.21.15.26:0/4009998684 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e2a399ab30 con 0x55e2a3abe710 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd36ffd700 10 client.27668 handle_command_reply: tid=0 2023-10-06T21:28:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 client.27668 shutdown 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 msgr2=0x55e2a3ab5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000bbc0 tx=0x7fcd5000bbf0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 shutdown_connections 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2a3a59890 0x55e2a3ab1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55e2a3abe710 0x55e2a3ac0b30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a3a4e0c0 0x55e2a3ab5a60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 --2- 172.21.15.26:0/4009998684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2a3a4e9e0 0x55e2a3ab5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 >> 172.21.15.26:0/4009998684 conn(0x55e2a39c3260 msgr2=0x55e2a3a57dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd36ffd700 0 client.27668 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 shutdown_connections 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 1 -- 172.21.15.26:0/4009998684 wait complete. 2023-10-06T21:28:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.382+0000 7fcd71521b80 20 client.27668 trim_cache size 0 max 16384 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:28:25.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:28:25.392 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:28:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 msgr2=0x7fcd400a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 0x7fcd400a3ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcd4809a380 tx=0x7fcd48096040 comp rx=0 tx=0).stop 2023-10-06T21:28:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 msgr2=0x7fcd580731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9a0 tx=0x7fcd4800ed60 comp rx=0 tx=0).stop 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 shutdown_connections 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd58073c30 0x7fcd581a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd581010b0 0x7fcd580731b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fcd400a15f0 0x7fcd400a3ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 --2- 172.21.15.26:0/4132544931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd581061d0 0x7fcd580736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 >> 172.21.15.26:0/4132544931 conn(0x7fcd580fc950 msgr2=0x7fcd580fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.383+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 shutdown_connections 2023-10-06T21:28:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:25.384+0000 7fcd5dbff700 1 -- 172.21.15.26:0/4132544931 wait complete. 2023-10-06T21:28:25.409 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:28:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:26 smithi167 ceph-mon[134494]: pgmap v571: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:26 smithi026 ceph-mon[98490]: pgmap v571: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:26 smithi078 ceph-mon[134896]: pgmap v571: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:28 smithi167 ceph-mon[134494]: pgmap v572: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:28 smithi026 ceph-mon[98490]: pgmap v572: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:28 smithi078 ceph-mon[134896]: pgmap v572: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:28:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:30 smithi167 ceph-mon[134494]: pgmap v573: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:28:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:30 smithi026 ceph-mon[98490]: pgmap v573: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:28:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:30 smithi078 ceph-mon[134896]: pgmap v573: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:28:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:32 smithi167 ceph-mon[134494]: pgmap v574: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:28:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:32 smithi026 ceph-mon[98490]: pgmap v574: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:28:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:32 smithi078 ceph-mon[134896]: pgmap v574: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:28:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:34 smithi167 ceph-mon[134494]: pgmap v575: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:34 smithi026 ceph-mon[98490]: pgmap v575: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:34 smithi078 ceph-mon[134896]: pgmap v575: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:36 smithi167 ceph-mon[134494]: pgmap v576: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:36 smithi026 ceph-mon[98490]: pgmap v576: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:37.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:36 smithi078 ceph-mon[134896]: pgmap v576: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:38 smithi167 ceph-mon[134494]: pgmap v577: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:38 smithi026 ceph-mon[98490]: pgmap v577: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:38 smithi078 ceph-mon[134896]: pgmap v577: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:40 smithi167 ceph-mon[134494]: pgmap v578: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:40 smithi026 ceph-mon[98490]: pgmap v578: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:40 smithi078 ceph-mon[134896]: pgmap v578: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:43.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:42 smithi167 ceph-mon[134494]: pgmap v579: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:42 smithi026 ceph-mon[98490]: pgmap v579: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:42 smithi078 ceph-mon[134896]: pgmap v579: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:44 smithi167 ceph-mon[134494]: pgmap v580: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:44 smithi026 ceph-mon[98490]: pgmap v580: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:44 smithi078 ceph-mon[134896]: pgmap v580: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:46 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:46.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:46 smithi167 ceph-mon[134494]: pgmap v581: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:46.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:46 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:46.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:46 smithi026 ceph-mon[98490]: pgmap v581: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:46.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:46 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:46.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:46 smithi078 ceph-mon[134896]: pgmap v581: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:48 smithi167 ceph-mon[134494]: pgmap v582: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:48 smithi026 ceph-mon[98490]: pgmap v582: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:48 smithi078 ceph-mon[134896]: pgmap v582: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:50 smithi167 ceph-mon[134494]: pgmap v583: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:50 smithi026 ceph-mon[98490]: pgmap v583: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:50 smithi078 ceph-mon[134896]: pgmap v583: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:52 smithi167 ceph-mon[134494]: pgmap v584: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:52 smithi026 ceph-mon[98490]: pgmap v584: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:52 smithi078 ceph-mon[134896]: pgmap v584: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:54 smithi167 ceph-mon[134494]: pgmap v585: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:54 smithi026 ceph-mon[98490]: pgmap v585: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:54 smithi078 ceph-mon[134896]: pgmap v585: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:28:55.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:28:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 -- 172.21.15.26:0/3139154443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c10b470 msgr2=0x7fe61c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 --2- 172.21.15.26:0/3139154443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c10b470 0x7fe61c10d930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe61400b0a0 tx=0x7fe61400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 -- 172.21.15.26:0/3139154443 shutdown_connections 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 --2- 172.21.15.26:0/3139154443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c10b470 0x7fe61c10d930 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 --2- 172.21.15.26:0/3139154443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c102050 0x7fe61c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 --2- 172.21.15.26:0/3139154443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 0x7fe61c10ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 -- 172.21.15.26:0/3139154443 >> 172.21.15.26:0/3139154443 conn(0x7fe61c0fb890 msgr2=0x7fe61c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 -- 172.21.15.26:0/3139154443 shutdown_connections 2023-10-06T21:28:55.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.635+0000 7fe620fff700 1 -- 172.21.15.26:0/3139154443 wait complete. 2023-10-06T21:28:55.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.637+0000 7fe620fff700 1 Processor -- start 2023-10-06T21:28:55.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.637+0000 7fe620fff700 1 -- start start 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.637+0000 7fe620fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.637+0000 7fe620fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 0x7fe61c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe620fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 0x7fe61c1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe620fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c100a10 con 0x7fe61c102970 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe620fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1a57a0 con 0x7fe61c102050 2023-10-06T21:28:55.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe620fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1a58e0 con 0x7fe61c10b470 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57722/0 (socket says 172.21.15.26:57722) 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 -- 172.21.15.26:0/645455802 learned_addr learned my addr 172.21.15.26:0/645455802 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe619d9b700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 0x7fe61c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 0x7fe61c1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 -- 172.21.15.26:0/645455802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 msgr2=0x7fe61c1a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 0x7fe61c1a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 -- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 msgr2=0x7fe61c1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 0x7fe61c1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 -- 172.21.15.26:0/645455802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614009d00 con 0x7fe61c102050 2023-10-06T21:28:55.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 0x7fe61c1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:28:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.638+0000 7fe61a59c700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9a0 tx=0x7fe60c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.639+0000 7fe60b7fe700 1 -- 172.21.15.26:0/645455802 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c009790 con 0x7fe61c102050 2023-10-06T21:28:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.639+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c1a5b40 con 0x7fe61c102050 2023-10-06T21:28:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.639+0000 7fe60b7fe700 1 -- 172.21.15.26:0/645455802 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe60c00cbd0 con 0x7fe61c102050 2023-10-06T21:28:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.639+0000 7fe60b7fe700 1 -- 172.21.15.26:0/645455802 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c018680 con 0x7fe61c102050 2023-10-06T21:28:55.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.639+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c1a6020 con 0x7fe61c102050 2023-10-06T21:28:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.644+0000 7fe634839b80 1 Processor -- start 2023-10-06T21:28:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.644+0000 7fe60b7fe700 1 -- 172.21.15.26:0/645455802 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe60c00cd40 con 0x7fe61c102050 2023-10-06T21:28:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.644+0000 7fe634839b80 1 -- start start 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.644+0000 7fe634839b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.644+0000 7fe634839b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x56255658c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe634839b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 0x56255658ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe634839b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625564ea460 con 0x56255658c890 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe634839b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625564dbc20 con 0x5625565810c0 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe634839b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625564d9540 con 0x5625565819e0 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57734/0 (socket says 172.21.15.26:57734) 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 -- 172.21.15.26:0/1659231304 learned_addr learned my addr 172.21.15.26:0/1659231304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61a59c700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x56255658c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe619d9b700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 0x56255658ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 -- 172.21.15.26:0/1659231304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 msgr2=0x56255658c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x56255658c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 -- 172.21.15.26:0/1659231304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 msgr2=0x56255658ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 0x56255658ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe61ad9d700 1 -- 172.21.15.26:0/1659231304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625562ffde0 con 0x5625565810c0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.645+0000 7fe619d9b700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 0x56255658ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe614009c20 tx=0x7fe614009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62948900010c475d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe60b7fe700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 0x7fe6040a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe608ff9700 1 -- 172.21.15.26:0/1659231304 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400e040 con 0x5625565810c0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe608ff9700 1 -- 172.21.15.26:0/1659231304 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe614009490 con 0x5625565810c0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe608ff9700 1 -- 172.21.15.26:0/1659231304 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61401d6d0 con 0x5625565810c0 2023-10-06T21:28:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe60b7fe700 1 -- 172.21.15.26:0/645455802 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe60c014070 con 0x7fe61c102050 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.646+0000 7fe61a59c700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 0x7fe6040a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 -- 172.21.15.26:0/1659231304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 msgr2=0x5625565814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe61a59c700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 0x7fe6040a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0cac30 tx=0x7fe60c0d0d90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe614009c20 tx=0x7fe614009c50 comp rx=0 tx=0).stop 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 -- 172.21.15.26:0/1659231304 shutdown_connections 2023-10-06T21:28:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56255658c890 0x56255658ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565810c0 0x5625565814a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 --2- 172.21.15.26:0/1659231304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x56255658c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 -- 172.21.15.26:0/1659231304 >> 172.21.15.26:0/1659231304 conn(0x5625564f6260 msgr2=0x56255657d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 -- 172.21.15.26:0/1659231304 shutdown_connections 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.647+0000 7fe634839b80 1 -- 172.21.15.26:0/1659231304 wait complete. 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.648+0000 7fe634839b80 1 -- start start 2023-10-06T21:28:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.648+0000 7fe634839b80 1 Processor -- start 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x56255658c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x562556589c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625564d4e30 con 0x5625565810c0 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556471710 con 0x56255658c890 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625564713b0 con 0x5625565819e0 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe634839b80 10 client.0 _ll_register_callbacks cb 0x5625562ce800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe61ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x56255658c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe619d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:59888/0 (socket says 172.21.15.26:59888) 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe619d9b700 1 -- 172.21.15.26:0/1167789765 learned_addr learned my addr 172.21.15.26:0/1167789765 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.649+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x562556589c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 -- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 msgr2=0x562556589c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x562556589c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 -- 172.21.15.26:0/1167789765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 msgr2=0x56255658c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x56255658c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 -- 172.21.15.26:0/1167789765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c10e5f0 con 0x5625565819e0 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe619d9b700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe610009bc0 tx=0x7fe610009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d0e6fcc8979ccf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe601ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61001b440 con 0x5625565819e0 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562556574f10 con 0x5625565819e0 2023-10-06T21:28:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.650+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe610011440 con 0x5625565819e0 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe610011600 con 0x5625565819e0 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 10 client.27517 fetch_fsmap learned FSMap version 67 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5625562e8c60 con 0x5625565819e0 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562556574f10 con 0x5625565819e0 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe61001b980 con 0x5625565819e0 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 10 client.27517 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 10 client.27517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 20 client.27517 populate_metadata read hostname 'smithi026' 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56255659c260 0x56255659e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 4 client.27517 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.651+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625564cde10 con 0x56255659c260 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.652+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56255659c260 0x56255659e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.652+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56255659c260 0x56255659e680 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95de79022a7b332 server_cookie=633609d4d4a842ef in_seq=0 out_seq=0 2023-10-06T21:28:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.652+0000 7fe601ffb700 10 client.27517 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe601ffb700 1 -- 172.21.15.26:0/1167789765 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5625564cde10 con 0x56255659c260 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe601ffb700 10 client.27517 handle_command_reply: tid=0 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 client.27517 shutdown 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 msgr2=0x562556589510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe610009bc0 tx=0x7fe610009ed0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 shutdown_connections 2023-10-06T21:28:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x56255659c260 0x56255659e680 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x562556589c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x56255658c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 --2- 172.21.15.26:0/1167789765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565819e0 0x562556589510 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe601ffb700 0 client.27517 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 >> 172.21.15.26:0/1167789765 conn(0x5625564f6260 msgr2=0x56255657cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 shutdown_connections 2023-10-06T21:28:55.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.653+0000 7fe634839b80 1 -- 172.21.15.26:0/1167789765 wait complete. 2023-10-06T21:28:55.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.654+0000 7fe634839b80 20 client.27517 trim_cache size 0 max 16384 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 Processor -- start 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 -- start start 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x562556583f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56255658c890 0x562556588db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556355920 con 0x5625565810c0 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625565814d0 con 0x5625565819e0 2023-10-06T21:28:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.672+0000 7fe634839b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556588a80 con 0x56255658c890 2023-10-06T21:28:55.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57750/0 (socket says 172.21.15.26:57750) 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 -- 172.21.15.26:0/2257611369 learned_addr learned my addr 172.21.15.26:0/2257611369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x562556583f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 -- 172.21.15.26:0/2257611369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56255658c890 msgr2=0x562556588db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56255658c890 0x562556588db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 -- 172.21.15.26:0/2257611369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 msgr2=0x562556583f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x562556583f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 -- 172.21.15.26:0/2257611369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562556326250 con 0x5625565819e0 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.673+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe610000c00 tx=0x7fe610003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611e03a0ee89d9c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe6097fa700 1 -- 172.21.15.26:0/2257611369 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610005040 con 0x5625565819e0 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe6097fa700 1 -- 172.21.15.26:0/2257611369 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe610005660 con 0x5625565819e0 2023-10-06T21:28:55.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe6097fa700 1 -- 172.21.15.26:0/2257611369 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610004710 con 0x5625565819e0 2023-10-06T21:28:55.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 -- 172.21.15.26:0/2257611369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 msgr2=0x56255658be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe610000c00 tx=0x7fe610003960 comp rx=0 tx=0).stop 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 -- 172.21.15.26:0/2257611369 shutdown_connections 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56255658c890 0x562556588db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565810c0 0x562556583f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 --2- 172.21.15.26:0/2257611369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625565819e0 0x56255658be30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 -- 172.21.15.26:0/2257611369 >> 172.21.15.26:0/2257611369 conn(0x5625564f6260 msgr2=0x5625565847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 -- 172.21.15.26:0/2257611369 shutdown_connections 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.674+0000 7fe634839b80 1 -- 172.21.15.26:0/2257611369 wait complete. 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.675+0000 7fe634839b80 1 -- start start 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.675+0000 7fe634839b80 1 Processor -- start 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.675+0000 7fe634839b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565810c0 0x5625565930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565819e0 0x5625565913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556591e20 con 0x5625565819e0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556591f90 con 0x56255658c890 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x562556592150 con 0x5625565810c0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe634839b80 10 client.0 _ll_register_callbacks cb 0x7fe610004600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57764/0 (socket says 172.21.15.26:57764) 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 -- 172.21.15.26:0/2905984344 learned_addr learned my addr 172.21.15.26:0/2905984344 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565819e0 0x5625565913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565810c0 0x5625565930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 -- 172.21.15.26:0/2905984344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565810c0 msgr2=0x5625565930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565810c0 0x5625565930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 -- 172.21.15.26:0/2905984344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565819e0 msgr2=0x5625565913a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565819e0 0x5625565913a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.676+0000 7fe61ad9d700 1 -- 172.21.15.26:0/2905984344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625562e8c60 con 0x56255658c890 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe61ad9d700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe614000f80 tx=0x7fe61401f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a380b6ea228d7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61400e040 con 0x56255658c890 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe614009490 con 0x56255658c890 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe61400bab0 con 0x56255658c890 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe614004dd0 con 0x56255658c890 2023-10-06T21:28:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 10 client.27686 fetch_fsmap learned FSMap version 67 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5625562fe2b0 con 0x56255658c890 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61403e080 con 0x56255658c890 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe614024070 con 0x56255658c890 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 10 client.27686 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 10 client.27686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 20 client.27686 populate_metadata read hostname 'smithi026' 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625565f1020 0x5625565f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 4 client.27686 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.677+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625564cdb30 con 0x5625565f1020 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625565f1020 0x5625565f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:28:55.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe619d9b700 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625565f1020 0x5625565f3440 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea548720fd9fa19 server_cookie=9b12afd6e36443c2 in_seq=0 out_seq=0 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe601ffb700 10 client.27686 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe601ffb700 1 -- 172.21.15.26:0/2905984344 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5625564cdb30 con 0x5625565f1020 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe601ffb700 10 client.27686 handle_command_reply: tid=0 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.678+0000 7fe634839b80 1 client.27686 shutdown 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 msgr2=0x5625565918e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe614000f80 tx=0x7fe61401f040 comp rx=0 tx=0).stop 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 shutdown_connections 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625565f1020 0x5625565f3440 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56255658c890 0x5625565918e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625565810c0 0x5625565930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 --2- 172.21.15.26:0/2905984344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625565819e0 0x5625565913a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe601ffb700 0 client.27686 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:28:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 >> 172.21.15.26:0/2905984344 conn(0x5625564f6260 msgr2=0x562556588db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 shutdown_connections 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 1 -- 172.21.15.26:0/2905984344 wait complete. 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.679+0000 7fe634839b80 20 client.27686 trim_cache size 0 max 16384 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:28:55.688 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:28:55.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:28:55.689 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:28:55.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:28:55.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:28:55.689 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:28:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 msgr2=0x7fe6040a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 0x7fe6040a3ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0cac30 tx=0x7fe60c0d0d90 comp rx=0 tx=0).stop 2023-10-06T21:28:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 msgr2=0x7fe61c0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:28:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9a0 tx=0x7fe60c00ed60 comp rx=0 tx=0).stop 2023-10-06T21:28:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 shutdown_connections 2023-10-06T21:28:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe61c10b470 0x7fe61c1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe61c102050 0x7fe61c0ffca0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe6040a15f0 0x7fe6040a3ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 --2- 172.21.15.26:0/645455802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe61c102970 0x7fe61c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:28:55.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 >> 172.21.15.26:0/645455802 conn(0x7fe61c0fb890 msgr2=0x7fe61c10d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:28:55.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 shutdown_connections 2023-10-06T21:28:55.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:28:55.680+0000 7fe620fff700 1 -- 172.21.15.26:0/645455802 wait complete. 2023-10-06T21:28:55.703 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:28:56.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:56 smithi167 ceph-mon[134494]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:56.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:56 smithi167 ceph-mon[134494]: pgmap v586: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:56 smithi026 ceph-mon[98490]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:56.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:56 smithi026 ceph-mon[98490]: pgmap v586: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:56 smithi078 ceph-mon[134896]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-06T21:28:56.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:56 smithi078 ceph-mon[134896]: pgmap v586: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:28:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:28:58 smithi167 ceph-mon[134494]: pgmap v587: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:28:58 smithi026 ceph-mon[98490]: pgmap v587: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:28:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:28:58 smithi078 ceph-mon[134896]: pgmap v587: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:00 smithi167 ceph-mon[134494]: pgmap v588: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:00 smithi026 ceph-mon[98490]: pgmap v588: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:00 smithi078 ceph-mon[134896]: pgmap v588: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:02 smithi167 ceph-mon[134494]: pgmap v589: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:29:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:02 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:02 smithi026 ceph-mon[98490]: pgmap v589: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:29:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:02 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:02 smithi078 ceph-mon[134896]: pgmap v589: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:29:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:02 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:29:04.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:03 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:04.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:03 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:04.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:03 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:29:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:03 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:03 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:04.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:03 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:29:04.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:03 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:29:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:03 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:29:04.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:03 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:29:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:04 smithi167 ceph-mon[134494]: pgmap v590: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:29:05.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:04 smithi026 ceph-mon[98490]: pgmap v590: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:29:05.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:04 smithi078 ceph-mon[134896]: pgmap v590: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:29:07.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:06 smithi167 ceph-mon[134494]: pgmap v591: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:29:07.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:06 smithi026 ceph-mon[98490]: pgmap v591: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:29:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:06 smithi078 ceph-mon[134896]: pgmap v591: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-10-06T21:29:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:08 smithi167 ceph-mon[134494]: pgmap v592: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:29:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:08 smithi026 ceph-mon[98490]: pgmap v592: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:29:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:08 smithi078 ceph-mon[134896]: pgmap v592: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:29:11.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:10 smithi167 ceph-mon[134494]: pgmap v593: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:29:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:10 smithi026 ceph-mon[98490]: pgmap v593: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:29:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:10 smithi078 ceph-mon[134896]: pgmap v593: 97 pgs: 97 active+clean; 3.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:29:12.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:11 smithi167 ceph-mon[134494]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:29:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:11 smithi026 ceph-mon[98490]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:29:12.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:11 smithi078 ceph-mon[134896]: osdmap e114: 12 total, 12 up, 12 in 2023-10-06T21:29:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:12 smithi167 ceph-mon[134494]: pgmap v595: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:29:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:12 smithi026 ceph-mon[98490]: pgmap v595: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:29:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:12 smithi078 ceph-mon[134896]: pgmap v595: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:29:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:14 smithi167 ceph-mon[134494]: pgmap v596: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:29:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:14 smithi026 ceph-mon[98490]: pgmap v596: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:29:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:14 smithi078 ceph-mon[134896]: pgmap v596: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:29:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:16 smithi167 ceph-mon[134494]: pgmap v597: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:16 smithi026 ceph-mon[98490]: pgmap v597: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:16 smithi078 ceph-mon[134896]: pgmap v597: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:19.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:18 smithi167 ceph-mon[134494]: pgmap v598: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:18 smithi167 ceph-mon[134494]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:29:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:18 smithi026 ceph-mon[98490]: pgmap v598: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:18 smithi026 ceph-mon[98490]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:29:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:18 smithi078 ceph-mon[134896]: pgmap v598: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:18 smithi078 ceph-mon[134896]: osdmap e115: 12 total, 12 up, 12 in 2023-10-06T21:29:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:20 smithi167 ceph-mon[134494]: pgmap v600: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:29:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:20 smithi026 ceph-mon[98490]: pgmap v600: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:29:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:20 smithi078 ceph-mon[134896]: pgmap v600: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:29:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:22 smithi167 ceph-mon[134494]: pgmap v601: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:22 smithi026 ceph-mon[98490]: pgmap v601: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:22 smithi078 ceph-mon[134896]: pgmap v601: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:24.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:24 smithi167 ceph-mon[134494]: pgmap v602: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:24 smithi026 ceph-mon[98490]: pgmap v602: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:24 smithi078 ceph-mon[134896]: pgmap v602: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:29:25.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:29:25.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 -- 172.21.15.26:0/2167169103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101810 msgr2=0x7f16bc105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/2167169103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101810 0x7f16bc105850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009990 tx=0x7f16ac009ca0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 -- 172.21.15.26:0/2167169103 shutdown_connections 2023-10-06T21:29:25.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/2167169103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc105f80 0x7f16bc10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/2167169103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc100e60 0x7f16bc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/2167169103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101810 0x7f16bc105850 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 -- 172.21.15.26:0/2167169103 >> 172.21.15.26:0/2167169103 conn(0x7f16bc0fc6c0 msgr2=0x7f16bc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 -- 172.21.15.26:0/2167169103 shutdown_connections 2023-10-06T21:29:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.927+0000 7f16c2f9d700 1 -- 172.21.15.26:0/2167169103 wait complete. 2023-10-06T21:29:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.929+0000 7f16c2f9d700 1 Processor -- start 2023-10-06T21:29:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.929+0000 7f16c2f9d700 1 -- start start 2023-10-06T21:29:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc100e60 0x7f16bc198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc19dad0 0x7f16bc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19e550 con 0x7f16bc100e60 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19e6c0 con 0x7f16bc105f80 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c2f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19e880 con 0x7f16bc19dad0 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc100e60 0x7f16bc198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34874/0 (socket says 172.21.15.26:34874) 2023-10-06T21:29:25.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 -- 172.21.15.26:0/1599792941 learned_addr learned my addr 172.21.15.26:0/1599792941 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c279c700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc19dad0 0x7f16bc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 -- 172.21.15.26:0/1599792941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc19dad0 msgr2=0x7f16bc19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc19dad0 0x7f16bc19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 -- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc100e60 msgr2=0x7f16bc198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc100e60 0x7f16bc198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.930+0000 7f16c179a700 1 -- 172.21.15.26:0/1599792941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0095f0 con 0x7f16bc105f80 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16c179a700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16b2ffd700 1 -- 172.21.15.26:0/1599792941 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01d070 con 0x7f16bc105f80 2023-10-06T21:29:25.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16b2ffd700 1 -- 172.21.15.26:0/1599792941 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ac021410 con 0x7f16bc105f80 2023-10-06T21:29:25.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16b2ffd700 1 -- 172.21.15.26:0/1599792941 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac017650 con 0x7f16bc105f80 2023-10-06T21:29:25.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc1a5d80 con 0x7f16bc105f80 2023-10-06T21:29:25.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.931+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a61b0 con 0x7f16bc105f80 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.935+0000 7f16b2ffd700 1 -- 172.21.15.26:0/1599792941 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f16ac021580 con 0x7f16bc105f80 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 Processor -- start 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 -- start start 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 0x560e1babe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bac9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 0x560e1bacbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1ba27460 con 0x560e1babe2a0 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1ba18c20 con 0x560e1bac9a70 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.936+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1ba16540 con 0x560e1babebc0 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c279c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 0x560e1babe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 0x560e1bacbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c279c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 0x560e1babe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47062/0 (socket says 172.21.15.26:47062) 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c279c700 1 -- 172.21.15.26:0/1342142166 learned_addr learned my addr 172.21.15.26:0/1342142166 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bac9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16b2ffd700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 0x7f16a80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 -- 172.21.15.26:0/1342142166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 msgr2=0x560e1bac9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bac9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 -- 172.21.15.26:0/1342142166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 msgr2=0x560e1babe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 0x560e1babe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c179a700 1 -- 172.21.15.26:0/1342142166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e1b83cde0 con 0x560e1bac9a70 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.937+0000 7f16c279c700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 0x7f16a80a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16b2ffd700 1 -- 172.21.15.26:0/1599792941 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f16ac0ce420 con 0x7f16bc105f80 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16c179a700 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 0x560e1bacbe60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0cbb00 tx=0x7f16ac090b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd69f58439473968 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16a7fff700 1 -- 172.21.15.26:0/1342142166 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0d19c0 con 0x560e1bac9a70 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16a7fff700 1 -- 172.21.15.26:0/1342142166 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ac09b1f0 con 0x560e1bac9a70 2023-10-06T21:29:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16a7fff700 1 -- 172.21.15.26:0/1342142166 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac06b420 con 0x560e1bac9a70 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.938+0000 7f16c279c700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 0x7f16a80a3b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f16b40075c0 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1342142166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 msgr2=0x560e1bacbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 0x560e1bacbe60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0cbb00 tx=0x7f16ac090b90 comp rx=0 tx=0).stop 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1342142166 shutdown_connections 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1bac9a70 0x560e1bacbe60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1babe2a0 0x560e1babe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1342142166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bac9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1342142166 >> 172.21.15.26:0/1342142166 conn(0x560e1ba33260 msgr2=0x560e1baba630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1342142166 shutdown_connections 2023-10-06T21:29:25.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.939+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1342142166 wait complete. 2023-10-06T21:29:25.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.940+0000 7f16d7b5bb80 1 -- start start 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.940+0000 7f16d7b5bb80 1 Processor -- start 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1baca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1baca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1ba11e30 con 0x560e1bac9a70 2023-10-06T21:29:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1b9ae710 con 0x560e1babe2a0 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1b9ae3b0 con 0x560e1babebc0 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16d7b5bb80 10 client.0 _ll_register_callbacks cb 0x560e1b80b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34898/0 (socket says 172.21.15.26:34898) 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 -- 172.21.15.26:0/415527914 learned_addr learned my addr 172.21.15.26:0/415527914 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c179a700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1baca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c279c700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1baca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 -- 172.21.15.26:0/415527914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 msgr2=0x560e1baca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1baca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 -- 172.21.15.26:0/415527914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 msgr2=0x560e1baca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1baca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.941+0000 7f16c1f9b700 1 -- 172.21.15.26:0/415527914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a61b0 con 0x560e1babe2a0 2023-10-06T21:29:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f16b800b960 tx=0x7f16b800bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4b84f16cdc30c56 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8016070 con 0x560e1babe2a0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f16b8003d00 con 0x560e1babe2a0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e1bab2160 con 0x560e1babe2a0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b8011440 con 0x560e1babe2a0 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16d7b5bb80 10 client.27698 fetch_fsmap learned FSMap version 67 2023-10-06T21:29:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.942+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x560e1b825c60 con 0x560e1babe2a0 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b8027080 con 0x560e1babe2a0 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f16b801b540 con 0x560e1babe2a0 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 10 client.27698 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 10 client.27698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 20 client.27698 populate_metadata read hostname 'smithi026' 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bad91e0 0x560e1badb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 4 client.27698 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e1ba0ae10 con 0x560e1bad91e0 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16c179a700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bad91e0 0x560e1badb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16c179a700 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bad91e0 0x560e1badb600 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f192f385071b3d60 server_cookie=e0a327546c4d12d6 in_seq=0 out_seq=0 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.943+0000 7f16a57fa700 10 client.27698 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.944+0000 7f16a57fa700 1 -- 172.21.15.26:0/415527914 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e1ba0ae10 con 0x560e1bad91e0 2023-10-06T21:29:25.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.944+0000 7f16a57fa700 10 client.27698 handle_command_reply: tid=0 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.944+0000 7f16d7b5bb80 1 client.27698 shutdown 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.944+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 msgr2=0x560e1bac9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.944+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f16b800b960 tx=0x7f16b800bc70 comp rx=0 tx=0).stop 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 shutdown_connections 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1baca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bad91e0 0x560e1badb600 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bac9e50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/415527914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1baca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 >> 172.21.15.26:0/415527914 conn(0x560e1ba33260 msgr2=0x560e1baba220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16a57fa700 0 client.27698 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 shutdown_connections 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/415527914 wait complete. 2023-10-06T21:29:25.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.945+0000 7f16d7b5bb80 20 client.27698 trim_cache size 0 max 16384 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.962+0000 7f16d7b5bb80 1 Processor -- start 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.962+0000 7f16d7b5bb80 1 -- start start 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bad45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bac8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1b892920 con 0x560e1bac9a70 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bac9170 con 0x560e1babe2a0 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bac7110 con 0x560e1babebc0 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34906/0 (socket says 172.21.15.26:34906) 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1156531958 learned_addr learned my addr 172.21.15.26:0/1156531958 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c179a700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bad45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c279c700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bac8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1156531958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 msgr2=0x560e1bad45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bad45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1156531958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 msgr2=0x560e1bac8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bac8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.963+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1156531958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e1b863250 con 0x560e1babe2a0 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16c179a700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bad45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b870 tx=0x7f16b800b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c975b66f786e239 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16b0ff9700 1 -- 172.21.15.26:0/1156531958 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b802a070 con 0x560e1babe2a0 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16b0ff9700 1 -- 172.21.15.26:0/1156531958 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e1bab2160 con 0x560e1babe2a0 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16b0ff9700 1 -- 172.21.15.26:0/1156531958 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8016040 con 0x560e1babe2a0 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1156531958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 msgr2=0x560e1baca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.964+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b870 tx=0x7f16b800b5a0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1156531958 shutdown_connections 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bac8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1baca560 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1156531958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bad45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1156531958 >> 172.21.15.26:0/1156531958 conn(0x560e1ba33260 msgr2=0x560e1bac9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1156531958 shutdown_connections 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1156531958 wait complete. 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 -- start start 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.965+0000 7f16d7b5bb80 1 Processor -- start 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bacd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bb26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bb26550 con 0x560e1bac9a70 2023-10-06T21:29:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bb266c0 con 0x560e1babe2a0 2023-10-06T21:29:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bb26880 con 0x560e1babebc0 2023-10-06T21:29:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16d7b5bb80 10 client.0 _ll_register_callbacks cb 0x7f16b8004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16c1f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34920/0 (socket says 172.21.15.26:34920) 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.966+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1242771853 learned_addr learned my addr 172.21.15.26:0/1242771853 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c279c700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bb26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c179a700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bacd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1242771853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 msgr2=0x560e1bacd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bacd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1242771853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 msgr2=0x560e1bb26010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bb26010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 -- 172.21.15.26:0/1242771853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e1b825c60 con 0x560e1babe2a0 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16c1f9b700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b8f0 tx=0x7f16b801b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8d761bce55ce3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8016030 con 0x560e1babe2a0 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f16b8000e40 con 0x560e1babe2a0 2023-10-06T21:29:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.967+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16ac09b1f0 con 0x560e1babe2a0 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b802a040 con 0x560e1babe2a0 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 10 client.27704 fetch_fsmap learned FSMap version 67 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x560e1b83b2b0 con 0x560e1babe2a0 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b803b020 con 0x560e1babe2a0 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f16b8037500 con 0x560e1babe2a0 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 10 client.27704 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 20 client.27704 populate_metadata read hostname 'smithi026' 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bb2dc50 0x560e1bb30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 4 client.27704 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.968+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e1ba0ab30 con 0x560e1bb2dc50 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16c179a700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bb2dc50 0x560e1bb30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16c179a700 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bb2dc50 0x560e1bb30040 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbbdb5b0d8f0638 server_cookie=2373b2faf408d538 in_seq=0 out_seq=0 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16a57fa700 10 client.27704 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16a57fa700 1 -- 172.21.15.26:0/1242771853 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560e1ba0ab30 con 0x560e1bb2dc50 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16a57fa700 10 client.27704 handle_command_reply: tid=0 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16d7b5bb80 1 client.27704 shutdown 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 msgr2=0x560e1bacd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b8f0 tx=0x7f16b801b4d0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.969+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 shutdown_connections 2023-10-06T21:29:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e1bac9a70 0x560e1bb26010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x560e1bb2dc50 0x560e1bb30040 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e1babe2a0 0x560e1bacd060 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 --2- 172.21.15.26:0/1242771853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e1babebc0 0x560e1bacd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 >> 172.21.15.26:0/1242771853 conn(0x560e1ba33260 msgr2=0x560e1baca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16a57fa700 0 client.27704 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 shutdown_connections 2023-10-06T21:29:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 1 -- 172.21.15.26:0/1242771853 wait complete. 2023-10-06T21:29:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16d7b5bb80 20 client.27704 trim_cache size 0 max 16384 2023-10-06T21:29:25.989 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:29:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:29:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:29:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:29:25.991 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:29:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 msgr2=0x7f16a80a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 0x7f16a80a3b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f16b40075c0 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 msgr2=0x7f16bc199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.970+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac005740 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 shutdown_connections 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16bc19dad0 0x7f16bc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16a80a1670 0x7f16a80a3b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16bc100e60 0x7f16bc198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 --2- 172.21.15.26:0/1599792941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc105f80 0x7f16bc199460 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 >> 172.21.15.26:0/1599792941 conn(0x7f16bc0fc6c0 msgr2=0x7f16bc0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 shutdown_connections 2023-10-06T21:29:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:25.971+0000 7f16c2f9d700 1 -- 172.21.15.26:0/1599792941 wait complete. 2023-10-06T21:29:25.994 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:29:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:26 smithi167 ceph-mon[134494]: pgmap v603: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:26 smithi026 ceph-mon[98490]: pgmap v603: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:26 smithi078 ceph-mon[134896]: pgmap v603: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:28 smithi167 ceph-mon[134494]: pgmap v604: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:28 smithi026 ceph-mon[98490]: pgmap v604: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:28 smithi078 ceph-mon[134896]: pgmap v604: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:29:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:30 smithi167 ceph-mon[134494]: pgmap v605: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:29:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:30 smithi026 ceph-mon[98490]: pgmap v605: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:29:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:30 smithi078 ceph-mon[134896]: pgmap v605: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:29:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:32 smithi167 ceph-mon[134494]: pgmap v606: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:29:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:32 smithi026 ceph-mon[98490]: pgmap v606: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:29:33.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:32 smithi078 ceph-mon[134896]: pgmap v606: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:29:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:34 smithi167 ceph-mon[134494]: pgmap v607: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:34 smithi026 ceph-mon[98490]: pgmap v607: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:34 smithi078 ceph-mon[134896]: pgmap v607: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:37.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:36 smithi167 ceph-mon[134494]: pgmap v608: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:36 smithi026 ceph-mon[98490]: pgmap v608: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:37.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:36 smithi078 ceph-mon[134896]: pgmap v608: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:39.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:38 smithi167 ceph-mon[134494]: pgmap v609: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:38 smithi026 ceph-mon[98490]: pgmap v609: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:38 smithi078 ceph-mon[134896]: pgmap v609: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:40 smithi167 ceph-mon[134494]: pgmap v610: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:40 smithi026 ceph-mon[98490]: pgmap v610: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:40 smithi078 ceph-mon[134896]: pgmap v610: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:43.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:42 smithi167 ceph-mon[134494]: pgmap v611: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:42 smithi026 ceph-mon[98490]: pgmap v611: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:42 smithi078 ceph-mon[134896]: pgmap v611: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:44.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:44 smithi167 ceph-mon[134494]: pgmap v612: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:44 smithi026 ceph-mon[98490]: pgmap v612: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:44 smithi078 ceph-mon[134896]: pgmap v612: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:46 smithi167 ceph-mon[134494]: pgmap v613: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:46 smithi026 ceph-mon[98490]: pgmap v613: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:46 smithi078 ceph-mon[134896]: pgmap v613: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:48 smithi167 ceph-mon[134494]: pgmap v614: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:48 smithi026 ceph-mon[98490]: pgmap v614: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:48 smithi078 ceph-mon[134896]: pgmap v614: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:50 smithi167 ceph-mon[134494]: pgmap v615: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:50 smithi026 ceph-mon[98490]: pgmap v615: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:50 smithi078 ceph-mon[134896]: pgmap v615: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:52 smithi167 ceph-mon[134494]: pgmap v616: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:52 smithi026 ceph-mon[98490]: pgmap v616: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:52 smithi078 ceph-mon[134896]: pgmap v616: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:54 smithi167 ceph-mon[134494]: pgmap v617: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:54 smithi026 ceph-mon[98490]: pgmap v617: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:54 smithi078 ceph-mon[134896]: pgmap v617: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:29:55.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.249+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3324622104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 msgr2=0x7f9ff010ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.249+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3324622104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff010ffa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800b0a0 tx=0x7f9fe800b3b0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3324622104 shutdown_connections 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3324622104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff010ffa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3324622104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff0103300 0x7f9ff01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3324622104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103cb0 0x7f9ff0107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3324622104 >> 172.21.15.26:0/3324622104 conn(0x7f9ff00feb40 msgr2=0x7f9ff0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3324622104 shutdown_connections 2023-10-06T21:29:56.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.250+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3324622104 wait complete. 2023-10-06T21:29:56.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.251+0000 7f9ff4bff700 1 Processor -- start 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.251+0000 7f9ff4bff700 1 -- start start 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff019dc20 0x7f9ff019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff019e6a0 con 0x7f9ff0108420 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff019e810 con 0x7f9ff0103300 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9ff4bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff019e9d0 con 0x7f9ff019dc20 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.252+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fedd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34164/0 (socket says 172.21.15.26:34164) 2023-10-06T21:29:56.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 -- 172.21.15.26:0/3202273410 learned_addr learned my addr 172.21.15.26:0/3202273410 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff019dc20 0x7f9ff019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 -- 172.21.15.26:0/3202273410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff019dc20 msgr2=0x7f9ff019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff019dc20 0x7f9ff019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 -- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 msgr2=0x7f9ff01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 -- 172.21.15.26:0/3202273410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8009d00 con 0x7f9ff0103300 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.253+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d950 tx=0x7f9fe000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.254+0000 7f9fdf7fe700 1 -- 172.21.15.26:0/3202273410 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000f890 con 0x7f9ff0103300 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.254+0000 7f9fdf7fe700 1 -- 172.21.15.26:0/3202273410 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe0010460 con 0x7f9ff0103300 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.254+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff01a5f30 con 0x7f9ff0103300 2023-10-06T21:29:56.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.254+0000 7f9fdf7fe700 1 -- 172.21.15.26:0/3202273410 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000e610 con 0x7f9ff0103300 2023-10-06T21:29:56.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.254+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff01a63b0 con 0x7f9ff0103300 2023-10-06T21:29:56.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.257+0000 7f9fdf7fe700 1 -- 172.21.15.26:0/3202273410 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f9fe00108d0 con 0x7f9ff0103300 2023-10-06T21:29:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.259+0000 7fa008601b80 1 Processor -- start 2023-10-06T21:29:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.259+0000 7f9fdf7fe700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 0x7f9fd80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7f9fdf7fe700 1 -- 172.21.15.26:0/3202273410 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fe00ccf00 con 0x7f9ff0103300 2023-10-06T21:29:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 -- start start 2023-10-06T21:29:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 0x7f9fd80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df20c0 0x55abe2df24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2dfd890 0x55abe2dffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d5b460 con 0x55abe2dfd890 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d4cc20 con 0x55abe2df29e0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7fa008601b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d4a540 con 0x55abe2df20c0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7f9fedd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7f9fedd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34176/0 (socket says 172.21.15.26:34176) 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.260+0000 7f9fedd9b700 1 -- 172.21.15.26:0/1464548363 learned_addr learned my addr 172.21.15.26:0/1464548363 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fee59c700 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df20c0 0x55abe2df24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9feed9d700 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2dfd890 0x55abe2dffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 0x7f9fd80a3ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fd0 tx=0x7f9fe800b7f0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 -- 172.21.15.26:0/1464548363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df20c0 msgr2=0x55abe2df24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df20c0 0x55abe2df24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 -- 172.21.15.26:0/1464548363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2dfd890 msgr2=0x55abe2dffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2dfd890 0x55abe2dffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 -- 172.21.15.26:0/1464548363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abe2b70de0 con 0x55abe2df29e0 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40103d0 tx=0x7f9fe40106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a78b9b3ce20a1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.261+0000 7f9fdcff9700 1 -- 172.21.15.26:0/1464548363 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4011440 con 0x55abe2df29e0 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7f9fdcff9700 1 -- 172.21.15.26:0/1464548363 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe4015070 con 0x55abe2df29e0 2023-10-06T21:29:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7f9fdcff9700 1 -- 172.21.15.26:0/1464548363 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40198b0 con 0x55abe2df29e0 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 -- 172.21.15.26:0/1464548363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 msgr2=0x55abe2dfd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40103d0 tx=0x7f9fe40106e0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 -- 172.21.15.26:0/1464548363 shutdown_connections 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2dfd890 0x55abe2dffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df20c0 0x55abe2df24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 --2- 172.21.15.26:0/1464548363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfd350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.262+0000 7fa008601b80 1 -- 172.21.15.26:0/1464548363 >> 172.21.15.26:0/1464548363 conn(0x55abe2d67260 msgr2=0x55abe2dee450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.263+0000 7fa008601b80 1 -- 172.21.15.26:0/1464548363 shutdown_connections 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.263+0000 7fa008601b80 1 -- 172.21.15.26:0/1464548363 wait complete. 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.263+0000 7fa008601b80 1 -- start start 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.263+0000 7fa008601b80 1 Processor -- start 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2dfa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2dfd890 0x55abe2dfb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d45e30 con 0x55abe2df20c0 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2ce2710 con 0x55abe2df29e0 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2ce23b0 con 0x55abe2dfd890 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7fa008601b80 10 client.0 _ll_register_callbacks cb 0x55abe2b3f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7f9fedd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2dfa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7f9fedd9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34178/0 (socket says 172.21.15.26:34178) 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7f9fedd9b700 1 -- 172.21.15.26:0/2970324091 learned_addr learned my addr 172.21.15.26:0/2970324091 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.264+0000 7f9feed9d700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2dfd890 0x55abe2dfb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 -- 172.21.15.26:0/2970324091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2dfd890 msgr2=0x55abe2dfb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2dfd890 0x55abe2dfb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 -- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 msgr2=0x55abe2dfab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 -- 172.21.15.26:0/2970324091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4007c10 con 0x55abe2df20c0 2023-10-06T21:29:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fee59c700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2dfa620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00ca5e0 tx=0x7f9fe009a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f524375c64d95f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:29:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe009a2d0 con 0x55abe2df20c0 2023-10-06T21:29:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abe2de5f10 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.265+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fe0099360 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe00b8170 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 10 client.17334 fetch_fsmap learned FSMap version 67 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55abe2b59c60 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe0063080 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fe0098dd0 con 0x55abe2df20c0 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 10 client.17334 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 20 client.17334 populate_metadata read hostname 'smithi026' 2023-10-06T21:29:56.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e0d3b0 0x55abe2e0f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 4 client.17334 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abe2d3ee10 con 0x55abe2e0d3b0 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.266+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e0d3b0 0x55abe2e0f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.267+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e0d3b0 0x55abe2e0f7d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617aba597ad7569b server_cookie=fad2442419aa3378 in_seq=0 out_seq=0 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.267+0000 7f9fd5ffb700 10 client.17334 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/2970324091 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abe2d3ee10 con 0x55abe2e0d3b0 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7f9fd5ffb700 10 client.17334 handle_command_reply: tid=0 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 client.17334 shutdown 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 msgr2=0x55abe2dfa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2dfa620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00ca5e0 tx=0x7f9fe009a6d0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 shutdown_connections 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2dfd890 0x55abe2dfb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2dfa620 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e0d3b0 0x55abe2e0f7d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 --2- 172.21.15.26:0/2970324091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2df29e0 0x55abe2dfab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 >> 172.21.15.26:0/2970324091 conn(0x55abe2d67260 msgr2=0x55abe2dedfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7f9fd5ffb700 0 client.17334 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 shutdown_connections 2023-10-06T21:29:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.268+0000 7fa008601b80 1 -- 172.21.15.26:0/2970324091 wait complete. 2023-10-06T21:29:56.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.269+0000 7fa008601b80 20 client.17334 trim_cache size 0 max 16384 2023-10-06T21:29:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 Processor -- start 2023-10-06T21:29:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 -- start start 2023-10-06T21:29:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2df5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2df9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2bc6920 con 0x55abe2df20c0 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2df2720 con 0x55abe2dfd890 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7fa008601b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e09240 con 0x55abe2df29e0 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7f9feed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7f9feed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34190/0 (socket says 172.21.15.26:34190) 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7f9feed9d700 1 -- 172.21.15.26:0/3204186592 learned_addr learned my addr 172.21.15.26:0/3204186592 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2df5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.289+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2df9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 -- 172.21.15.26:0/3204186592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 msgr2=0x55abe2df9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2df9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 -- 172.21.15.26:0/3204186592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 msgr2=0x55abe2df5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2df5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 -- 172.21.15.26:0/3204186592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abe2b97250 con 0x55abe2dfd890 2023-10-06T21:29:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8003d00 tx=0x7f9fe801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80a7fc6d26f329e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9fdd7fa700 1 -- 172.21.15.26:0/3204186592 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800e040 con 0x55abe2dfd890 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9fdd7fa700 1 -- 172.21.15.26:0/3204186592 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe8003e70 con 0x55abe2dfd890 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.290+0000 7f9fdd7fa700 1 -- 172.21.15.26:0/3204186592 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe801c990 con 0x55abe2dfd890 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- 172.21.15.26:0/3204186592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 msgr2=0x55abe2df5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8003d00 tx=0x7f9fe801c000 comp rx=0 tx=0).stop 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- 172.21.15.26:0/3204186592 shutdown_connections 2023-10-06T21:29:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2df5990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2df5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 --2- 172.21.15.26:0/3204186592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2df9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- 172.21.15.26:0/3204186592 >> 172.21.15.26:0/3204186592 conn(0x55abe2d67260 msgr2=0x55abe2e08b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- 172.21.15.26:0/3204186592 shutdown_connections 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- 172.21.15.26:0/3204186592 wait complete. 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.291+0000 7fa008601b80 1 -- start start 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.292+0000 7fa008601b80 1 Processor -- start 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.292+0000 7fa008601b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2e040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.292+0000 7fa008601b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2e02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7fa008601b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7fa008601b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e02e80 con 0x55abe2df20c0 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7fa008601b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e02ff0 con 0x55abe2dfd890 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7fa008601b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e031b0 con 0x55abe2df29e0 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7fa008601b80 10 client.0 _ll_register_callbacks cb 0x7f9fe009a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:29:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2e040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:34194/0 (socket says 172.21.15.26:34194) 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 -- 172.21.15.26:0/3385185790 learned_addr learned my addr 172.21.15.26:0/3385185790 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9fedd9b700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2e02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 -- 172.21.15.26:0/3385185790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 msgr2=0x55abe2e02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2e02400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 -- 172.21.15.26:0/3385185790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 msgr2=0x55abe2e040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2e040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9feed9d700 1 -- 172.21.15.26:0/3385185790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff0110810 con 0x55abe2dfd890 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.293+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2e040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9feed9d700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800b3f0 tx=0x7f9fe801cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50e99e52efc6849 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9fd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800e070 con 0x55abe2dfd890 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fe4015070 con 0x55abe2dfd890 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fe8023040 con 0x55abe2dfd890 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe80054d0 con 0x55abe2dfd890 2023-10-06T21:29:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7fa008601b80 10 client.27722 fetch_fsmap learned FSMap version 67 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55abe2b6f2b0 con 0x55abe2dfd890 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.294+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe4015070 con 0x55abe2dfd890 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fe8033990 con 0x55abe2dfd890 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 10 client.27722 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 10 client.27722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 20 client.27722 populate_metadata read hostname 'smithi026' 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e620c0 0x55abe2e644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 4 client.27722 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:29:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55abe2d3eb30 con 0x55abe2e620c0 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e620c0 0x55abe2e644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7f9fd5ffb700 10 client.27722 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.295+0000 7f9fee59c700 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e620c0 0x55abe2e644b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d948cb8c8f372b7b server_cookie=e72bc399b7c1d621 in_seq=0 out_seq=0 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7f9fd5ffb700 1 -- 172.21.15.26:0/3385185790 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55abe2d3eb30 con 0x55abe2e620c0 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7f9fd5ffb700 10 client.27722 handle_command_reply: tid=0 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 client.27722 shutdown 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 msgr2=0x55abe2e02940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800b3f0 tx=0x7f9fe801cd00 comp rx=0 tx=0).stop 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 shutdown_connections 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55abe2e620c0 0x55abe2e644b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55abe2dfd890 0x55abe2e02940 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55abe2df20c0 0x55abe2e040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7f9fd5ffb700 0 client.27722 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 --2- 172.21.15.26:0/3385185790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55abe2df29e0 0x55abe2e02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 >> 172.21.15.26:0/3385185790 conn(0x55abe2d67260 msgr2=0x55abe2dfbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 shutdown_connections 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 1 -- 172.21.15.26:0/3385185790 wait complete. 2023-10-06T21:29:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.296+0000 7fa008601b80 20 client.27722 trim_cache size 0 max 16384 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:29:56.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:29:56.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:29:56.310 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:29:56.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 msgr2=0x7f9fd80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 0x7f9fd80a3ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fd0 tx=0x7f9fe800b7f0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 msgr2=0x7f9ff0199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d950 tx=0x7f9fe000dc60 comp rx=0 tx=0).stop 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 shutdown_connections 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ff019dc20 0x7f9ff019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ff0103300 0x7f9ff0199280 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f9fd80a15f0 0x7f9fd80a3ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 --2- 172.21.15.26:0/3202273410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff0108420 0x7f9ff01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:29:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 >> 172.21.15.26:0/3202273410 conn(0x7f9ff00feb40 msgr2=0x7f9ff00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:29:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 shutdown_connections 2023-10-06T21:29:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:29:56.297+0000 7f9ff4bff700 1 -- 172.21.15.26:0/3202273410 wait complete. 2023-10-06T21:29:56.321 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:29:57.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:56 smithi167 ceph-mon[134494]: pgmap v618: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:57.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:56 smithi026 ceph-mon[98490]: pgmap v618: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:56 smithi078 ceph-mon[134896]: pgmap v618: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:29:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:29:58 smithi167 ceph-mon[134494]: pgmap v619: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:29:58 smithi026 ceph-mon[98490]: pgmap v619: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:29:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:29:58 smithi078 ceph-mon[134896]: pgmap v619: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:00 smithi167 ceph-mon[134494]: pgmap v620: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:00 smithi167 ceph-mon[134494]: overall HEALTH_OK 2023-10-06T21:30:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:00 smithi026 ceph-mon[98490]: pgmap v620: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:00 smithi026 ceph-mon[98490]: overall HEALTH_OK 2023-10-06T21:30:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:00 smithi078 ceph-mon[134896]: pgmap v620: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:00 smithi078 ceph-mon[134896]: overall HEALTH_OK 2023-10-06T21:30:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:02 smithi167 ceph-mon[134494]: pgmap v621: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:30:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:02 smithi026 ceph-mon[98490]: pgmap v621: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:30:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:02 smithi078 ceph-mon[134896]: pgmap v621: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:30:04.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:03 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:04.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:03 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:04.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:03 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:30:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:04 smithi167 ceph-mon[134494]: pgmap v622: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:30:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:04 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:04 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:04 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:30:05.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:04 smithi026 ceph-mon[98490]: pgmap v622: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:30:05.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:04 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:05.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:04 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:05.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:04 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:30:05.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:04 smithi078 ceph-mon[134896]: pgmap v622: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:30:05.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:04 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:30:05.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:04 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:30:05.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:04 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:30:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:06 smithi167 ceph-mon[134494]: pgmap v623: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:07.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:06 smithi026 ceph-mon[98490]: pgmap v623: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:06 smithi078 ceph-mon[134896]: pgmap v623: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:08 smithi167 ceph-mon[134494]: pgmap v624: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:30:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:08 smithi026 ceph-mon[98490]: pgmap v624: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:30:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:08 smithi078 ceph-mon[134896]: pgmap v624: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-10-06T21:30:11.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:10 smithi167 ceph-mon[134494]: pgmap v625: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:11.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:10 smithi026 ceph-mon[98490]: pgmap v625: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:11.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:10 smithi078 ceph-mon[134896]: pgmap v625: 97 pgs: 97 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-10-06T21:30:12.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:11 smithi167 ceph-mon[134494]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:30:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:11 smithi026 ceph-mon[98490]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:30:12.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:11 smithi078 ceph-mon[134896]: osdmap e116: 12 total, 12 up, 12 in 2023-10-06T21:30:13.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:12 smithi167 ceph-mon[134494]: pgmap v627: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:30:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:12 smithi026 ceph-mon[98490]: pgmap v627: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:30:13.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:12 smithi078 ceph-mon[134896]: pgmap v627: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:30:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:14 smithi167 ceph-mon[134494]: pgmap v628: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:30:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:14 smithi026 ceph-mon[98490]: pgmap v628: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:30:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:14 smithi078 ceph-mon[134896]: pgmap v628: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-06T21:30:17.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:16 smithi167 ceph-mon[134494]: pgmap v629: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:16 smithi026 ceph-mon[98490]: pgmap v629: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:16 smithi078 ceph-mon[134896]: pgmap v629: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:18 smithi167 ceph-mon[134494]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:18 smithi167 ceph-mon[134494]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:30:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:18 smithi026 ceph-mon[98490]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:18 smithi026 ceph-mon[98490]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:30:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:18 smithi078 ceph-mon[134896]: pgmap v630: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:18 smithi078 ceph-mon[134896]: osdmap e117: 12 total, 12 up, 12 in 2023-10-06T21:30:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:20 smithi167 ceph-mon[134494]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:30:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:20 smithi026 ceph-mon[98490]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:30:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:20 smithi078 ceph-mon[134896]: pgmap v632: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:30:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:22 smithi167 ceph-mon[134494]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:22 smithi026 ceph-mon[98490]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:22 smithi078 ceph-mon[134896]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:24.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:24 smithi167 ceph-mon[134494]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:24 smithi026 ceph-mon[98490]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:24 smithi078 ceph-mon[134896]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:30:26.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:30:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 -- 172.21.15.26:0/416363086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b601029d0 msgr2=0x7f6b6010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 --2- 172.21.15.26:0/416363086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b601029d0 0x7f6b6010aeb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009990 tx=0x7f6b54009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 -- 172.21.15.26:0/416363086 shutdown_connections 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 --2- 172.21.15.26:0/416363086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b6010b480 0x7f6b6010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 --2- 172.21.15.26:0/416363086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 0x7f6b60102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 --2- 172.21.15.26:0/416363086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b601029d0 0x7f6b6010aeb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 -- 172.21.15.26:0/416363086 >> 172.21.15.26:0/416363086 conn(0x7f6b600fb890 msgr2=0x7f6b600fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 -- 172.21.15.26:0/416363086 shutdown_connections 2023-10-06T21:30:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.540+0000 7f6b651ff700 1 -- 172.21.15.26:0/416363086 wait complete. 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.541+0000 7f6b651ff700 1 Processor -- start 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.541+0000 7f6b651ff700 1 -- start start 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 0x7f6b60196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b601029d0 0x7f6b60197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019bfc0 con 0x7f6b601020b0 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019c130 con 0x7f6b6010b480 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b651ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019c2f0 con 0x7f6b601029d0 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 0x7f6b60196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b601029d0 0x7f6b60197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35762/0 (socket says 172.21.15.26:35762) 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 -- 172.21.15.26:0/4276806049 learned_addr learned my addr 172.21.15.26:0/4276806049 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 -- 172.21.15.26:0/4276806049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b601029d0 msgr2=0x7f6b60197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b601029d0 0x7f6b60197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 -- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 msgr2=0x7f6b60196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 0x7f6b60196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.542+0000 7f6b5f59e700 1 -- 172.21.15.26:0/4276806049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b540095f0 con 0x7f6b6010b480 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ef30 tx=0x7f6b5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b47fff700 1 -- 172.21.15.26:0/4276806049 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58018070 con 0x7f6b6010b480 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b47fff700 1 -- 172.21.15.26:0/4276806049 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5800f040 con 0x7f6b6010b480 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6019c570 con 0x7f6b6010b480 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b47fff700 1 -- 172.21.15.26:0/4276806049 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800cd00 con 0x7f6b6010b480 2023-10-06T21:30:26.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.543+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b601a3e10 con 0x7f6b6010b480 2023-10-06T21:30:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.547+0000 7f6b47fff700 1 -- 172.21.15.26:0/4276806049 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6b58003a00 con 0x7f6b6010b480 2023-10-06T21:30:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.549+0000 7f6b78a61b80 1 Processor -- start 2023-10-06T21:30:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.549+0000 7f6b47fff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 0x7f6b480a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.549+0000 7f6b78a61b80 1 -- start start 2023-10-06T21:30:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.549+0000 7f6b47fff700 1 -- 172.21.15.26:0/4276806049 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b580ccc40 con 0x7f6b6010b480 2023-10-06T21:30:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.549+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 0x7f6b480a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae9e0 0x5560c8bb9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8b17460 con 0x5560c8bae9e0 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8b08c20 con 0x5560c8bb9890 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8b06540 con 0x5560c8bae0c0 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:57436/0 (socket says 172.21.15.26:57436) 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/4133934656 learned_addr learned my addr 172.21.15.26:0/4133934656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.550+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae9e0 0x5560c8bb9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/4133934656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 msgr2=0x5560c8bbbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 0x7f6b480a3ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b601985f0 tx=0x7f6b5400b850 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/4133934656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae9e0 msgr2=0x5560c8bb9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae9e0 0x5560c8bb9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/4133934656 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560c892cde0 con 0x5560c8bae0c0 2023-10-06T21:30:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009d80 tx=0x7f6b50011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f36f768e67f037e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b457fa700 1 -- 172.21.15.26:0/4133934656 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50004230 con 0x5560c8bae0c0 2023-10-06T21:30:26.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b457fa700 1 -- 172.21.15.26:0/4133934656 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b50004390 con 0x5560c8bae0c0 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.551+0000 7f6b457fa700 1 -- 172.21.15.26:0/4133934656 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50019880 con 0x5560c8bae0c0 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 -- 172.21.15.26:0/4133934656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 msgr2=0x5560c8bae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009d80 tx=0x7f6b50011720 comp rx=0 tx=0).stop 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 -- 172.21.15.26:0/4133934656 shutdown_connections 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae0c0 0x5560c8bae4a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/4133934656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae9e0 0x5560c8bb9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 -- 172.21.15.26:0/4133934656 >> 172.21.15.26:0/4133934656 conn(0x5560c8b23260 msgr2=0x5560c8baa450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 -- 172.21.15.26:0/4133934656 shutdown_connections 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.552+0000 7f6b78a61b80 1 -- 172.21.15.26:0/4133934656 wait complete. 2023-10-06T21:30:26.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.553+0000 7f6b78a61b80 1 -- start start 2023-10-06T21:30:26.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.553+0000 7f6b78a61b80 1 Processor -- start 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae9e0 0x5560c8bc36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8b01e30 con 0x5560c8bae0c0 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8a9e710 con 0x5560c8bb9890 2023-10-06T21:30:26.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8a9e3b0 con 0x5560c8bae9e0 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b78a61b80 10 client.0 _ll_register_callbacks cb 0x5560c88fb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49296/0 (socket says 172.21.15.26:49296) 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/1316042376 learned_addr learned my addr 172.21.15.26:0/1316042376 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.554+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae9e0 0x5560c8bc36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/1316042376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae9e0 msgr2=0x5560c8bc36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae9e0 0x5560c8bc36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 msgr2=0x5560c8bbd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 -- 172.21.15.26:0/1316042376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b500099e0 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b50000c00 tx=0x7f6b50005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f4f78e3f7de13a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.555+0000 7f6b42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50005ce0 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560c8ba1f10 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b50003680 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b50003840 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b78a61b80 10 client.17352 fetch_fsmap learned FSMap version 67 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5560c8915c60 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560c8ba1f10 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.556+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6b50021040 con 0x5560c8bae0c0 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 10 client.17352 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:30:26.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 20 client.17352 populate_metadata read hostname 'smithi026' 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8bcd550 0x5560c8bcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 4 client.17352 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560c8afae10 con 0x5560c8bcd550 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8bcd550 0x5560c8bcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8bcd550 0x5560c8bcf970 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2f052c893f4fb0 server_cookie=80dd3137f021f044 in_seq=0 out_seq=0 2023-10-06T21:30:26.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.557+0000 7f6b42ff5700 10 client.17352 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:26.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b42ff5700 1 -- 172.21.15.26:0/1316042376 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5560c8afae10 con 0x5560c8bcd550 2023-10-06T21:30:26.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b42ff5700 10 client.17352 handle_command_reply: tid=0 2023-10-06T21:30:26.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 client.17352 shutdown 2023-10-06T21:30:26.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 msgr2=0x5560c8bc3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6b50000c00 tx=0x7f6b50005270 comp rx=0 tx=0).stop 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 shutdown_connections 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8bcd550 0x5560c8bcf970 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bb9890 0x5560c8bbd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b42ff5700 0 client.17352 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3190 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/1316042376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bae9e0 0x5560c8bc36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.558+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 >> 172.21.15.26:0/1316042376 conn(0x5560c8b23260 msgr2=0x5560c8baa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.559+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 shutdown_connections 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.559+0000 7f6b78a61b80 1 -- 172.21.15.26:0/1316042376 wait complete. 2023-10-06T21:30:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.559+0000 7f6b78a61b80 20 client.17352 trim_cache size 0 max 16384 2023-10-06T21:30:26.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 Processor -- start 2023-10-06T21:30:26.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 -- start start 2023-10-06T21:30:26.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bbc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bb6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8982920 con 0x5560c8bae0c0 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8baedc0 con 0x5560c8bae9e0 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8babd40 con 0x5560c8bb9890 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35790/0 (socket says 172.21.15.26:35790) 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b5e59c700 1 -- 172.21.15.26:0/65443038 learned_addr learned my addr 172.21.15.26:0/65443038 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bbc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.581+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bb6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 -- 172.21.15.26:0/65443038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 msgr2=0x5560c8bb6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bb6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 -- 172.21.15.26:0/65443038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 msgr2=0x5560c8bbc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bbc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 -- 172.21.15.26:0/65443038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560c8ba6140 con 0x5560c8bae9e0 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bb6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bbc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6b540098e0 tx=0x7f6b54022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be48b1d6341a379 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b45ffb700 1 -- 172.21.15.26:0/65443038 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54036b30 con 0x5560c8bae9e0 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b45ffb700 1 -- 172.21.15.26:0/65443038 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5401e040 con 0x5560c8bae9e0 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.582+0000 7f6b45ffb700 1 -- 172.21.15.26:0/65443038 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004c60 con 0x5560c8bae9e0 2023-10-06T21:30:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 -- 172.21.15.26:0/65443038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 msgr2=0x5560c8bb5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6b540098e0 tx=0x7f6b54022000 comp rx=0 tx=0).stop 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 -- 172.21.15.26:0/65443038 shutdown_connections 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bb6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bbc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/65443038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bb5bc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 -- 172.21.15.26:0/65443038 >> 172.21.15.26:0/65443038 conn(0x5560c8b23260 msgr2=0x5560c8bb9010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 -- 172.21.15.26:0/65443038 shutdown_connections 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.583+0000 7f6b78a61b80 1 -- 172.21.15.26:0/65443038 wait complete. 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.584+0000 7f6b78a61b80 1 -- start start 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.584+0000 7f6b78a61b80 1 Processor -- start 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.584+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.584+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.584+0000 7f6b78a61b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bc1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8bc1fc0 con 0x5560c8bae0c0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8bc2130 con 0x5560c8bae9e0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b78a61b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560c8bc22f0 con 0x5560c8bb9890 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b78a61b80 10 client.0 _ll_register_callbacks cb 0x7f6b54009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:35804/0 (socket says 172.21.15.26:35804) 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 -- 172.21.15.26:0/2653102750 learned_addr learned my addr 172.21.15.26:0/2653102750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5f59e700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bc1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 -- 172.21.15.26:0/2653102750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 msgr2=0x5560c8bc1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bc1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 -- 172.21.15.26:0/2653102750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 msgr2=0x5560c8bc3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.585+0000 7f6b5e59c700 1 -- 172.21.15.26:0/2653102750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560c8915c60 con 0x5560c8bae9e0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b5e59c700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400b650 tx=0x7f6b5400bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e609275d2de7193 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b42ff5700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004580 con 0x5560c8bae9e0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b5402c040 con 0x5560c8bae9e0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b50004390 con 0x5560c8bae9e0 2023-10-06T21:30:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b540046e0 con 0x5560c8bae9e0 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b78a61b80 10 client.27740 fetch_fsmap learned FSMap version 67 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7f6b6010e5e0 con 0x5560c8bae9e0 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.586+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b50004390 con 0x5560c8bae9e0 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6b5400f790 con 0x5560c8bae9e0 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 10 client.27740 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 10 client.27740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 20 client.27740 populate_metadata read hostname 'smithi026' 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8c223e0 0x5560c8c24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 4 client.27740 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560c8afab30 con 0x5560c8c223e0 2023-10-06T21:30:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8c223e0 0x5560c8c24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b5ed9d700 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8c223e0 0x5560c8c24800 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8404a16a89c63a server_cookie=4b1e2e2da85bf54d in_seq=0 out_seq=0 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.587+0000 7f6b42ff5700 10 client.27740 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b42ff5700 1 -- 172.21.15.26:0/2653102750 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5560c8afab30 con 0x5560c8c223e0 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b42ff5700 10 client.27740 handle_command_reply: tid=0 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 client.27740 shutdown 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 msgr2=0x5560c8bc1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400b650 tx=0x7f6b5400bee0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 shutdown_connections 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5560c8bb9890 0x5560c8bc1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5560c8bae0c0 0x5560c8bc3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5560c8c223e0 0x5560c8c24800 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 --2- 172.21.15.26:0/2653102750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560c8bae9e0 0x5560c8bc1540 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b42ff5700 0 client.27740 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 >> 172.21.15.26:0/2653102750 conn(0x5560c8b23260 msgr2=0x5560c8bb5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 shutdown_connections 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 1 -- 172.21.15.26:0/2653102750 wait complete. 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.588+0000 7f6b78a61b80 20 client.27740 trim_cache size 0 max 16384 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:30:26.596 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:30:26.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:30:26.597 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:30:26.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:30:26.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:30:26.597 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 msgr2=0x7f6b480a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 0x7f6b480a3ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b601985f0 tx=0x7f6b5400b850 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 msgr2=0x7f6b6019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ef30 tx=0x7f6b5800c3a0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 shutdown_connections 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b6010b480 0x7f6b6019b980 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6b480a15f0 0x7f6b480a3ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b601020b0 0x7f6b60196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 --2- 172.21.15.26:0/4276806049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b601029d0 0x7f6b60197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 >> 172.21.15.26:0/4276806049 conn(0x7f6b600fb890 msgr2=0x7f6b6010d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 shutdown_connections 2023-10-06T21:30:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:26.589+0000 7f6b651ff700 1 -- 172.21.15.26:0/4276806049 wait complete. 2023-10-06T21:30:26.611 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:30:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:26 smithi167 ceph-mon[134494]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:26 smithi026 ceph-mon[98490]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:26 smithi078 ceph-mon[134896]: pgmap v635: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:28 smithi167 ceph-mon[134494]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:28 smithi026 ceph-mon[98490]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:28 smithi078 ceph-mon[134896]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:30:31.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:30 smithi167 ceph-mon[134494]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:30:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:30 smithi026 ceph-mon[98490]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:30:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:30 smithi078 ceph-mon[134896]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:30:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:32 smithi167 ceph-mon[134494]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:30:33.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:32 smithi026 ceph-mon[98490]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:30:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:32 smithi078 ceph-mon[134896]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:30:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:34 smithi167 ceph-mon[134494]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:34 smithi026 ceph-mon[98490]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:34 smithi078 ceph-mon[134896]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:36 smithi167 ceph-mon[134494]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:36 smithi026 ceph-mon[98490]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:36 smithi078 ceph-mon[134896]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:38 smithi167 ceph-mon[134494]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:38 smithi026 ceph-mon[98490]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:38 smithi078 ceph-mon[134896]: pgmap v641: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:40 smithi167 ceph-mon[134494]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:40 smithi026 ceph-mon[98490]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:40 smithi078 ceph-mon[134896]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:43.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:42 smithi167 ceph-mon[134494]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:42 smithi026 ceph-mon[98490]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:42 smithi078 ceph-mon[134896]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:44.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:44 smithi167 ceph-mon[134494]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:44 smithi026 ceph-mon[98490]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:44 smithi078 ceph-mon[134896]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:46 smithi167 ceph-mon[134494]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:46 smithi026 ceph-mon[98490]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:47.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:46 smithi078 ceph-mon[134896]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:49.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:48 smithi167 ceph-mon[134494]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:48 smithi026 ceph-mon[98490]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:48 smithi078 ceph-mon[134896]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:50 smithi167 ceph-mon[134494]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:50 smithi026 ceph-mon[98490]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:50 smithi078 ceph-mon[134896]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:53.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:52 smithi167 ceph-mon[134494]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:52 smithi026 ceph-mon[98490]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:52 smithi078 ceph-mon[134896]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:54.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:54 smithi167 ceph-mon[134494]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:54 smithi026 ceph-mon[98490]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:54 smithi078 ceph-mon[134896]: pgmap v649: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:30:56.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.901+0000 7f66381ff700 1 -- 172.21.15.26:0/3756480166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66301005b0 msgr2=0x7f663010d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.901+0000 7f66381ff700 1 --2- 172.21.15.26:0/3756480166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66301005b0 0x7f663010d1e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6620009990 tx=0x7f6620009ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 -- 172.21.15.26:0/3756480166 shutdown_connections 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 --2- 172.21.15.26:0/3756480166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663010d7b0 0x7f663010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 --2- 172.21.15.26:0/3756480166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66300ffc00 0x7f66300fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 --2- 172.21.15.26:0/3756480166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66301005b0 0x7f663010d1e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 -- 172.21.15.26:0/3756480166 >> 172.21.15.26:0/3756480166 conn(0x7f66300fb830 msgr2=0x7f66300fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 -- 172.21.15.26:0/3756480166 shutdown_connections 2023-10-06T21:30:56.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.902+0000 7f66381ff700 1 -- 172.21.15.26:0/3756480166 wait complete. 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 Processor -- start 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 -- start start 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301005b0 0x7f66301996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663010d7b0 0x7f663019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e140 con 0x7f663010d7b0 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e2b0 con 0x7f66300ffc00 2023-10-06T21:30:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f66381ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e470 con 0x7f66301005b0 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.903+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41830/0 (socket says 172.21.15.26:41830) 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 -- 172.21.15.26:0/3470440883 learned_addr learned my addr 172.21.15.26:0/3470440883 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f663679c700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663010d7b0 0x7f663019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f663579a700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301005b0 0x7f66301996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 -- 172.21.15.26:0/3470440883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301005b0 msgr2=0x7f66301996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301005b0 0x7f66301996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 -- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663010d7b0 msgr2=0x7f663019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663010d7b0 0x7f663019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 -- 172.21.15.26:0/3470440883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200095f0 con 0x7f66300ffc00 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6635f9b700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9a0 tx=0x7f662c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.904+0000 7f6626ffd700 1 -- 172.21.15.26:0/3470440883 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c009790 con 0x7f66300ffc00 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.905+0000 7f6626ffd700 1 -- 172.21.15.26:0/3470440883 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662c00cbd0 con 0x7f66300ffc00 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.905+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663019e750 con 0x7f66300ffc00 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.905+0000 7f6626ffd700 1 -- 172.21.15.26:0/3470440883 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c018680 con 0x7f66300ffc00 2023-10-06T21:30:56.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.905+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301a5f90 con 0x7f66300ffc00 2023-10-06T21:30:56.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.908+0000 7f6626ffd700 1 -- 172.21.15.26:0/3470440883 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f662c00cd40 con 0x7f66300ffc00 2023-10-06T21:30:56.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.909+0000 7f6626ffd700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 0x7f661c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.910+0000 7f6626ffd700 1 -- 172.21.15.26:0/3470440883 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f662c014070 con 0x7f66300ffc00 2023-10-06T21:30:56.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.910+0000 7f663579a700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 0x7f661c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.910+0000 7f664bb07b80 1 Processor -- start 2023-10-06T21:30:56.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 -- start start 2023-10-06T21:30:56.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f663579a700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 0x7f661c0a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f663019a770 tx=0x7f662000b260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c26350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c28c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b84460 con 0x5605f4c1b9e0 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b75c20 con 0x5605f4c1b0c0 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f664bb07b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b73540 con 0x5605f4c26890 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41840/0 (socket says 172.21.15.26:41840) 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 -- 172.21.15.26:0/2646747672 learned_addr learned my addr 172.21.15.26:0/2646747672 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f663579a700 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c26350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f663679c700 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c28c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 -- 172.21.15.26:0/2646747672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 msgr2=0x5605f4c28c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c28c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 -- 172.21.15.26:0/2646747672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 msgr2=0x5605f4c26350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.911+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c26350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.912+0000 7f6635f9b700 1 -- 172.21.15.26:0/2646747672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f4999de0 con 0x5605f4c1b0c0 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.912+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a2120 tx=0x7f662c099ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800724c729e4c564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.912+0000 7f661bfff700 1 -- 172.21.15.26:0/2646747672 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c09e410 con 0x5605f4c1b0c0 2023-10-06T21:30:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.912+0000 7f661bfff700 1 -- 172.21.15.26:0/2646747672 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662c0b0020 con 0x5605f4c1b0c0 2023-10-06T21:30:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.912+0000 7f661bfff700 1 -- 172.21.15.26:0/2646747672 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0999d0 con 0x5605f4c1b0c0 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 -- 172.21.15.26:0/2646747672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 msgr2=0x5605f4c1b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f662c0a2120 tx=0x7f662c099ca0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 -- 172.21.15.26:0/2646747672 shutdown_connections 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c28c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1b4a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2646747672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c26350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 -- 172.21.15.26:0/2646747672 >> 172.21.15.26:0/2646747672 conn(0x5605f4b90260 msgr2=0x5605f4c17450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 -- 172.21.15.26:0/2646747672 shutdown_connections 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.913+0000 7f664bb07b80 1 -- 172.21.15.26:0/2646747672 wait complete. 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.914+0000 7f664bb07b80 1 -- start start 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.914+0000 7f664bb07b80 1 Processor -- start 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b6ee30 con 0x5605f4c1b0c0 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b0b710 con 0x5605f4c26890 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4b0b3b0 con 0x5605f4c1b9e0 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f664bb07b80 10 client.0 _ll_register_callbacks cb 0x5605f4968800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41856/0 (socket says 172.21.15.26:41856) 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 -- 172.21.15.26:0/2892706920 learned_addr learned my addr 172.21.15.26:0/2892706920 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663579a700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 -- 172.21.15.26:0/2892706920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 msgr2=0x5605f4c23ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c23ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 -- 172.21.15.26:0/2892706920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 msgr2=0x5605f4c23660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.915+0000 7f663679c700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c23660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f663679c700 1 -- 172.21.15.26:0/2892706920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301a5f90 con 0x5605f4c26890 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f663679c700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6628011cc0 tx=0x7f6628007d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698044f0a25cb37b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f66197fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628007c20 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605f4c0ef10 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f662800f8a0 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f662800fa60 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f664bb07b80 10 client.27752 fetch_fsmap learned FSMap version 67 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.916+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5605f4982c60 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605f4c0ef10 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6628015430 con 0x5605f4c26890 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 10 client.27752 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 10 client.27752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:56.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 20 client.27752 populate_metadata read hostname 'smithi026' 2023-10-06T21:30:56.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c364d0 0x5605f4c388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 4 client.27752 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:30:56.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605f4b67e10 con 0x5605f4c364d0 2023-10-06T21:30:56.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.917+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c364d0 0x5605f4c388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.918+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c364d0 0x5605f4c388f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899dec8d97b5221b server_cookie=c053fd7712fe7f2b in_seq=0 out_seq=0 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.918+0000 7f66197fa700 10 client.27752 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.918+0000 7f66197fa700 1 -- 172.21.15.26:0/2892706920 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5605f4b67e10 con 0x5605f4c364d0 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.918+0000 7f66197fa700 10 client.27752 handle_command_reply: tid=0 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.918+0000 7f664bb07b80 1 client.27752 shutdown 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 msgr2=0x5605f4c240e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6628011cc0 tx=0x7f6628007d80 comp rx=0 tx=0).stop 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 shutdown_connections 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c364d0 0x5605f4c388f0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c240e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2892706920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c23ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 >> 172.21.15.26:0/2892706920 conn(0x5605f4b90260 msgr2=0x5605f4c16fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f66197fa700 0 client.27752 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 shutdown_connections 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 1 -- 172.21.15.26:0/2892706920 wait complete. 2023-10-06T21:30:56.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.919+0000 7f664bb07b80 20 client.27752 trim_cache size 0 max 16384 2023-10-06T21:30:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 Processor -- start 2023-10-06T21:30:56.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 -- start start 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c2d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f49ef920 con 0x5605f4c1b0c0 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4c1f4d0 con 0x5605f4c26890 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f664bb07b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4c249b0 con 0x5605f4c1b9e0 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f663679c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f663679c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41870/0 (socket says 172.21.15.26:41870) 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f663679c700 1 -- 172.21.15.26:0/3846554599 learned_addr learned my addr 172.21.15.26:0/3846554599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50020/0 (socket says 172.21.15.26:50020) 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.943+0000 7f663579a700 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c2d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 -- 172.21.15.26:0/3846554599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 msgr2=0x5605f4c2d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c2d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 -- 172.21.15.26:0/3846554599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 msgr2=0x5605f4c1e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 -- 172.21.15.26:0/3846554599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f49c0250 con 0x5605f4c26890 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f663679c700 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6628012010 tx=0x7f6628006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff0f913cb1e00ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f6624ff9700 1 -- 172.21.15.26:0/3846554599 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280110d0 con 0x5605f4c26890 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f6624ff9700 1 -- 172.21.15.26:0/3846554599 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6628015a00 con 0x5605f4c26890 2023-10-06T21:30:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.944+0000 7f6624ff9700 1 -- 172.21.15.26:0/3846554599 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280050b0 con 0x5605f4c26890 2023-10-06T21:30:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 -- 172.21.15.26:0/3846554599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 msgr2=0x5605f4c208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6628012010 tx=0x7f6628006000 comp rx=0 tx=0).stop 2023-10-06T21:30:56.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 -- 172.21.15.26:0/3846554599 shutdown_connections 2023-10-06T21:30:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c26890 0x5605f4c208f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b0c0 0x5605f4c1e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 --2- 172.21.15.26:0/3846554599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c1b9e0 0x5605f4c2d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 -- 172.21.15.26:0/3846554599 >> 172.21.15.26:0/3846554599 conn(0x5605f4b90260 msgr2=0x5605f4c1e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 -- 172.21.15.26:0/3846554599 shutdown_connections 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.945+0000 7f664bb07b80 1 -- 172.21.15.26:0/3846554599 wait complete. 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.946+0000 7f664bb07b80 1 -- start start 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.946+0000 7f664bb07b80 1 Processor -- start 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.946+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.946+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c83dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f664bb07b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f664bb07b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4c7e820 con 0x5605f4c1b9e0 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f664bb07b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4c7e990 con 0x5605f4c1b0c0 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f664bb07b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f4c7eb50 con 0x5605f4c26890 2023-10-06T21:30:56.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f664bb07b80 10 client.0 _ll_register_callbacks cb 0x5605f4968800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:41876/0 (socket says 172.21.15.26:41876) 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 -- 172.21.15.26:0/2256829155 learned_addr learned my addr 172.21.15.26:0/2256829155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f663579a700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c83dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f663679c700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 -- 172.21.15.26:0/2256829155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 msgr2=0x5605f4c7e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c7e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 -- 172.21.15.26:0/2256829155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 msgr2=0x5605f4c83dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c83dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 -- 172.21.15.26:0/2256829155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605f4982c60 con 0x5605f4c1b0c0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f663679c700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c7e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f6635f9b700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f662c0967c0 tx=0x7f662c00efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44774dd95753d88d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.947+0000 7f66197fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c09e410 con 0x5605f4c1b0c0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f662c0b1070 con 0x5605f4c1b0c0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f662c0b0020 con 0x5605f4c1b0c0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f662c07e440 con 0x5605f4c1b0c0 2023-10-06T21:30:56.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 10 client.27758 fetch_fsmap learned FSMap version 67 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5605f49982b0 con 0x5605f4c1b0c0 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f662c0b0020 con 0x5605f4c1b0c0 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f662c0a22c0 con 0x5605f4c1b0c0 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 10 client.27758 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 10 client.27758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 20 client.27758 populate_metadata read hostname 'smithi026' 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c8b340 0x5605f4c8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 4 client.27758 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.948+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605f4b67b30 con 0x5605f4c8b340 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f663579a700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c8b340 0x5605f4c8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:30:56.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f66197fa700 10 client.27758 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f663579a700 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c8b340 0x5605f4c8d760 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99f0a5e0a49e4c3 server_cookie=7a782801e3365c72 in_seq=0 out_seq=0 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f66197fa700 1 -- 172.21.15.26:0/2256829155 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5605f4b67b30 con 0x5605f4c8b340 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f66197fa700 10 client.27758 handle_command_reply: tid=0 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f664bb07b80 1 client.27758 shutdown 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 msgr2=0x5605f4c83890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f662c0967c0 tx=0x7f662c00efa0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 shutdown_connections 2023-10-06T21:30:56.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.949+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605f4c26890 0x5605f4c7e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5605f4c8b340 0x5605f4c8d760 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f4c1b0c0 0x5605f4c83890 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 --2- 172.21.15.26:0/2256829155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f4c1b9e0 0x5605f4c83dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66197fa700 0 client.27758 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 >> 172.21.15.26:0/2256829155 conn(0x5605f4b90260 msgr2=0x5605f4c24f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 shutdown_connections 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 1 -- 172.21.15.26:0/2256829155 wait complete. 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f664bb07b80 20 client.27758 trim_cache size 0 max 16384 2023-10-06T21:30:56.959 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:30:56.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:30:56.961 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:30:56.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 msgr2=0x7f661c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 0x7f661c0a3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f663019a770 tx=0x7f662000b260 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 msgr2=0x7f6630199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9a0 tx=0x7f662c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.950+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 shutdown_connections 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f661c0a1670 0x7f661c0a3b30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663010d7b0 0x7f663019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66300ffc00 0x7f6630199160 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 --2- 172.21.15.26:0/3470440883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66301005b0 0x7f66301996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 >> 172.21.15.26:0/3470440883 conn(0x7f66300fb830 msgr2=0x7f663010eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 shutdown_connections 2023-10-06T21:30:56.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:30:56.951+0000 7f66381ff700 1 -- 172.21.15.26:0/3470440883 wait complete. 2023-10-06T21:30:56.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:56 smithi026 ceph-mon[98490]: pgmap v650: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:56.975 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:30:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:56 smithi167 ceph-mon[134494]: pgmap v650: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:57.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:56 smithi078 ceph-mon[134896]: pgmap v650: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:30:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:30:58 smithi167 ceph-mon[134494]: pgmap v651: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:30:58 smithi026 ceph-mon[98490]: pgmap v651: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:30:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:30:58 smithi078 ceph-mon[134896]: pgmap v651: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:00 smithi167 ceph-mon[134494]: pgmap v652: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:00 smithi026 ceph-mon[98490]: pgmap v652: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:00 smithi078 ceph-mon[134896]: pgmap v652: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:03.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:02 smithi167 ceph-mon[134494]: pgmap v653: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-10-06T21:31:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:02 smithi026 ceph-mon[98490]: pgmap v653: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-10-06T21:31:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:02 smithi078 ceph-mon[134896]: pgmap v653: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 19 op/s 2023-10-06T21:31:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:04 smithi167 ceph-mon[134494]: pgmap v654: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:31:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:04 smithi026 ceph-mon[98490]: pgmap v654: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:31:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:04 smithi078 ceph-mon[134896]: pgmap v654: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:31:05.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:05 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:05.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:05 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:06.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:05 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:31:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:06 smithi026 ceph-mon[98490]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:31:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:06 smithi026 ceph-mon[98490]: pgmap v656: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:06 smithi078 ceph-mon[134896]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:31:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:06 smithi078 ceph-mon[134896]: pgmap v656: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:07.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:06 smithi167 ceph-mon[134494]: osdmap e118: 12 total, 12 up, 12 in 2023-10-06T21:31:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:06 smithi167 ceph-mon[134494]: pgmap v656: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: pgmap v657: 97 pgs: 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:08.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:31:08.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:08 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:31:08.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: pgmap v657: 97 pgs: 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:31:08.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:31:08.825 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:08 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: pgmap v657: 97 pgs: 97 active+clean; 1.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi167 to 196.2M 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi167 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi078 to 196.2M 2023-10-06T21:31:08.872 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:08 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi078 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:10.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:09 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:10.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:10.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:09 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:31:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:09 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:31:11.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:10 smithi167 ceph-mon[134494]: pgmap v658: 97 pgs: 97 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:10 smithi167 ceph-mon[134494]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:31:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:10 smithi167 ceph-mon[134494]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:10 smithi026 ceph-mon[98490]: pgmap v658: 97 pgs: 97 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:10 smithi026 ceph-mon[98490]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:31:11.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:10 smithi026 ceph-mon[98490]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:10 smithi078 ceph-mon[134896]: pgmap v658: 97 pgs: 97 active+clean; 1.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:31:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:10 smithi078 ceph-mon[134896]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-10-06T21:31:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:10 smithi078 ceph-mon[134896]: Unable to set osd_memory_target on smithi026 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-10-06T21:31:13.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:12 smithi167 ceph-mon[134494]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-10-06T21:31:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:12 smithi026 ceph-mon[98490]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-10-06T21:31:13.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:12 smithi078 ceph-mon[134896]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-10-06T21:31:14.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:14 smithi167 ceph-mon[134494]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:31:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:14 smithi167 ceph-mon[134494]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:31:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:14 smithi026 ceph-mon[98490]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:31:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:14 smithi026 ceph-mon[98490]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:31:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:14 smithi078 ceph-mon[134896]: osdmap e119: 12 total, 12 up, 12 in 2023-10-06T21:31:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:14 smithi078 ceph-mon[134896]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:31:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:16 smithi167 ceph-mon[134494]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:16 smithi026 ceph-mon[98490]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:16 smithi078 ceph-mon[134896]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:18 smithi167 ceph-mon[134494]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:18 smithi026 ceph-mon[98490]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:18 smithi078 ceph-mon[134896]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:21.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:20 smithi167 ceph-mon[134494]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:20 smithi026 ceph-mon[98490]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:20 smithi078 ceph-mon[134896]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:31:23.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:22 smithi167 ceph-mon[134494]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:31:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:22 smithi026 ceph-mon[98490]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:31:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:22 smithi078 ceph-mon[134896]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:31:24.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:24 smithi167 ceph-mon[134494]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:31:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:24 smithi026 ceph-mon[98490]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:31:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:24 smithi078 ceph-mon[134896]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:31:26.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:31:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:26 smithi167 ceph-mon[134494]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:31:27.047 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:26 smithi026 ceph-mon[98490]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:31:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:26 smithi078 ceph-mon[134896]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:31:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 -- 172.21.15.26:0/2638852427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c100580 msgr2=0x7f6b0c10d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 --2- 172.21.15.26:0/2638852427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c100580 0x7f6b0c10d240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009990 tx=0x7f6b00009ca0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 -- 172.21.15.26:0/2638852427 shutdown_connections 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 --2- 172.21.15.26:0/2638852427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 0x7f6b0c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 --2- 172.21.15.26:0/2638852427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 --2- 172.21.15.26:0/2638852427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c100580 0x7f6b0c10d240 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 -- 172.21.15.26:0/2638852427 >> 172.21.15.26:0/2638852427 conn(0x7f6b0c0fb830 msgr2=0x7f6b0c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 -- 172.21.15.26:0/2638852427 shutdown_connections 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.200+0000 7f6b10dff700 1 -- 172.21.15.26:0/2638852427 wait complete. 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.201+0000 7f6b10dff700 1 Processor -- start 2023-10-06T21:31:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 -- start start 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c100580 0x7f6b0c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 0x7f6b0c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c19e190 con 0x7f6b0c100580 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c19e300 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.202+0000 7f6b10dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c19e4c0 con 0x7f6b0c10d780 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45066/0 (socket says 172.21.15.26:45066) 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 -- 172.21.15.26:0/1658774741 learned_addr learned my addr 172.21.15.26:0/1658774741 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c100580 0x7f6b0c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0ad9d700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 0x7f6b0c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 -- 172.21.15.26:0/1658774741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 msgr2=0x7f6b0c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 0x7f6b0c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 -- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c100580 msgr2=0x7f6b0c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c100580 0x7f6b0c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 -- 172.21.15.26:0/1658774741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000095f0 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.203+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9a0 tx=0x7f6afc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.204+0000 7f6afb7fe700 1 -- 172.21.15.26:0/1658774741 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc009790 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.204+0000 7f6afb7fe700 1 -- 172.21.15.26:0/1658774741 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc00cbd0 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.204+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c19e7a0 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.204+0000 7f6afb7fe700 1 -- 172.21.15.26:0/1658774741 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc018680 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.204+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1a5fe0 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.209+0000 7f6afb7fe700 1 -- 172.21.15.26:0/1658774741 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f6afc00cd40 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 Processor -- start 2023-10-06T21:31:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 -- start start 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9772a460 con 0x55ad977cc890 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9771bc20 con 0x55ad977c10c0 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.210+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad97719540 con 0x55ad977c19e0 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6afb7fe700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 0x7f6af40a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6afb7fe700 1 -- 172.21.15.26:0/1658774741 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6afc014070 con 0x7f6b0c0ffbd0 2023-10-06T21:31:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b09d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52842/0 (socket says 172.21.15.26:52842) 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977c14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45080/0 (socket says 172.21.15.26:45080) 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2046603169 learned_addr learned my addr 172.21.15.26:0/2046603169 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0ad9d700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 0x7f6af40a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2046603169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 msgr2=0x55ad977c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2046603169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 msgr2=0x55ad977cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.211+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2046603169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9753fde0 con 0x55ad977c19e0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.212+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c5b0 tx=0x7f6afc099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a6288534c66a16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.212+0000 7f6af8ff9700 1 -- 172.21.15.26:0/2046603169 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc09e410 con 0x55ad977c19e0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.212+0000 7f6b0ad9d700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 0x7f6af40a3bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009fd0 tx=0x7f6b0400b040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.212+0000 7f6af8ff9700 1 -- 172.21.15.26:0/2046603169 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc0a2650 con 0x55ad977c19e0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.212+0000 7f6af8ff9700 1 -- 172.21.15.26:0/2046603169 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc09a7d0 con 0x55ad977c19e0 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2046603169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 msgr2=0x55ad977cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c5b0 tx=0x7f6afc099c20 comp rx=0 tx=0).stop 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2046603169 shutdown_connections 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2046603169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977cc350 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2046603169 >> 172.21.15.26:0/2046603169 conn(0x55ad97736260 msgr2=0x55ad977bd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2046603169 shutdown_connections 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.213+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2046603169 wait complete. 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.214+0000 7f6b2480eb80 1 -- start start 2023-10-06T21:31:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.214+0000 7f6b2480eb80 1 Processor -- start 2023-10-06T21:31:27.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977cc890 0x55ad977c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad97714e30 con 0x55ad977c10c0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad976b1710 con 0x55ad977cc890 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad976b13b0 con 0x55ad977c19e0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b2480eb80 10 client.0 _ll_register_callbacks cb 0x55ad9750e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.215+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46744/0 (socket says 172.21.15.26:46744) 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2972093983 learned_addr learned my addr 172.21.15.26:0/2972093983 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0ad9d700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977cc890 0x55ad977c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2972093983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 msgr2=0x55ad977c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977cc890 msgr2=0x55ad977c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977cc890 0x55ad977c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 -- 172.21.15.26:0/2972093983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1a5fe0 con 0x55ad977c10c0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6afc000c00 tx=0x7f6afc0cd3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311def588425844 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.216+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc09a720 con 0x55ad977c10c0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6afc09e0f0 con 0x55ad977c10c0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad977b4f10 con 0x55ad977c10c0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6afc09e410 con 0x55ad977c10c0 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 10 client.17388 fetch_fsmap learned FSMap version 67 2023-10-06T21:31:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55ad97528c60 con 0x55ad977c10c0 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc0a6080 con 0x55ad977c10c0 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6afc099490 con 0x55ad977c10c0 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 10 client.17388 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 20 client.17388 populate_metadata read hostname 'smithi026' 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad977dbff0 0x55ad977de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 4 client.17388 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.217+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad9770de10 con 0x55ad977dbff0 2023-10-06T21:31:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.218+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad977dbff0 0x55ad977de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.218+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad977dbff0 0x55ad977de3e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3ead2566bba05c server_cookie=727e4cfb39e8b0e3 in_seq=0 out_seq=0 2023-10-06T21:31:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.218+0000 7f6af1ffb700 10 client.17388 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6af1ffb700 1 -- 172.21.15.26:0/2972093983 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ad9770de10 con 0x55ad977dbff0 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6af1ffb700 10 client.17388 handle_command_reply: tid=0 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 client.17388 shutdown 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 msgr2=0x55ad977cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6afc000c00 tx=0x7f6afc0cd3a0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 shutdown_connections 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad977dbff0 0x55ad977de3e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977cc890 0x55ad977c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977cc2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6af1ffb700 0 client.17388 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/2972093983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.219+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 >> 172.21.15.26:0/2972093983 conn(0x55ad97736260 msgr2=0x55ad977bcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.220+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 shutdown_connections 2023-10-06T21:31:27.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.220+0000 7f6b2480eb80 1 -- 172.21.15.26:0/2972093983 wait complete. 2023-10-06T21:31:27.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.220+0000 7f6b2480eb80 20 client.17388 trim_cache size 0 max 16384 2023-10-06T21:31:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 Processor -- start 2023-10-06T21:31:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 -- start start 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad97595920 con 0x55ad977cc890 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad977ca4f0 con 0x55ad977c10c0 2023-10-06T21:31:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad977c0f00 con 0x55ad977c19e0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45098/0 (socket says 172.21.15.26:45098) 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b0ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.250+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3847863404 learned_addr learned my addr 172.21.15.26:0/3847863404 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3847863404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 msgr2=0x55ad977c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3847863404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 msgr2=0x55ad977c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3847863404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad977b9140 con 0x55ad977c10c0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009230 tx=0x7f6afc098870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8043bf073e581c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6af97fa700 1 -- 172.21.15.26:0/3847863404 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc099c20 con 0x55ad977c10c0 2023-10-06T21:31:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6af97fa700 1 -- 172.21.15.26:0/3847863404 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad977b4f10 con 0x55ad977c10c0 2023-10-06T21:31:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.251+0000 7f6af97fa700 1 -- 172.21.15.26:0/3847863404 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc09a520 con 0x55ad977c10c0 2023-10-06T21:31:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3847863404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 msgr2=0x55ad977cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009230 tx=0x7f6afc098870 comp rx=0 tx=0).stop 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3847863404 shutdown_connections 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977cc890 0x55ad977c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c10c0 0x55ad977cbe90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3847863404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977c19e0 0x55ad977c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3847863404 >> 172.21.15.26:0/3847863404 conn(0x55ad97736260 msgr2=0x55ad977bbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3847863404 shutdown_connections 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.252+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3847863404 wait complete. 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.253+0000 7f6b2480eb80 1 -- start start 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.253+0000 7f6b2480eb80 1 Processor -- start 2023-10-06T21:31:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.253+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.253+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b2480eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977cc890 0x55ad977d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9782efd0 con 0x55ad977c10c0 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9782f140 con 0x55ad977c19e0 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b2480eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9782f300 con 0x55ad977cc890 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b2480eb80 10 client.0 _ll_register_callbacks cb 0x7f6afc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b09d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46772/0 (socket says 172.21.15.26:46772) 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3882258474 learned_addr learned my addr 172.21.15.26:0/3882258474 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b09d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:45100/0 (socket says 172.21.15.26:45100) 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0ad9d700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977cc890 0x55ad977d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3882258474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977cc890 msgr2=0x55ad977d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977cc890 0x55ad977d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 msgr2=0x55ad977d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b0a59c700 1 -- 172.21.15.26:0/3882258474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad97528c60 con 0x55ad977c10c0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.254+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6b0a59c700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6afc098610 tx=0x7f6afc010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f0e16012f8ffa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc09a720 con 0x55ad977c10c0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6afc09f490 con 0x55ad977c10c0 2023-10-06T21:31:27.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6afc0a2650 con 0x55ad977c10c0 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6afc099c20 con 0x55ad977c10c0 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6b2480eb80 10 client.17394 fetch_fsmap learned FSMap version 67 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55ad97566250 con 0x55ad977c10c0 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.255+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6afc0a6030 con 0x55ad977c10c0 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6afc0a8070 con 0x55ad977c10c0 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 10 client.17394 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 20 client.17394 populate_metadata read hostname 'smithi026' 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad97830590 0x55ad97832980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 4 client.17394 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:31:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad9770db30 con 0x55ad97830590 2023-10-06T21:31:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.256+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad97830590 0x55ad97832980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b09d9b700 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad97830590 0x55ad97832980 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af5acbb22695503 server_cookie=c8def034a55fab35 in_seq=0 out_seq=0 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6af1ffb700 10 client.17394 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6af1ffb700 1 -- 172.21.15.26:0/3882258474 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ad9770db30 con 0x55ad97830590 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6af1ffb700 10 client.17394 handle_command_reply: tid=0 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 client.17394 shutdown 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 msgr2=0x55ad977d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6afc098610 tx=0x7f6afc010aa0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 shutdown_connections 2023-10-06T21:31:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ad977cc890 0x55ad977d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ad977c10c0 0x55ad977d13c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55ad97830590 0x55ad97832980 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 --2- 172.21.15.26:0/3882258474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad977c19e0 0x55ad977d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 >> 172.21.15.26:0/3882258474 conn(0x55ad97736260 msgr2=0x55ad977bc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6af1ffb700 0 client.17394 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 shutdown_connections 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 1 -- 172.21.15.26:0/3882258474 wait complete. 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.257+0000 7f6b2480eb80 20 client.17394 trim_cache size 0 max 16384 2023-10-06T21:31:27.265 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:31:27.266 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 msgr2=0x7f6af40a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 0x7f6af40a3bb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009fd0 tx=0x7f6b0400b040 comp rx=0 tx=0).stop 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 msgr2=0x7f6b0c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9a0 tx=0x7f6afc00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 shutdown_connections 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b0c10d780 0x7f6b0c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b0c0ffbd0 0x7f6b0c1991b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f6af40a16f0 0x7f6af40a3bb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 --2- 172.21.15.26:0/1658774741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c100580 0x7f6b0c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 >> 172.21.15.26:0/1658774741 conn(0x7f6b0c0fb830 msgr2=0x7f6b0c10eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 shutdown_connections 2023-10-06T21:31:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:27.258+0000 7f6b10dff700 1 -- 172.21.15.26:0/1658774741 wait complete. 2023-10-06T21:31:27.279 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:31:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:28 smithi167 ceph-mon[134494]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:28 smithi026 ceph-mon[98490]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:28 smithi078 ceph-mon[134896]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:31.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:30 smithi167 ceph-mon[134494]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:30 smithi026 ceph-mon[98490]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:30 smithi078 ceph-mon[134896]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:33.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:32 smithi167 ceph-mon[134494]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:32 smithi026 ceph-mon[98490]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:33.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:32 smithi078 ceph-mon[134896]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:34.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:34 smithi167 ceph-mon[134494]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:34.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:34 smithi026 ceph-mon[98490]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:34 smithi078 ceph-mon[134896]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:37.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:36 smithi167 ceph-mon[134494]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:36 smithi026 ceph-mon[98490]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:36 smithi078 ceph-mon[134896]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:39.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:38 smithi167 ceph-mon[134494]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:38 smithi026 ceph-mon[98490]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:38 smithi078 ceph-mon[134896]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:41.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:40 smithi167 ceph-mon[134494]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:40 smithi026 ceph-mon[98490]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:40 smithi078 ceph-mon[134896]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:43.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:42 smithi167 ceph-mon[134494]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:42 smithi026 ceph-mon[98490]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:42 smithi078 ceph-mon[134896]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:44.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:44 smithi167 ceph-mon[134494]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:44 smithi026 ceph-mon[98490]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:44 smithi078 ceph-mon[134896]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:47.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:46 smithi167 ceph-mon[134494]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:46 smithi026 ceph-mon[98490]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:46 smithi078 ceph-mon[134896]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:49.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:48 smithi167 ceph-mon[134494]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:48 smithi026 ceph-mon[98490]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:48 smithi078 ceph-mon[134896]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:51.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:50 smithi167 ceph-mon[134494]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:50 smithi026 ceph-mon[98490]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:50 smithi078 ceph-mon[134896]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:53.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:52 smithi167 ceph-mon[134494]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:52 smithi026 ceph-mon[98490]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:52 smithi078 ceph-mon[134896]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:54.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:54 smithi167 ceph-mon[134494]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:54 smithi026 ceph-mon[98490]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:54 smithi078 ceph-mon[134896]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:31:57.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:56 smithi167 ceph-mon[134494]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:56 smithi026 ceph-mon[98490]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:56 smithi078 ceph-mon[134896]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:31:57.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 -- 172.21.15.26:0/1413952705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681010b0 msgr2=0x7fb168101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1413952705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681010b0 0x7fb168101490 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb164009990 tx=0x7fb164009ca0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 -- 172.21.15.26:0/1413952705 shutdown_connections 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1413952705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1681061d0 0x7fb16810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1413952705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681010b0 0x7fb168101490 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1413952705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb168101a60 0x7fb168105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 -- 172.21.15.26:0/1413952705 >> 172.21.15.26:0/1413952705 conn(0x7fb1680fc930 msgr2=0x7fb1680fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 -- 172.21.15.26:0/1413952705 shutdown_connections 2023-10-06T21:31:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.521+0000 7fb16fdff700 1 -- 172.21.15.26:0/1413952705 wait complete. 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.522+0000 7fb16fdff700 1 Processor -- start 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.522+0000 7fb16fdff700 1 -- start start 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 0x7fb168196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681061d0 0x7fb168197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16819c410 con 0x7fb1681010b0 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16819c580 con 0x7fb16819b990 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16fdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16819c740 con 0x7fb1681061d0 2023-10-06T21:31:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 0x7fb168196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59280/0 (socket says 172.21.15.26:59280) 2023-10-06T21:31:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 -- 172.21.15.26:0/1543725332 learned_addr learned my addr 172.21.15.26:0/1543725332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 0x7fb168196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52566/0 (socket says 172.21.15.26:52566) 2023-10-06T21:31:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16d39a700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681061d0 0x7fb168197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 -- 172.21.15.26:0/1543725332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681061d0 msgr2=0x7fb168197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681061d0 0x7fb168197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 -- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 msgr2=0x7fb168196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 0x7fb168196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.523+0000 7fb16e39c700 1 -- 172.21.15.26:0/1543725332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009c40 con 0x7fb16819b990 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c370 tx=0x7fb16000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb15effd700 1 -- 172.21.15.26:0/1543725332 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb16819b990 2023-10-06T21:31:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1640095f0 con 0x7fb16819b990 2023-10-06T21:31:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb15effd700 1 -- 172.21.15.26:0/1543725332 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16000f040 con 0x7fb16819b990 2023-10-06T21:31:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb15effd700 1 -- 172.21.15.26:0/1543725332 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000e050 con 0x7fb16819b990 2023-10-06T21:31:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.524+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681a3f90 con 0x7fb16819b990 2023-10-06T21:31:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.528+0000 7fb15effd700 1 -- 172.21.15.26:0/1543725332 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb16000cb60 con 0x7fb16819b990 2023-10-06T21:31:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.529+0000 7fb15effd700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 0x7fb1540a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.529+0000 7fb15effd700 1 -- 172.21.15.26:0/1543725332 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb1600cca60 con 0x7fb16819b990 2023-10-06T21:31:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.530+0000 7fb16db9b700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 0x7fb1540a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.530+0000 7fb183679b80 1 Processor -- start 2023-10-06T21:31:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.530+0000 7fb183679b80 1 -- start start 2023-10-06T21:31:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.530+0000 7fb16db9b700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 0x7fb1540a3ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b2e0 tx=0x7fb164005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6aa5460 con 0x5625e6b4bca0 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6a96c20 con 0x5625e6b40df0 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb183679b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6a94540 con 0x5625e6b404d0 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59292/0 (socket says 172.21.15.26:59292) 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 -- 172.21.15.26:0/2000927654 learned_addr learned my addr 172.21.15.26:0/2000927654 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16e39c700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16db9b700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 -- 172.21.15.26:0/2000927654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 msgr2=0x5625e6b408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 -- 172.21.15.26:0/2000927654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 msgr2=0x5625e6b4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16d39a700 1 -- 172.21.15.26:0/2000927654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e68bade0 con 0x5625e6b40df0 2023-10-06T21:31:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.531+0000 7fb16e39c700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:31:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.532+0000 7fb16d39a700 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb158009cf0 tx=0x7fb158011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408bb4d5a5eccf05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.532+0000 7fb153fff700 1 -- 172.21.15.26:0/2000927654 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158011e30 con 0x5625e6b40df0 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.532+0000 7fb153fff700 1 -- 172.21.15.26:0/2000927654 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb158004510 con 0x5625e6b40df0 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.532+0000 7fb153fff700 1 -- 172.21.15.26:0/2000927654 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158005280 con 0x5625e6b40df0 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 -- 172.21.15.26:0/2000927654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 msgr2=0x5625e6b4b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb158009cf0 tx=0x7fb158011440 comp rx=0 tx=0).stop 2023-10-06T21:31:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 -- 172.21.15.26:0/2000927654 shutdown_connections 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 --2- 172.21.15.26:0/2000927654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b4b760 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 -- 172.21.15.26:0/2000927654 >> 172.21.15.26:0/2000927654 conn(0x5625e6ab1260 msgr2=0x5625e6b38450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 -- 172.21.15.26:0/2000927654 shutdown_connections 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.533+0000 7fb183679b80 1 -- 172.21.15.26:0/2000927654 wait complete. 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.534+0000 7fb183679b80 1 -- start start 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.534+0000 7fb183679b80 1 Processor -- start 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b48a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6a8fe30 con 0x5625e6b4bca0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6a2c710 con 0x5625e6b40df0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6a2c3b0 con 0x5625e6b404d0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb183679b80 10 client.0 _ll_register_callbacks cb 0x5625e6889800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb16d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b48a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb16d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59298/0 (socket says 172.21.15.26:59298) 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb16d39a700 1 -- 172.21.15.26:0/811670281 learned_addr learned my addr 172.21.15.26:0/811670281 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.535+0000 7fb16e39c700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 -- 172.21.15.26:0/811670281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 msgr2=0x5625e6b48a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b48a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 -- 172.21.15.26:0/811670281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 msgr2=0x5625e6b494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 -- 172.21.15.26:0/811670281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb158009920 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb16d39a700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb158011840 tx=0x7fb158005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8253cfc60f8eb276 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb1517fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158004360 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb158003680 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625e6b2ff10 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.536+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb158003840 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 10 client.27788 fetch_fsmap learned FSMap version 67 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5625e68a3c60 con 0x5625e6b40df0 2023-10-06T21:31:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625e6b2ff10 con 0x5625e6b40df0 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb158022400 con 0x5625e6b40df0 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 10 client.27788 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 20 client.27788 populate_metadata read hostname 'smithi026' 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6b5b680 0x5625e6b5daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 4 client.27788 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:31:57.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625e6a88e10 con 0x5625e6b5b680 2023-10-06T21:31:57.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.537+0000 7fb16db9b700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6b5b680 0x5625e6b5daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.538+0000 7fb16db9b700 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6b5b680 0x5625e6b5daa0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1863950e035ec6 server_cookie=af4234e00a34a9b6 in_seq=0 out_seq=0 2023-10-06T21:31:57.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.538+0000 7fb1517fa700 10 client.27788 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.538+0000 7fb1517fa700 1 -- 172.21.15.26:0/811670281 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5625e6a88e10 con 0x5625e6b5b680 2023-10-06T21:31:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.538+0000 7fb1517fa700 10 client.27788 handle_command_reply: tid=0 2023-10-06T21:31:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 client.27788 shutdown 2023-10-06T21:31:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 msgr2=0x5625e6b48f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb158011840 tx=0x7fb158005e40 comp rx=0 tx=0).stop 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 shutdown_connections 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b404d0 0x5625e6b48a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6b5b680 0x5625e6b5daa0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 --2- 172.21.15.26:0/811670281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b40df0 0x5625e6b48f80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb1517fa700 0 client.27788 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 >> 172.21.15.26:0/811670281 conn(0x5625e6ab1260 msgr2=0x5625e6b37fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 shutdown_connections 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 1 -- 172.21.15.26:0/811670281 wait complete. 2023-10-06T21:31:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.539+0000 7fb183679b80 20 client.27788 trim_cache size 0 max 16384 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.556+0000 7fb183679b80 1 Processor -- start 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 -- start start 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 0x5625e6b43370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b49310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6910920 con 0x5625e6b404d0 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6b438b0 con 0x5625e6b4bca0 2023-10-06T21:31:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb183679b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6b47000 con 0x5625e6b40df0 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb16e39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 0x5625e6b43370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb16e39c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59310/0 (socket says 172.21.15.26:59310) 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb16e39c700 1 -- 172.21.15.26:0/1802142049 learned_addr learned my addr 172.21.15.26:0/1802142049 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.557+0000 7fb16d39a700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b49310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 -- 172.21.15.26:0/1802142049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 msgr2=0x5625e6b49310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b49310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 -- 172.21.15.26:0/1802142049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 msgr2=0x5625e6b43370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 0x5625e6b43370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 -- 172.21.15.26:0/1802142049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e6b34140 con 0x5625e6b4bca0 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16d39a700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b49310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16db9b700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 0x5625e6b43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb16e39c700 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb160099960 tx=0x7fb160084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0344f8033d03cee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb15cbff700 1 -- 172.21.15.26:0/1802142049 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16009a0d0 con 0x5625e6b4bca0 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb15cbff700 1 -- 172.21.15.26:0/1802142049 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1600999d0 con 0x5625e6b4bca0 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.558+0000 7fb15cbff700 1 -- 172.21.15.26:0/1802142049 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160099c30 con 0x5625e6b4bca0 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- 172.21.15.26:0/1802142049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 msgr2=0x5625e6b47960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb160099960 tx=0x7fb160084040 comp rx=0 tx=0).stop 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- 172.21.15.26:0/1802142049 shutdown_connections 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b4bca0 0x5625e6b47960 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b404d0 0x5625e6b43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 --2- 172.21.15.26:0/1802142049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b49310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- 172.21.15.26:0/1802142049 >> 172.21.15.26:0/1802142049 conn(0x5625e6ab1260 msgr2=0x5625e6b47250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- 172.21.15.26:0/1802142049 shutdown_connections 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- 172.21.15.26:0/1802142049 wait complete. 2023-10-06T21:31:57.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.559+0000 7fb183679b80 1 -- start start 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 Processor -- start 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6b51380 con 0x5625e6b4bca0 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6b514f0 con 0x5625e6b404d0 2023-10-06T21:31:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e6b516b0 con 0x5625e6b40df0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb183679b80 10 client.0 _ll_register_callbacks cb 0x7fb158009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.560+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:59324/0 (socket says 172.21.15.26:59324) 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 -- 172.21.15.26:0/4289357749 learned_addr learned my addr 172.21.15.26:0/4289357749 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16e39c700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16d39a700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 -- 172.21.15.26:0/4289357749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 msgr2=0x5625e6b50900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b50900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 -- 172.21.15.26:0/4289357749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 msgr2=0x5625e6b50e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b50e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 -- 172.21.15.26:0/4289357749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16402c040 con 0x5625e6b404d0 2023-10-06T21:31:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16e39c700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb16db9b700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb1640098e0 tx=0x7fb164009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20c8b8cc30bef86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb1517fa700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.561+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164035420 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb158004510 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb164035a90 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb164035580 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 10 client.27794 fetch_fsmap learned FSMap version 67 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7fb1681a3f90 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb16404a080 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb164004d80 con 0x5625e6b404d0 2023-10-06T21:31:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 10 client.27794 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 10 client.27794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 20 client.27794 populate_metadata read hostname 'smithi026' 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6bb09e0 0x5625e6bb2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 4 client.27794 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.562+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625e6a88b30 con 0x5625e6bb09e0 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb16d39a700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6bb09e0 0x5625e6bb2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb16d39a700 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6bb09e0 0x5625e6bb2e00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ae375a6cb2664 server_cookie=1e112a40f995da48 in_seq=0 out_seq=0 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb1517fa700 10 client.27794 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb1517fa700 1 -- 172.21.15.26:0/4289357749 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5625e6a88b30 con 0x5625e6bb09e0 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb1517fa700 10 client.27794 handle_command_reply: tid=0 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 client.27794 shutdown 2023-10-06T21:31:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 msgr2=0x5625e6b525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb1640098e0 tx=0x7fb164009ca0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 shutdown_connections 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5625e6b4bca0 0x5625e6b50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.563+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x5625e6bb09e0 0x5625e6bb2e00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625e6b404d0 0x5625e6b525d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb1517fa700 0 client.27794 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 1 --2- 172.21.15.26:0/4289357749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625e6b40df0 0x5625e6b50900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 >> 172.21.15.26:0/4289357749 conn(0x5625e6ab1260 msgr2=0x5625e6b49260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 shutdown_connections 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 1 -- 172.21.15.26:0/4289357749 wait complete. 2023-10-06T21:31:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.564+0000 7fb183679b80 20 client.27794 trim_cache size 0 max 16384 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:31:57.572 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:31:57.573 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:31:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 msgr2=0x7fb1540a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 0x7fb1540a3ab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b2e0 tx=0x7fb164005fb0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 msgr2=0x7fb16819be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c370 tx=0x7fb16000c680 comp rx=0 tx=0).stop 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 shutdown_connections 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16819b990 0x7fb16819be10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb1540a15f0 0x7fb1540a3ab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1681010b0 0x7fb168196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 --2- 172.21.15.26:0/1543725332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1681061d0 0x7fb168197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 >> 172.21.15.26:0/1543725332 conn(0x7fb1680fc930 msgr2=0x7fb168104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 shutdown_connections 2023-10-06T21:31:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:31:57.565+0000 7fb16fdff700 1 -- 172.21.15.26:0/1543725332 wait complete. 2023-10-06T21:31:57.590 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:31:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:31:58 smithi167 ceph-mon[134494]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:31:58 smithi026 ceph-mon[98490]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:31:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:31:58 smithi078 ceph-mon[134896]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:01.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:00 smithi167 ceph-mon[134494]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:00 smithi026 ceph-mon[98490]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:00 smithi078 ceph-mon[134896]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:03.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:02 smithi167 ceph-mon[134494]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:32:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:02 smithi026 ceph-mon[98490]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:32:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:02 smithi078 ceph-mon[134896]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:32:04.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:04 smithi167 ceph-mon[134494]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:32:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:04 smithi026 ceph-mon[98490]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:32:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:04 smithi078 ceph-mon[134896]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:32:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:06 smithi026 ceph-mon[98490]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:32:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:06 smithi026 ceph-mon[98490]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:06 smithi078 ceph-mon[134896]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:32:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:06 smithi078 ceph-mon[134896]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:06 smithi167 ceph-mon[134494]: osdmap e120: 12 total, 12 up, 12 in 2023-10-06T21:32:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:06 smithi167 ceph-mon[134494]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:09.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:08 smithi167 ceph-mon[134494]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:08 smithi026 ceph-mon[98490]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:08 smithi078 ceph-mon[134896]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:10 smithi167 ceph-mon[134494]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:11.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:10 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:11.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:10 smithi026 ceph-mon[98490]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:11.077 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:10 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:11.165 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:10 smithi078 ceph-mon[134896]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:32:11.165 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:10 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:32:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:11 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:11 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:12.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:11 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:32:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:11 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:11 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:12.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:11 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:32:12.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:11 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:32:12.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:11 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:32:12.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:11 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:32:13.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:12 smithi167 ceph-mon[134494]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:32:13.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:12 smithi026 ceph-mon[98490]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:32:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:12 smithi078 ceph-mon[134896]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:32:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:14 smithi167 ceph-mon[134494]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:32:14.783 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:14 smithi167 ceph-mon[134494]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:32:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:14 smithi026 ceph-mon[98490]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:32:14.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:14 smithi026 ceph-mon[98490]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:32:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:14 smithi078 ceph-mon[134896]: osdmap e121: 12 total, 12 up, 12 in 2023-10-06T21:32:14.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:14 smithi078 ceph-mon[134896]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:32:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:16 smithi167 ceph-mon[134494]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:16 smithi026 ceph-mon[98490]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:17.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:16 smithi078 ceph-mon[134896]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:19.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:18 smithi167 ceph-mon[134494]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:18 smithi026 ceph-mon[98490]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:19.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:18 smithi078 ceph-mon[134896]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:21.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:20 smithi167 ceph-mon[134494]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:20 smithi026 ceph-mon[98490]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:20 smithi078 ceph-mon[134896]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:32:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:22 smithi026 ceph-mon[98490]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:32:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:22 smithi078 ceph-mon[134896]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:32:23.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:22 smithi167 ceph-mon[134494]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:32:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:24 smithi026 ceph-mon[98490]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:32:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:24 smithi078 ceph-mon[134896]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:32:25.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:24 smithi167 ceph-mon[134494]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:32:27.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:26 smithi167 ceph-mon[134494]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:32:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:26 smithi026 ceph-mon[98490]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:32:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:26 smithi078 ceph-mon[134896]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:32:27.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 -- 172.21.15.26:0/4023797350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5578101ff0 msgr2=0x7f55781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 --2- 172.21.15.26:0/4023797350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5578101ff0 0x7f55781023d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5568009990 tx=0x7f5568009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 -- 172.21.15.26:0/4023797350 shutdown_connections 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 --2- 172.21.15.26:0/4023797350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f557810b410 0x7f557810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 --2- 172.21.15.26:0/4023797350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5578101ff0 0x7f55781023d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 --2- 172.21.15.26:0/4023797350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 0x7f557810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 -- 172.21.15.26:0/4023797350 >> 172.21.15.26:0/4023797350 conn(0x7f55780fb830 msgr2=0x7f55780fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 -- 172.21.15.26:0/4023797350 shutdown_connections 2023-10-06T21:32:27.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.842+0000 7f557d7ff700 1 -- 172.21.15.26:0/4023797350 wait complete. 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.843+0000 7f557d7ff700 1 Processor -- start 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.843+0000 7f557d7ff700 1 -- start start 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 0x7f55781997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f557810b410 0x7f557819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557819e240 con 0x7f557810b410 2023-10-06T21:32:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557819e3b0 con 0x7f5578101ff0 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f557d7ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557819e570 con 0x7f5578102910 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40618/0 (socket says 172.21.15.26:40618) 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 -- 172.21.15.26:0/1163144946 learned_addr learned my addr 172.21.15.26:0/1163144946 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f55767fc700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 0x7f55781997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f55777fe700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f557810b410 0x7f557819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 -- 172.21.15.26:0/1163144946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 msgr2=0x7f55781997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 0x7f55781997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 -- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f557810b410 msgr2=0x7f557819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f557810b410 0x7f557819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.844+0000 7f5576ffd700 1 -- 172.21.15.26:0/1163144946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680095f0 con 0x7f5578101ff0 2023-10-06T21:32:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f555ffff700 1 -- 172.21.15.26:0/1163144946 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556801d070 con 0x7f5578101ff0 2023-10-06T21:32:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f555ffff700 1 -- 172.21.15.26:0/1163144946 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5568022470 con 0x7f5578101ff0 2023-10-06T21:32:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f555ffff700 1 -- 172.21.15.26:0/1163144946 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568017650 con 0x7f5578101ff0 2023-10-06T21:32:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557819e7f0 con 0x7f5578101ff0 2023-10-06T21:32:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.845+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557819ea70 con 0x7f5578101ff0 2023-10-06T21:32:27.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.848+0000 7f555ffff700 1 -- 172.21.15.26:0/1163144946 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f556800fad0 con 0x7f5578101ff0 2023-10-06T21:32:27.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.849+0000 7f55911ecb80 1 Processor -- start 2023-10-06T21:32:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 -- start start 2023-10-06T21:32:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f555ffff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 0x7f55600a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f555ffff700 1 -- 172.21.15.26:0/1163144946 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f55680ce3b0 con 0x7f5578101ff0 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d0c0 0x556e9632d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55777fe700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 0x7f55600a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 0x556e96338350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96296460 con 0x556e9632d9e0 2023-10-06T21:32:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96287c20 con 0x556e96338890 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.850+0000 7f55911ecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96285540 con 0x556e9632d0c0 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 0x556e96338350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d0c0 0x556e9632d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 0x556e96338350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56362/0 (socket says 172.21.15.26:56362) 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55767fc700 1 -- 172.21.15.26:0/1634798144 learned_addr learned my addr 172.21.15.26:0/1634798144 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 0x7f55600a3b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f557000f540 tx=0x7f5570009920 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 -- 172.21.15.26:0/1634798144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d0c0 msgr2=0x556e9632d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d0c0 0x556e9632d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 -- 172.21.15.26:0/1634798144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 msgr2=0x556e96338350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 0x556e96338350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 -- 172.21.15.26:0/1634798144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e960abde0 con 0x556e96338890 2023-10-06T21:32:27.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.851+0000 7f55777fe700 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633ac80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5570019780 tx=0x7f557001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f73d855bf48b5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.852+0000 7f555d7fa700 1 -- 172.21.15.26:0/1634798144 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000e040 con 0x556e96338890 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.852+0000 7f555d7fa700 1 -- 172.21.15.26:0/1634798144 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f557000f040 con 0x556e96338890 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.852+0000 7f555d7fa700 1 -- 172.21.15.26:0/1634798144 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570004020 con 0x556e96338890 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.852+0000 7f55911ecb80 1 -- 172.21.15.26:0/1634798144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 msgr2=0x556e9633ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.852+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633ac80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5570019780 tx=0x7f557001c000 comp rx=0 tx=0).stop 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 -- 172.21.15.26:0/1634798144 shutdown_connections 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633ac80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d0c0 0x556e9632d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1634798144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d9e0 0x556e96338350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 -- 172.21.15.26:0/1634798144 >> 172.21.15.26:0/1634798144 conn(0x556e962a2260 msgr2=0x556e96329450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 -- 172.21.15.26:0/1634798144 shutdown_connections 2023-10-06T21:32:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 -- 172.21.15.26:0/1634798144 wait complete. 2023-10-06T21:32:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.853+0000 7f55911ecb80 1 -- start start 2023-10-06T21:32:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 Processor -- start 2023-10-06T21:32:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e96335630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96280e30 con 0x556e9632d0c0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9621d710 con 0x556e96338890 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9621d3b0 con 0x556e9632d9e0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.854+0000 7f55911ecb80 10 client.0 _ll_register_callbacks cb 0x556e9607a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f55767fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e96335630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f55777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f55767fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:33714/0 (socket says 172.21.15.26:33714) 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f55767fc700 1 -- 172.21.15.26:0/3790113443 learned_addr learned my addr 172.21.15.26:0/3790113443 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f55777fe700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40648/0 (socket says 172.21.15.26:40648) 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 -- 172.21.15.26:0/3790113443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 msgr2=0x556e96335b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 -- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 msgr2=0x556e963360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 -- 172.21.15.26:0/3790113443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5570019460 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.855+0000 7f5576ffd700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e96335630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f55680cba90 tx=0x7f556809ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d3c56fcb4333bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556809b3b0 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e96320f10 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f556809f500 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55680a36b0 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f55911ecb80 10 client.17424 fetch_fsmap learned FSMap version 67 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x556e96094c60 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55680b7070 con 0x556e9632d0c0 2023-10-06T21:32:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.856+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55680a3220 con 0x556e9632d0c0 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 10 client.17424 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 20 client.17424 populate_metadata read hostname 'smithi026' 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e96348080 0x556e9634a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 4 client.17424 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e96279e10 con 0x556e96348080 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55767fc700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e96348080 0x556e9634a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f55767fc700 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e96348080 0x556e9634a470 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c514df30a64a269 server_cookie=a07b585601de3caa in_seq=0 out_seq=0 2023-10-06T21:32:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.857+0000 7f555aff5700 10 client.17424 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f555aff5700 1 -- 172.21.15.26:0/3790113443 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e96279e10 con 0x556e96348080 2023-10-06T21:32:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f555aff5700 10 client.17424 handle_command_reply: tid=0 2023-10-06T21:32:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 client.17424 shutdown 2023-10-06T21:32:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 msgr2=0x556e96335630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e96335630 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f55680cba90 tx=0x7f556809ad10 comp rx=0 tx=0).stop 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 shutdown_connections 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e96348080 0x556e9634a470 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.858+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e96335630 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 1 --2- 172.21.15.26:0/3790113443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 >> 172.21.15.26:0/3790113443 conn(0x556e962a2260 msgr2=0x556e96328fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f555aff5700 0 client.17424 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 shutdown_connections 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 1 -- 172.21.15.26:0/3790113443 wait complete. 2023-10-06T21:32:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.859+0000 7f55911ecb80 20 client.17424 trim_cache size 0 max 16384 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 Processor -- start 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 -- start start 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9632ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96101920 con 0x556e9632d0c0 2023-10-06T21:32:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e96325310 con 0x556e96338890 2023-10-06T21:32:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55911ecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9632d6f0 con 0x556e9632d9e0 2023-10-06T21:32:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.878+0000 7f55777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9632ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:40664/0 (socket says 172.21.15.26:40664) 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 -- 172.21.15.26:0/2644240455 learned_addr learned my addr 172.21.15.26:0/2644240455 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55767fc700 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 -- 172.21.15.26:0/2644240455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 msgr2=0x556e96335ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 -- 172.21.15.26:0/2644240455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 msgr2=0x556e9632ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9632ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 -- 172.21.15.26:0/2644240455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e96327e90 con 0x556e96338890 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f5576ffd700 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9632ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:32:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.879+0000 7f55777fe700 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5570019d70 tx=0x7f5570017b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ed04682dc7e4db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f555dffb700 1 -- 172.21.15.26:0/2644240455 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000e040 con 0x556e96338890 2023-10-06T21:32:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f555dffb700 1 -- 172.21.15.26:0/2644240455 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5570017dd0 con 0x556e96338890 2023-10-06T21:32:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f555dffb700 1 -- 172.21.15.26:0/2644240455 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570003d80 con 0x556e96338890 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 -- 172.21.15.26:0/2644240455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 msgr2=0x556e963342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5570019d70 tx=0x7f5570017b60 comp rx=0 tx=0).stop 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 -- 172.21.15.26:0/2644240455 shutdown_connections 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e963342a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9632ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 --2- 172.21.15.26:0/2644240455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e96335ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 -- 172.21.15.26:0/2644240455 >> 172.21.15.26:0/2644240455 conn(0x556e962a2260 msgr2=0x556e96333b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.880+0000 7f55911ecb80 1 -- 172.21.15.26:0/2644240455 shutdown_connections 2023-10-06T21:32:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.881+0000 7f55911ecb80 1 -- 172.21.15.26:0/2644240455 wait complete. 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.881+0000 7f55911ecb80 1 -- start start 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.881+0000 7f55911ecb80 1 Processor -- start 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e9633d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9633deb0 con 0x556e9632d0c0 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9633e020 con 0x556e96338890 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9633e1e0 con 0x556e9632d9e0 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55911ecb80 10 client.0 _ll_register_callbacks cb 0x7f556809b230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f5576ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56390/0 (socket says 172.21.15.26:56390) 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f5576ffd700 1 -- 172.21.15.26:0/1491642775 learned_addr learned my addr 172.21.15.26:0/1491642775 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.882+0000 7f55777fe700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f55767fc700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e9633d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 -- 172.21.15.26:0/1491642775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 msgr2=0x556e9633d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e9633d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 -- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 msgr2=0x556e9633d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 -- 172.21.15.26:0/1491642775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e96094c60 con 0x556e9632d0c0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f5576ffd700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f55680978b0 tx=0x7f556809ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513cc6d31550cbeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f555aff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556809f500 con 0x556e9632d0c0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f557000f040 con 0x556e9632d0c0 2023-10-06T21:32:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f556809b3b0 con 0x556e9632d0c0 2023-10-06T21:32:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.883+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55680a36b0 con 0x556e9632d0c0 2023-10-06T21:32:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 10 client.17430 fetch_fsmap learned FSMap version 67 2023-10-06T21:32:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x556e960abde0 con 0x556e9632d0c0 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55680a7020 con 0x556e9632d0c0 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55680a12b0 con 0x556e9632d0c0 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 10 client.17430 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 20 client.17430 populate_metadata read hostname 'smithi026' 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e9639ce60 0x556e9639f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 4 client.17430 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556e96279b30 con 0x556e9639ce60 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.884+0000 7f55767fc700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e9639ce60 0x556e9639f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55767fc700 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e9639ce60 0x556e9639f280 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd3b290c9944dbc server_cookie=49832799dd2e9285 in_seq=0 out_seq=0 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f555aff5700 10 client.17430 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f555aff5700 1 -- 172.21.15.26:0/1491642775 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556e96279b30 con 0x556e9639ce60 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f555aff5700 10 client.17430 handle_command_reply: tid=0 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 client.17430 shutdown 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 msgr2=0x556e9633f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f55680978b0 tx=0x7f556809ac40 comp rx=0 tx=0).stop 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 shutdown_connections 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556e9639ce60 0x556e9639f280 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e96338890 0x556e9633d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556e9632d0c0 0x556e9633f100 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 --2- 172.21.15.26:0/1491642775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556e9632d9e0 0x556e9633d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 >> 172.21.15.26:0/1491642775 conn(0x556e962a2260 msgr2=0x556e96337760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f555aff5700 0 client.17430 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 shutdown_connections 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.885+0000 7f55911ecb80 1 -- 172.21.15.26:0/1491642775 wait complete. 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f55911ecb80 20 client.17430 trim_cache size 0 max 16384 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:32:27.893 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:32:27.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:32:27.894 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:32:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 msgr2=0x7f55600a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 0x7f55600a3b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f557000f540 tx=0x7f5570009920 comp rx=0 tx=0).stop 2023-10-06T21:32:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 msgr2=0x7f5578199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568005740 comp rx=0 tx=0).stop 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 shutdown_connections 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f55600a1670 0x7f55600a3b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f557810b410 0x7f557819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5578101ff0 0x7f5578199260 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 --2- 172.21.15.26:0/1163144946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5578102910 0x7f55781997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 >> 172.21.15.26:0/1163144946 conn(0x7f55780fb830 msgr2=0x7f557810d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.886+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 shutdown_connections 2023-10-06T21:32:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:27.887+0000 7f557d7ff700 1 -- 172.21.15.26:0/1163144946 wait complete. 2023-10-06T21:32:27.909 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:32:29.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:28 smithi167 ceph-mon[134494]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:28 smithi026 ceph-mon[98490]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:28 smithi078 ceph-mon[134896]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:30 smithi026 ceph-mon[98490]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:30 smithi078 ceph-mon[134896]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:31.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:30 smithi167 ceph-mon[134494]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:32 smithi026 ceph-mon[98490]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:32 smithi078 ceph-mon[134896]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:32 smithi167 ceph-mon[134494]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:34 smithi026 ceph-mon[98490]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:34 smithi078 ceph-mon[134896]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:35.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:34 smithi167 ceph-mon[134494]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:37.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:36 smithi167 ceph-mon[134494]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:36 smithi026 ceph-mon[98490]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:36 smithi078 ceph-mon[134896]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:38 smithi026 ceph-mon[98490]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:38 smithi078 ceph-mon[134896]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:38 smithi167 ceph-mon[134494]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:40 smithi026 ceph-mon[98490]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:40 smithi078 ceph-mon[134896]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:41.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:40 smithi167 ceph-mon[134494]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:42 smithi026 ceph-mon[98490]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:42 smithi078 ceph-mon[134896]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:42 smithi167 ceph-mon[134494]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:44 smithi026 ceph-mon[98490]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:44 smithi078 ceph-mon[134896]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:45.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:44 smithi167 ceph-mon[134494]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:46 smithi026 ceph-mon[98490]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:47.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:46 smithi078 ceph-mon[134896]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:47.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:46 smithi167 ceph-mon[134494]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:48 smithi026 ceph-mon[98490]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:48 smithi078 ceph-mon[134896]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:49.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:48 smithi167 ceph-mon[134494]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:50 smithi026 ceph-mon[98490]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:50 smithi078 ceph-mon[134896]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:51.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:50 smithi167 ceph-mon[134494]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:52 smithi026 ceph-mon[98490]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:53.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:52 smithi078 ceph-mon[134896]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:53.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:52 smithi167 ceph-mon[134494]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:54 smithi026 ceph-mon[98490]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:54 smithi078 ceph-mon[134896]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:55.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:54 smithi167 ceph-mon[134494]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:32:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:56 smithi026 ceph-mon[98490]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:56 smithi078 ceph-mon[134896]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:56 smithi167 ceph-mon[134494]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:32:57.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:32:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 -- 172.21.15.26:0/396437636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8801027c0 msgr2=0x7fe88010aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 --2- 172.21.15.26:0/396437636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8801027c0 0x7fe88010aca0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe870009990 tx=0x7fe870009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 -- 172.21.15.26:0/396437636 shutdown_connections 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 --2- 172.21.15.26:0/396437636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88010b2c0 0x7fe88010d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 --2- 172.21.15.26:0/396437636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 --2- 172.21.15.26:0/396437636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8801027c0 0x7fe88010aca0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 -- 172.21.15.26:0/396437636 >> 172.21.15.26:0/396437636 conn(0x7fe8800fb6e0 msgr2=0x7fe8800fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.140+0000 7fe88719d700 1 -- 172.21.15.26:0/396437636 shutdown_connections 2023-10-06T21:32:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.141+0000 7fe88719d700 1 -- 172.21.15.26:0/396437636 wait complete. 2023-10-06T21:32:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.142+0000 7fe88719d700 1 Processor -- start 2023-10-06T21:32:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.142+0000 7fe88719d700 1 -- start start 2023-10-06T21:32:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8801027c0 0x7fe880199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 0x7fe88019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019df00 con 0x7fe88010b2c0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019e070 con 0x7fe880101ea0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88719d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88019e230 con 0x7fe8801027c0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51692/0 (socket says 172.21.15.26:51692) 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 -- 172.21.15.26:0/4132643264 learned_addr learned my addr 172.21.15.26:0/4132643264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88599a700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8801027c0 0x7fe880199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88699c700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 0x7fe88019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 -- 172.21.15.26:0/4132643264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8801027c0 msgr2=0x7fe880199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8801027c0 0x7fe880199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 -- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 msgr2=0x7fe88019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 0x7fe88019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88619b700 1 -- 172.21.15.26:0/4132643264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c009520 con 0x7fe880101ea0 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.143+0000 7fe88699c700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 0x7fe88019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe88619b700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00eb30 tx=0x7fe87c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe8777fe700 1 -- 172.21.15.26:0/4132643264 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c009880 con 0x7fe880101ea0 2023-10-06T21:32:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8700095f0 con 0x7fe880101ea0 2023-10-06T21:32:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe8777fe700 1 -- 172.21.15.26:0/4132643264 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c0044e0 con 0x7fe880101ea0 2023-10-06T21:32:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe8777fe700 1 -- 172.21.15.26:0/4132643264 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c003e90 con 0x7fe880101ea0 2023-10-06T21:32:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.144+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8801a5b70 con 0x7fe880101ea0 2023-10-06T21:32:58.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.148+0000 7fe8777fe700 1 -- 172.21.15.26:0/4132643264 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fe87c004000 con 0x7fe880101ea0 2023-10-06T21:32:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe8777fe700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 0x7fe86c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 Processor -- start 2023-10-06T21:32:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe8777fe700 1 -- 172.21.15.26:0/4132643264 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe87c014070 con 0x7fe880101ea0 2023-10-06T21:32:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 -- start start 2023-10-06T21:32:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe88599a700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 0x7fe86c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d916530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d918e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d874460 con 0x55a37d90b2a0 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d865c20 con 0x55a37d90bbc0 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.150+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d863540 con 0x55a37d916a70 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48454/0 (socket says 172.21.15.26:48454) 2023-10-06T21:32:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 -- 172.21.15.26:0/2976521853 learned_addr learned my addr 172.21.15.26:0/2976521853 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88599a700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 0x7fe86c0a3b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe88019a530 tx=0x7fe870005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88599a700 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d916530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88699c700 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d918e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 -- 172.21.15.26:0/2976521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 msgr2=0x55a37d918e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d918e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 -- 172.21.15.26:0/2976521853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 msgr2=0x55a37d916530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d916530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 -- 172.21.15.26:0/2976521853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37d689de0 con 0x55a37d90b2a0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.151+0000 7fe88619b700 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0a23f0 tx=0x7fe87c09a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f3669e3bdf2cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.152+0000 7fe874bfe700 1 -- 172.21.15.26:0/2976521853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c016330 con 0x55a37d90b2a0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.152+0000 7fe874bfe700 1 -- 172.21.15.26:0/2976521853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c09ac90 con 0x55a37d90b2a0 2023-10-06T21:32:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.152+0000 7fe874bfe700 1 -- 172.21.15.26:0/2976521853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c096ca0 con 0x55a37d90b2a0 2023-10-06T21:32:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.152+0000 7fe89bd91b80 1 -- 172.21.15.26:0/2976521853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 msgr2=0x55a37d90b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.152+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0a23f0 tx=0x7fe87c09a360 comp rx=0 tx=0).stop 2023-10-06T21:32:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 -- 172.21.15.26:0/2976521853 shutdown_connections 2023-10-06T21:32:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d918e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d90b680 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/2976521853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d916530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 -- 172.21.15.26:0/2976521853 >> 172.21.15.26:0/2976521853 conn(0x55a37d880260 msgr2=0x55a37d907630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 -- 172.21.15.26:0/2976521853 shutdown_connections 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.153+0000 7fe89bd91b80 1 -- 172.21.15.26:0/2976521853 wait complete. 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.154+0000 7fe89bd91b80 1 -- start start 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.154+0000 7fe89bd91b80 1 Processor -- start 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d913880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90bbc0 0x55a37d913dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d85ee30 con 0x55a37d90b2a0 2023-10-06T21:32:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d7fb710 con 0x55a37d916a70 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d7fb3b0 con 0x55a37d90bbc0 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe89bd91b80 10 client.0 _ll_register_callbacks cb 0x55a37d658800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d913880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88699c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88699c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51712/0 (socket says 172.21.15.26:51712) 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88699c700 1 -- 172.21.15.26:0/3411689890 learned_addr learned my addr 172.21.15.26:0/3411689890 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88619b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d913880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48466/0 (socket says 172.21.15.26:48466) 2023-10-06T21:32:58.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.155+0000 7fe88599a700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90bbc0 0x55a37d913dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 -- 172.21.15.26:0/3411689890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90bbc0 msgr2=0x55a37d913dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90bbc0 0x55a37d913dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 -- 172.21.15.26:0/3411689890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 msgr2=0x55a37d913880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d913880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 -- 172.21.15.26:0/3411689890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8801a5b70 con 0x55a37d916a70 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe88699c700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe878007b30 tx=0x7fe878007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9affc0b5b133fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87800e070 con 0x55a37d916a70 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a37d8ff160 con 0x55a37d916a70 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe878012b80 con 0x55a37d916a70 2023-10-06T21:32:58.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe878012d40 con 0x55a37d916a70 2023-10-06T21:32:58.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.156+0000 7fe89bd91b80 10 client.27824 fetch_fsmap learned FSMap version 67 2023-10-06T21:32:58.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55a37d672c60 con 0x55a37d916a70 2023-10-06T21:32:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a37d8ff160 con 0x55a37d916a70 2023-10-06T21:32:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe878014040 con 0x55a37d916a70 2023-10-06T21:32:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 10 client.27824 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:32:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 10 client.27824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:58.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 20 client.27824 populate_metadata read hostname 'smithi026' 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d926430 0x55a37d928850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 4 client.27824 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a37d857e10 con 0x55a37d926430 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.157+0000 7fe88619b700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d926430 0x55a37d928850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.158+0000 7fe88619b700 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d926430 0x55a37d928850 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76c10067173526d server_cookie=76d93db58561791e in_seq=0 out_seq=0 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.158+0000 7fe869ffb700 10 client.27824 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.158+0000 7fe869ffb700 1 -- 172.21.15.26:0/3411689890 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a37d857e10 con 0x55a37d926430 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.158+0000 7fe869ffb700 10 client.27824 handle_command_reply: tid=0 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 client.27824 shutdown 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 msgr2=0x55a37d914300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe878007b30 tx=0x7fe878007e40 comp rx=0 tx=0).stop 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 shutdown_connections 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d926430 0x55a37d928850 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d916a70 0x55a37d914300 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90b2a0 0x55a37d913880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe869ffb700 0 client.27824 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/3411689890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90bbc0 0x55a37d913dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 >> 172.21.15.26:0/3411689890 conn(0x55a37d880260 msgr2=0x55a37d907220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 shutdown_connections 2023-10-06T21:32:58.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 1 -- 172.21.15.26:0/3411689890 wait complete. 2023-10-06T21:32:58.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.159+0000 7fe89bd91b80 20 client.27824 trim_cache size 0 max 16384 2023-10-06T21:32:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.178+0000 7fe89bd91b80 1 Processor -- start 2023-10-06T21:32:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.178+0000 7fe89bd91b80 1 -- start start 2023-10-06T21:32:58.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.178+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.178+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90bbc0 0x55a37d91d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.178+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d914220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d6df920 con 0x55a37d90bbc0 2023-10-06T21:32:58.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d903c70 con 0x55a37d90b2a0 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d9084d0 con 0x55a37d916a70 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51718/0 (socket says 172.21.15.26:51718) 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88599a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90bbc0 0x55a37d91d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 -- 172.21.15.26:0/587874110 learned_addr learned my addr 172.21.15.26:0/587874110 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88699c700 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d914220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 -- 172.21.15.26:0/587874110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 msgr2=0x55a37d914220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d914220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 -- 172.21.15.26:0/587874110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90bbc0 msgr2=0x55a37d91d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90bbc0 0x55a37d91d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 -- 172.21.15.26:0/587874110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37d6b0250 con 0x55a37d90b2a0 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.179+0000 7fe88619b700 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009230 tx=0x7fe87c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68734494ad6b83cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe8753ff700 1 -- 172.21.15.26:0/587874110 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c09e6e0 con 0x55a37d90b2a0 2023-10-06T21:32:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe8753ff700 1 -- 172.21.15.26:0/587874110 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c00cc90 con 0x55a37d90b2a0 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe8753ff700 1 -- 172.21.15.26:0/587874110 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c09a7f0 con 0x55a37d90b2a0 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 -- 172.21.15.26:0/587874110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 msgr2=0x55a37d90e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009230 tx=0x7fe87c004750 comp rx=0 tx=0).stop 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 -- 172.21.15.26:0/587874110 shutdown_connections 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d916a70 0x55a37d914220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90b2a0 0x55a37d90e310 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/587874110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d90bbc0 0x55a37d91d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 -- 172.21.15.26:0/587874110 >> 172.21.15.26:0/587874110 conn(0x55a37d880260 msgr2=0x55a37d90ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 -- 172.21.15.26:0/587874110 shutdown_connections 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.180+0000 7fe89bd91b80 1 -- 172.21.15.26:0/587874110 wait complete. 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.181+0000 7fe89bd91b80 1 -- start start 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.181+0000 7fe89bd91b80 1 Processor -- start 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90b2a0 0x55a37d91af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 0x55a37d979260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d91bca0 con 0x55a37d916a70 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d91be10 con 0x55a37d90bbc0 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37d979820 con 0x55a37d90b2a0 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe89bd91b80 10 client.0 _ll_register_callbacks cb 0x7fe878007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51728/0 (socket says 172.21.15.26:51728) 2023-10-06T21:32:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 -- 172.21.15.26:0/1686757698 learned_addr learned my addr 172.21.15.26:0/1686757698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:32:58.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88619b700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90b2a0 0x55a37d91af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88699c700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 0x55a37d979260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 -- 172.21.15.26:0/1686757698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90b2a0 msgr2=0x55a37d91af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90b2a0 0x55a37d91af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 -- 172.21.15.26:0/1686757698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 msgr2=0x55a37d979260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 0x55a37d979260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88599a700 1 -- 172.21.15.26:0/1686757698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a37d689de0 con 0x55a37d90bbc0 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.182+0000 7fe88699c700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 0x55a37d979260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe88599a700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe870009990 tx=0x7fe870009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155b97810b0125d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:32:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870024870 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe8700249d0 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe87c09ac90 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe870024b90 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe89bd91b80 10 client.27830 fetch_fsmap learned FSMap version 67 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55a37d6882b0 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.183+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87c09ac90 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe870018400 con 0x55a37d90bbc0 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 10 client.27830 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 20 client.27830 populate_metadata read hostname 'smithi026' 2023-10-06T21:32:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d97af20 0x55a37d97d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 4 client.27830 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a37d857b30 con 0x55a37d97af20 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.184+0000 7fe88619b700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d97af20 0x55a37d97d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe88619b700 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d97af20 0x55a37d97d340 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a7ed4792f6e46d server_cookie=2bf26a55407cebf2 in_seq=0 out_seq=0 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe869ffb700 10 client.27830 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe869ffb700 1 -- 172.21.15.26:0/1686757698 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a37d857b30 con 0x55a37d97af20 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe869ffb700 10 client.27830 handle_command_reply: tid=0 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 client.27830 shutdown 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 msgr2=0x55a37d91b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe870009990 tx=0x7fe870009ca0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 shutdown_connections 2023-10-06T21:32:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a37d916a70 0x55a37d979260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a37d90b2a0 0x55a37d91af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55a37d97af20 0x55a37d97d340 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 --2- 172.21.15.26:0/1686757698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a37d90bbc0 0x55a37d91b4a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 >> 172.21.15.26:0/1686757698 conn(0x55a37d880260 msgr2=0x55a37d914170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe869ffb700 0 client.27830 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 shutdown_connections 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.185+0000 7fe89bd91b80 1 -- 172.21.15.26:0/1686757698 wait complete. 2023-10-06T21:32:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe89bd91b80 20 client.27830 trim_cache size 0 max 16384 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:32:58.203 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:32:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 msgr2=0x7fe86c0a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 0x7fe86c0a3b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe88019a530 tx=0x7fe870005fd0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 msgr2=0x7fe880198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:32:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00eb30 tx=0x7fe87c00ee40 comp rx=0 tx=0).stop 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 shutdown_connections 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fe86c0a1690 0x7fe86c0a3b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88010b2c0 0x7fe88019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe880101ea0 0x7fe880198f20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 --2- 172.21.15.26:0/4132643264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8801027c0 0x7fe880199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 >> 172.21.15.26:0/4132643264 conn(0x7fe8800fb6e0 msgr2=0x7fe8800ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 shutdown_connections 2023-10-06T21:32:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:32:58.186+0000 7fe88719d700 1 -- 172.21.15.26:0/4132643264 wait complete. 2023-10-06T21:32:58.215 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:32:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:32:58 smithi026 ceph-mon[98490]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:32:58 smithi078 ceph-mon[134896]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:32:59.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:32:58 smithi167 ceph-mon[134494]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:00 smithi026 ceph-mon[98490]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:00 smithi078 ceph-mon[134896]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:01.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:00 smithi167 ceph-mon[134494]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:02 smithi026 ceph-mon[98490]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:33:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:02 smithi078 ceph-mon[134896]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:33:03.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:02 smithi167 ceph-mon[134494]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2023-10-06T21:33:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:04 smithi026 ceph-mon[98490]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:33:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:04 smithi078 ceph-mon[134896]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:33:05.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:04 smithi167 ceph-mon[134494]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:33:06.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:06 smithi026 ceph-mon[98490]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:33:06.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:06 smithi026 ceph-mon[98490]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:06.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:06 smithi078 ceph-mon[134896]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:33:06.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:06 smithi078 ceph-mon[134896]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:06 smithi167 ceph-mon[134494]: osdmap e122: 12 total, 12 up, 12 in 2023-10-06T21:33:07.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:06 smithi167 ceph-mon[134494]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:08 smithi026 ceph-mon[98490]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:08 smithi078 ceph-mon[134896]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:08 smithi167 ceph-mon[134494]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:10 smithi026 ceph-mon[98490]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:10 smithi078 ceph-mon[134896]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:11.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:10 smithi167 ceph-mon[134494]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-10-06T21:33:13.078 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:12 smithi026 ceph-mon[98490]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:33:13.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:12 smithi078 ceph-mon[134896]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:33:13.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:12 smithi167 ceph-mon[134494]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-10-06T21:33:14.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:13 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:14.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:13 smithi026 ceph-mon[98490]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:33:14.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:13 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:13 smithi078 ceph-mon[134896]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:33:14.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:13 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:33:14.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:13 smithi167 ceph-mon[134494]: osdmap e123: 12 total, 12 up, 12 in 2023-10-06T21:33:15.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:33:15.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 125 B/s wr, 6 op/s 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:16.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:15 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:16.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:15 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:16.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:15 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:16.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:15 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:16.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:15 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:16.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:15 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:16.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:15 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:33:16.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:15 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:33:16.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:15 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:33:17.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:17 smithi167 ceph-mon[134494]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:17.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:17 smithi026 ceph-mon[98490]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:17.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:17 smithi078 ceph-mon[134896]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:18.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:18 smithi167 ceph-mon[134494]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:18.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:18 smithi026 ceph-mon[98490]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:18.620 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:18 smithi078 ceph-mon[134896]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:20 smithi026 ceph-mon[98490]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:20 smithi078 ceph-mon[134896]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:21.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:20 smithi167 ceph-mon[134494]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:33:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:22 smithi026 ceph-mon[98490]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:33:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:22 smithi078 ceph-mon[134896]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:33:23.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:22 smithi167 ceph-mon[134494]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:33:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:24 smithi026 ceph-mon[98490]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:33:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:24 smithi078 ceph-mon[134896]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:33:25.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:24 smithi167 ceph-mon[134494]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2023-10-06T21:33:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:26 smithi026 ceph-mon[98490]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:33:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:26 smithi078 ceph-mon[134896]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:33:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:26 smithi167 ceph-mon[134494]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-10-06T21:33:28.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:33:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 -- 172.21.15.26:0/3697132988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c1010e0 msgr2=0x7f912c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 --2- 172.21.15.26:0/3697132988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c1010e0 0x7f912c1014c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f911c009990 tx=0x7f911c009ca0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 -- 172.21.15.26:0/3697132988 shutdown_connections 2023-10-06T21:33:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 --2- 172.21.15.26:0/3697132988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c106200 0x7f912c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 --2- 172.21.15.26:0/3697132988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c1010e0 0x7f912c1014c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 --2- 172.21.15.26:0/3697132988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c101a90 0x7f912c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 -- 172.21.15.26:0/3697132988 >> 172.21.15.26:0/3697132988 conn(0x7f912c0fc960 msgr2=0x7f912c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 -- 172.21.15.26:0/3697132988 shutdown_connections 2023-10-06T21:33:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.433+0000 7f91323ff700 1 -- 172.21.15.26:0/3697132988 wait complete. 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.434+0000 7f91323ff700 1 Processor -- start 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.434+0000 7f91323ff700 1 -- start start 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c1010e0 0x7f912c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c19db20 0x7f912c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19e5a0 con 0x7f912c1010e0 2023-10-06T21:33:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19e710 con 0x7f912c106200 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f91323ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19e8d0 con 0x7f912c19db20 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38178/0 (socket says 172.21.15.26:38178) 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f912b7fe700 1 -- 172.21.15.26:0/2643744859 learned_addr learned my addr 172.21.15.26:0/2643744859 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f912bfff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c1010e0 0x7f912c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.435+0000 7f913099c700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c19db20 0x7f912c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 -- 172.21.15.26:0/2643744859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c19db20 msgr2=0x7f912c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c19db20 0x7f912c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 -- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c1010e0 msgr2=0x7f912c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c1010e0 0x7f912c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 -- 172.21.15.26:0/2643744859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0095f0 con 0x7f912c106200 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f912b7fe700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f912000e970 tx=0x7f912000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f91297fa700 1 -- 172.21.15.26:0/2643744859 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120009790 con 0x7f912c106200 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f91297fa700 1 -- 172.21.15.26:0/2643744859 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f912000c7e0 con 0x7f912c106200 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f91297fa700 1 -- 172.21.15.26:0/2643744859 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91200054e0 con 0x7f912c106200 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c1a5e30 con 0x7f912c106200 2023-10-06T21:33:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.436+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c1a6250 con 0x7f912c106200 2023-10-06T21:33:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.440+0000 7f91297fa700 1 -- 172.21.15.26:0/2643744859 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f912000c950 con 0x7f912c106200 2023-10-06T21:33:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.441+0000 7f9145c7ab80 1 Processor -- start 2023-10-06T21:33:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 -- start start 2023-10-06T21:33:28.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f91297fa700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 0x7f91140a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f91297fa700 1 -- 172.21.15.26:0/2643744859 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9120014070 con 0x7f912c106200 2023-10-06T21:33:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff0ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fe66460 con 0x55963ff08890 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fe57c20 con 0x55963fefd9e0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fe55540 con 0x55963fefd0c0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f913099c700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 0x7f91140a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38180/0 (socket says 172.21.15.26:38180) 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.442+0000 7f912b7fe700 1 -- 172.21.15.26:0/3718129089 learned_addr learned my addr 172.21.15.26:0/3718129089 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f913099c700 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff0ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912bfff700 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 -- 172.21.15.26:0/3718129089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 msgr2=0x55963fefd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 -- 172.21.15.26:0/3718129089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 msgr2=0x55963ff0ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff0ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 -- 172.21.15.26:0/3718129089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963fc7bde0 con 0x55963fefd9e0 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f912b7fe700 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9120096090 tx=0x7f912000cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1ddc0d743d32e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f913099c700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 0x7f91140a3ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9124009ec0 tx=0x7f91240092f0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f91127fc700 1 -- 172.21.15.26:0/3718129089 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912009e460 con 0x55963fefd9e0 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f91127fc700 1 -- 172.21.15.26:0/3718129089 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91200a26a0 con 0x55963fefd9e0 2023-10-06T21:33:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.443+0000 7f91127fc700 1 -- 172.21.15.26:0/3718129089 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912009a570 con 0x55963fefd9e0 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 -- 172.21.15.26:0/3718129089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 msgr2=0x55963ff08350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9120096090 tx=0x7f912000cc50 comp rx=0 tx=0).stop 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 -- 172.21.15.26:0/3718129089 shutdown_connections 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff0ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/3718129089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff08350 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 -- 172.21.15.26:0/3718129089 >> 172.21.15.26:0/3718129089 conn(0x55963fe72260 msgr2=0x55963fef9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.444+0000 7f9145c7ab80 1 -- 172.21.15.26:0/3718129089 shutdown_connections 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.445+0000 7f9145c7ab80 1 -- 172.21.15.26:0/3718129089 wait complete. 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.445+0000 7f9145c7ab80 1 -- start start 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.445+0000 7f9145c7ab80 1 Processor -- start 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963ff05640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fe50e30 con 0x55963ff08890 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fded710 con 0x55963fefd9e0 2023-10-06T21:33:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fded3b0 con 0x55963fefd0c0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f9145c7ab80 10 client.0 _ll_register_callbacks cb 0x55963fc4a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f913099c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f912b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38196/0 (socket says 172.21.15.26:38196) 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f912b7fe700 1 -- 172.21.15.26:0/1352086076 learned_addr learned my addr 172.21.15.26:0/1352086076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.446+0000 7f912bfff700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963ff05640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 -- 172.21.15.26:0/1352086076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 msgr2=0x55963ff05640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963ff05640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 -- 172.21.15.26:0/1352086076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 msgr2=0x55963ff060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 -- 172.21.15.26:0/1352086076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c1a6250 con 0x55963fefd9e0 2023-10-06T21:33:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912bfff700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963ff05640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f91200cd310 tx=0x7f91200047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c480dd98beaac08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f910b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912009e170 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9120063020 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55963fef0f10 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9120099800 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f9145c7ab80 10 client.27842 fetch_fsmap learned FSMap version 67 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.447+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55963fc64c60 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9120081020 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91200a2210 con 0x55963fefd9e0 2023-10-06T21:33:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 10 client.27842 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 10 client.27842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 20 client.27842 populate_metadata read hostname 'smithi026' 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff18130 0x55963ff1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 4 client.27842 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55963fe49e10 con 0x55963ff18130 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.448+0000 7f912bfff700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff18130 0x55963ff1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.449+0000 7f912bfff700 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff18130 0x55963ff1a550 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca8bb108081e018 server_cookie=449cca9090fbcc3f in_seq=0 out_seq=0 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.449+0000 7f910b7fe700 10 client.27842 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.449+0000 7f910b7fe700 1 -- 172.21.15.26:0/1352086076 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55963fe49e10 con 0x55963ff18130 2023-10-06T21:33:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.449+0000 7f910b7fe700 10 client.27842 handle_command_reply: tid=0 2023-10-06T21:33:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.449+0000 7f9145c7ab80 1 client.27842 shutdown 2023-10-06T21:33:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 msgr2=0x55963ff05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f91200cd310 tx=0x7f91200047c0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 shutdown_connections 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963ff08890 0x55963ff060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963ff05640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff18130 0x55963ff1a550 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1352086076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd9e0 0x55963ff05b80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f910b7fe700 0 client.27842 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 >> 172.21.15.26:0/1352086076 conn(0x55963fe72260 msgr2=0x55963fef8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 shutdown_connections 2023-10-06T21:33:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1352086076 wait complete. 2023-10-06T21:33:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.450+0000 7f9145c7ab80 20 client.27842 trim_cache size 0 max 16384 2023-10-06T21:33:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 Processor -- start 2023-10-06T21:33:28.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 -- start start 2023-10-06T21:33:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff05ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fcd1920 con 0x55963fefd9e0 2023-10-06T21:33:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963ff014d0 con 0x55963ff08890 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.468+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963fefe020 con 0x55963fefd0c0 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38212/0 (socket says 172.21.15.26:38212) 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 -- 172.21.15.26:0/1674478130 learned_addr learned my addr 172.21.15.26:0/1674478130 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff05ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f912bfff700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 -- 172.21.15.26:0/1674478130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 msgr2=0x55963fefff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 -- 172.21.15.26:0/1674478130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 msgr2=0x55963ff05ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff05ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f913099c700 1 -- 172.21.15.26:0/1674478130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963fef5140 con 0x55963ff08890 2023-10-06T21:33:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.469+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff05ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f913099c700 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f912401a9e0 tx=0x7f912401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bfda536430eb31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9112ffd700 1 -- 172.21.15.26:0/1674478130 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912400e070 con 0x55963ff08890 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9112ffd700 1 -- 172.21.15.26:0/1674478130 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f912400f040 con 0x55963ff08890 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9112ffd700 1 -- 172.21.15.26:0/1674478130 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124028ad0 con 0x55963ff08890 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1674478130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 msgr2=0x55963ff042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f912401a9e0 tx=0x7f912401d000 comp rx=0 tx=0).stop 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1674478130 shutdown_connections 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963ff08890 0x55963ff042d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963fefd0c0 0x55963fefff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.470+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1674478130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff05ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.471+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1674478130 >> 172.21.15.26:0/1674478130 conn(0x55963fe72260 msgr2=0x55963ff03bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.471+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1674478130 shutdown_connections 2023-10-06T21:33:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.471+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1674478130 wait complete. 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.471+0000 7f9145c7ab80 1 -- start start 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.471+0000 7f9145c7ab80 1 Processor -- start 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff0d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963ff08890 0x55963ff0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963ff0e310 con 0x55963fefd9e0 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963ff0e480 con 0x55963fefd0c0 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963ff6bf50 con 0x55963ff08890 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f9145c7ab80 10 client.0 _ll_register_callbacks cb 0x7f9124008750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38216/0 (socket says 172.21.15.26:38216) 2023-10-06T21:33:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 -- 172.21.15.26:0/1498566691 learned_addr learned my addr 172.21.15.26:0/1498566691 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff0d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f913099c700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963ff08890 0x55963ff0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 -- 172.21.15.26:0/1498566691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963ff08890 msgr2=0x55963ff0dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963ff08890 0x55963ff0dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 -- 172.21.15.26:0/1498566691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 msgr2=0x55963ff0d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff0d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.472+0000 7f912bfff700 1 -- 172.21.15.26:0/1498566691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55963fc64c60 con 0x55963fefd0c0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f912bfff700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f911c0098e0 tx=0x7f911c00bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ec6028d98b270d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f910b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c01d040 con 0x55963fefd0c0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f911c028350 con 0x55963fefd0c0 2023-10-06T21:33:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91200a26a0 con 0x55963fefd0c0 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f911c028510 con 0x55963fefd0c0 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f9145c7ab80 10 client.27848 fetch_fsmap learned FSMap version 67 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7f9124008570 con 0x55963fefd0c0 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.473+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91200a26a0 con 0x55963fefd0c0 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f911c0049f0 con 0x55963fefd0c0 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 10 client.27848 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 10 client.27848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 20 client.27848 populate_metadata read hostname 'smithi026' 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff6d620 0x55963ff6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 4 client.27848 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55963fe49b30 con 0x55963ff6d620 2023-10-06T21:33:28.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff6d620 0x55963ff6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f912b7fe700 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff6d620 0x55963ff6fa40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1734814c9385848 server_cookie=b393b0e7fc3aa074 in_seq=0 out_seq=0 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.474+0000 7f910b7fe700 10 client.27848 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f910b7fe700 1 -- 172.21.15.26:0/1498566691 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55963fe49b30 con 0x55963ff6d620 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f910b7fe700 10 client.27848 handle_command_reply: tid=0 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 client.27848 shutdown 2023-10-06T21:33:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 msgr2=0x55963ff0f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f911c0098e0 tx=0x7f911c00bc20 comp rx=0 tx=0).stop 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 shutdown_connections 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55963ff08890 0x55963ff0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55963ff6d620 0x55963ff6fa40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55963fefd0c0 0x55963ff0f420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 --2- 172.21.15.26:0/1498566691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55963fefd9e0 0x55963ff0d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f910b7fe700 0 client.27848 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 >> 172.21.15.26:0/1498566691 conn(0x55963fe72260 msgr2=0x55963ff077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 shutdown_connections 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 1 -- 172.21.15.26:0/1498566691 wait complete. 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.475+0000 7f9145c7ab80 20 client.27848 trim_cache size 0 max 16384 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:33:28.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 msgr2=0x7f91140a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 0x7f91140a3ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9124009ec0 tx=0x7f91240092f0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 msgr2=0x7f912c1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f912000e970 tx=0x7f912000ec80 comp rx=0 tx=0).stop 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 shutdown_connections 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f912c19db20 0x7f912c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f91140a15f0 0x7f91140a3ab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f912c1010e0 0x7f912c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 --2- 172.21.15.26:0/2643744859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f912c106200 0x7f912c1996c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 >> 172.21.15.26:0/2643744859 conn(0x7f912c0fc960 msgr2=0x7f912c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 shutdown_connections 2023-10-06T21:33:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:28.476+0000 7f91323ff700 1 -- 172.21.15.26:0/2643744859 wait complete. 2023-10-06T21:33:28.498 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:33:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:28 smithi026 ceph-mon[98490]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:28 smithi078 ceph-mon[134896]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:29.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:28 smithi167 ceph-mon[134494]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:30 smithi026 ceph-mon[98490]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:30 smithi078 ceph-mon[134896]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:30 smithi167 ceph-mon[134494]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:32 smithi026 ceph-mon[98490]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:32 smithi078 ceph-mon[134896]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:33.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:32 smithi167 ceph-mon[134494]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:34 smithi026 ceph-mon[98490]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:34 smithi078 ceph-mon[134896]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:35.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:34 smithi167 ceph-mon[134494]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:36 smithi026 ceph-mon[98490]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:36 smithi078 ceph-mon[134896]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:37.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:36 smithi167 ceph-mon[134494]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:38 smithi026 ceph-mon[98490]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:38 smithi078 ceph-mon[134896]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:38 smithi167 ceph-mon[134494]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:41.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:40 smithi026 ceph-mon[98490]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:40 smithi078 ceph-mon[134896]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:41.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:40 smithi167 ceph-mon[134494]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:42 smithi026 ceph-mon[98490]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:42 smithi078 ceph-mon[134896]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:42 smithi167 ceph-mon[134494]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:44 smithi026 ceph-mon[98490]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:44 smithi078 ceph-mon[134896]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:44 smithi167 ceph-mon[134494]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:46 smithi026 ceph-mon[98490]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:47.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:46 smithi078 ceph-mon[134896]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:47.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:46 smithi167 ceph-mon[134494]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:48 smithi026 ceph-mon[98490]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:48 smithi078 ceph-mon[134896]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:49.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:48 smithi167 ceph-mon[134494]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:50 smithi026 ceph-mon[98490]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:51.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:50 smithi078 ceph-mon[134896]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:51.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:50 smithi167 ceph-mon[134494]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:52 smithi026 ceph-mon[98490]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:52 smithi078 ceph-mon[134896]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:53.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:52 smithi167 ceph-mon[134494]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:54.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:54 smithi026 ceph-mon[98490]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:54.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:54 smithi078 ceph-mon[134896]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:55.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:54 smithi167 ceph-mon[134494]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:33:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:56 smithi026 ceph-mon[98490]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:56 smithi078 ceph-mon[134896]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:56 smithi167 ceph-mon[134494]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:33:58.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.725+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/1719790592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df8108420 msgr2=0x7f7df810ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.725+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/1719790592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df8108420 0x7f7df810ffa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7df000b0a0 tx=0x7f7df000b3b0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/1719790592 shutdown_connections 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/1719790592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df8108420 0x7f7df810ffa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/1719790592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 0x7f7df81036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/1719790592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8103cb0 0x7f7df8107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/1719790592 >> 172.21.15.26:0/1719790592 conn(0x7f7df80feb40 msgr2=0x7f7df8100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/1719790592 shutdown_connections 2023-10-06T21:33:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.726+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/1719790592 wait complete. 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 Processor -- start 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 -- start start 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 0x7f7df81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 0x7f7df819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df819e4f0 con 0x7f7df8108420 2023-10-06T21:33:58.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df819e660 con 0x7f7df819da70 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.727+0000 7f7dfd9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df819e820 con 0x7f7df8103300 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48574/0 (socket says 172.21.15.26:48574) 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 -- 172.21.15.26:0/186637114 learned_addr learned my addr 172.21.15.26:0/186637114 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df77fe700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 0x7f7df819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 0x7f7df81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 -- 172.21.15.26:0/186637114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 msgr2=0x7f7df81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 0x7f7df81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 -- 172.21.15.26:0/186637114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 msgr2=0x7f7df819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 0x7f7df819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 -- 172.21.15.26:0/186637114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0009d00 con 0x7f7df8108420 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df77fe700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 0x7f7df819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.728+0000 7f7df67fc700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e970 tx=0x7f7dec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.729+0000 7f7dfc9fd700 1 -- 172.21.15.26:0/186637114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec009790 con 0x7f7df8108420 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.729+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df81a5d80 con 0x7f7df8108420 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.729+0000 7f7dfc9fd700 1 -- 172.21.15.26:0/186637114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dec00c7e0 con 0x7f7df8108420 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.729+0000 7f7dfc9fd700 1 -- 172.21.15.26:0/186637114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0054e0 con 0x7f7df8108420 2023-10-06T21:33:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.729+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df81a6200 con 0x7f7df8108420 2023-10-06T21:33:58.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.733+0000 7f7dfc9fd700 1 -- 172.21.15.26:0/186637114 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7dec00c950 con 0x7f7df8108420 2023-10-06T21:33:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.734+0000 7f7dfc9fd700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 0x7f7de00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.734+0000 7f7dfc9fd700 1 -- 172.21.15.26:0/186637114 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7dec014070 con 0x7f7df8108420 2023-10-06T21:33:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.734+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 0x7f7de00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.735+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 0x7f7de00a3ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009c60 tx=0x7f7de8009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.735+0000 7f7e11385b80 1 Processor -- start 2023-10-06T21:33:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 -- start start 2023-10-06T21:33:58.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf6073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5fcf460 con 0x556cf6071890 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5fc0c20 con 0x556cf60669e0 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7e11385b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5fbe540 con 0x556cf60660c0 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52190/0 (socket says 172.21.15.26:52190) 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7df67fc700 1 -- 172.21.15.26:0/3856496239 learned_addr learned my addr 172.21.15.26:0/3856496239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.736+0000 7f7df77fe700 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf6073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 -- 172.21.15.26:0/3856496239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 msgr2=0x556cf60664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 -- 172.21.15.26:0/3856496239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 msgr2=0x556cf6073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf6073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 -- 172.21.15.26:0/3856496239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf5de4de0 con 0x556cf60669e0 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009230 tx=0x7f7dec00cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ea50c65b10edd1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7dddffb700 1 -- 172.21.15.26:0/3856496239 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec094ea0 con 0x556cf60669e0 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7dddffb700 1 -- 172.21.15.26:0/3856496239 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dec09a290 con 0x556cf60669e0 2023-10-06T21:33:58.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.737+0000 7f7dddffb700 1 -- 172.21.15.26:0/3856496239 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec09a750 con 0x556cf60669e0 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 -- 172.21.15.26:0/3856496239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 msgr2=0x556cf6071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009230 tx=0x7f7dec00cc50 comp rx=0 tx=0).stop 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 -- 172.21.15.26:0/3856496239 shutdown_connections 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf6073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3856496239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf6071350 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 -- 172.21.15.26:0/3856496239 >> 172.21.15.26:0/3856496239 conn(0x556cf5fdb260 msgr2=0x556cf6062450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 -- 172.21.15.26:0/3856496239 shutdown_connections 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.738+0000 7f7e11385b80 1 -- 172.21.15.26:0/3856496239 wait complete. 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.739+0000 7f7e11385b80 1 -- start start 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.739+0000 7f7e11385b80 1 Processor -- start 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf606e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5fb9e30 con 0x556cf6071890 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5f56710 con 0x556cf60669e0 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5f563b0 con 0x556cf60660c0 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7e11385b80 10 client.0 _ll_register_callbacks cb 0x556cf5db3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52200/0 (socket says 172.21.15.26:52200) 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 -- 172.21.15.26:0/3559401843 learned_addr learned my addr 172.21.15.26:0/3559401843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df77fe700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf606e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 -- 172.21.15.26:0/3559401843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 msgr2=0x556cf606e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf606e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 -- 172.21.15.26:0/3559401843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 msgr2=0x556cf606f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.740+0000 7f7df67fc700 1 -- 172.21.15.26:0/3559401843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df81a6200 con 0x556cf60669e0 2023-10-06T21:33:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0cd290 tx=0x7f7dec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682b0fefa59bdcd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:33:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec094ea0 con 0x556cf60669e0 2023-10-06T21:33:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cf6059f10 con 0x556cf60669e0 2023-10-06T21:33:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7dec099980 con 0x556cf60669e0 2023-10-06T21:33:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dec09a4f0 con 0x556cf60669e0 2023-10-06T21:33:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7e11385b80 10 client.27860 fetch_fsmap learned FSMap version 67 2023-10-06T21:33:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x556cf5dcdc60 con 0x556cf60669e0 2023-10-06T21:33:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.741+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556cf6059f10 con 0x556cf60669e0 2023-10-06T21:33:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7dec096080 con 0x556cf60669e0 2023-10-06T21:33:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 10 client.27860 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:33:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 10 client.27860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 20 client.27860 populate_metadata read hostname 'smithi026' 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf6081020 0x556cf6083440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 4 client.27860 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cf5fb2e10 con 0x556cf6081020 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf6081020 0x556cf6083440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf6081020 0x556cf6083440 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b27a96a852c5c8 server_cookie=3373fc6239ae140d in_seq=0 out_seq=0 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.742+0000 7f7dd6ffd700 10 client.27860 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3559401843 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556cf5fb2e10 con 0x556cf6081020 2023-10-06T21:33:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7dd6ffd700 10 client.27860 handle_command_reply: tid=0 2023-10-06T21:33:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 client.27860 shutdown 2023-10-06T21:33:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 msgr2=0x556cf606eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0cd290 tx=0x7f7dec0047c0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 shutdown_connections 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf606e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf6081020 0x556cf6083440 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3559401843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606eb20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 >> 172.21.15.26:0/3559401843 conn(0x556cf5fdb260 msgr2=0x556cf6062040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.743+0000 7f7dd6ffd700 0 client.27860 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.744+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 shutdown_connections 2023-10-06T21:33:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.744+0000 7f7e11385b80 1 -- 172.21.15.26:0/3559401843 wait complete. 2023-10-06T21:33:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.744+0000 7f7e11385b80 20 client.27860 trim_cache size 0 max 16384 2023-10-06T21:33:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 Processor -- start 2023-10-06T21:33:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 -- start start 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf5e3a920 con 0x556cf6071890 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf605c590 con 0x556cf60669e0 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.763+0000 7f7e11385b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf6078990 con 0x556cf60660c0 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52206/0 (socket says 172.21.15.26:52206) 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 -- 172.21.15.26:0/2961785375 learned_addr learned my addr 172.21.15.26:0/2961785375 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df77fe700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 -- 172.21.15.26:0/2961785375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 msgr2=0x556cf60783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60783d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 -- 172.21.15.26:0/2961785375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 msgr2=0x556cf606b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 -- 172.21.15.26:0/2961785375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cf5e0b250 con 0x556cf60669e0 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df77fe700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7df67fc700 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009340 tx=0x7f7dec005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347aa7fdef6c428d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7dde7fc700 1 -- 172.21.15.26:0/2961785375 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec094ea0 con 0x556cf60669e0 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7dde7fc700 1 -- 172.21.15.26:0/2961785375 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dec00cc50 con 0x556cf60669e0 2023-10-06T21:33:58.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.764+0000 7f7dde7fc700 1 -- 172.21.15.26:0/2961785375 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec09a4f0 con 0x556cf60669e0 2023-10-06T21:33:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 -- 172.21.15.26:0/2961785375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 msgr2=0x556cf606ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009340 tx=0x7f7dec005b40 comp rx=0 tx=0).stop 2023-10-06T21:33:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 -- 172.21.15.26:0/2961785375 shutdown_connections 2023-10-06T21:33:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf606b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60660c0 0x556cf60783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 --2- 172.21.15.26:0/2961785375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60669e0 0x556cf606ee90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 -- 172.21.15.26:0/2961785375 >> 172.21.15.26:0/2961785375 conn(0x556cf5fdb260 msgr2=0x556cf6069930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 -- 172.21.15.26:0/2961785375 shutdown_connections 2023-10-06T21:33:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.765+0000 7f7e11385b80 1 -- 172.21.15.26:0/2961785375 wait complete. 2023-10-06T21:33:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.766+0000 7f7e11385b80 1 -- start start 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.766+0000 7f7e11385b80 1 Processor -- start 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60669e0 0x556cf6075d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf60762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf60d3e10 con 0x556cf6071890 2023-10-06T21:33:58.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf60d3f50 con 0x556cf60660c0 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cf60d4110 con 0x556cf60669e0 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7e11385b80 10 client.0 _ll_register_callbacks cb 0x7f7dec005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:52210/0 (socket says 172.21.15.26:52210) 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 -- 172.21.15.26:0/3182889549 learned_addr learned my addr 172.21.15.26:0/3182889549 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:33:58.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60669e0 0x556cf6075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df77fe700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf60762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 -- 172.21.15.26:0/3182889549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60669e0 msgr2=0x556cf6075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60669e0 0x556cf6075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 -- 172.21.15.26:0/3182889549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 msgr2=0x556cf60762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf60762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df6ffd700 1 -- 172.21.15.26:0/3182889549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cf5dcdc60 con 0x556cf60660c0 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.767+0000 7f7df77fe700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf60762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7df6ffd700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011ab0 tx=0x7f7de80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef064ba9508d62d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8022030 con 0x556cf60660c0 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7de802b4f0 con 0x556cf60660c0 2023-10-06T21:33:58.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dec09a290 con 0x556cf60660c0 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7de801d820 con 0x556cf60660c0 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7e11385b80 10 client.27866 fetch_fsmap learned FSMap version 67 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x556cf5de32b0 con 0x556cf60660c0 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.768+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dec09a290 con 0x556cf60660c0 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7de80299c0 con 0x556cf60660c0 2023-10-06T21:33:58.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 10 client.27866 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 10 client.27866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 20 client.27866 populate_metadata read hostname 'smithi026' 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf60d5900 0x556cf60d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 4 client.27866 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556cf5fb2b30 con 0x556cf60d5900 2023-10-06T21:33:58.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf60d5900 0x556cf60d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7dd6ffd700 10 client.27866 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.769+0000 7f7df67fc700 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf60d5900 0x556cf60d7d20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525a0967e0d84b22 server_cookie=ece51ce938095b7b in_seq=0 out_seq=0 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7dd6ffd700 1 -- 172.21.15.26:0/3182889549 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556cf5fb2b30 con 0x556cf60d5900 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7dd6ffd700 10 client.27866 handle_command_reply: tid=0 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 client.27866 shutdown 2023-10-06T21:33:58.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 msgr2=0x556cf6075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011ab0 tx=0x7f7de80047c0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 shutdown_connections 2023-10-06T21:33:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cf6071890 0x556cf60762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x556cf60d5900 0x556cf60d7d20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556cf60660c0 0x556cf6075840 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 --2- 172.21.15.26:0/3182889549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556cf60669e0 0x556cf6075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 >> 172.21.15.26:0/3182889549 conn(0x556cf5fdb260 msgr2=0x556cf606eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7dd6ffd700 0 client.27866 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:33:58.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 shutdown_connections 2023-10-06T21:33:58.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 1 -- 172.21.15.26:0/3182889549 wait complete. 2023-10-06T21:33:58.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.770+0000 7f7e11385b80 20 client.27866 trim_cache size 0 max 16384 2023-10-06T21:33:58.785 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:33:58.785 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:33:58.785 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:33:58.785 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:33:58.786 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:33:58.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 msgr2=0x7f7de00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 0x7f7de00a3ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009c60 tx=0x7f7de8009260 comp rx=0 tx=0).stop 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 msgr2=0x7f7df8199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e970 tx=0x7f7dec00ec80 comp rx=0 tx=0).stop 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 shutdown_connections 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7df819da70 0x7f7df819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df8103300 0x7f7df81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7de00a15f0 0x7f7de00a3ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 --2- 172.21.15.26:0/186637114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df8108420 0x7f7df8199610 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:33:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 >> 172.21.15.26:0/186637114 conn(0x7f7df80feb40 msgr2=0x7f7df80ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:33:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 shutdown_connections 2023-10-06T21:33:58.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:33:58.771+0000 7f7dfd9ff700 1 -- 172.21.15.26:0/186637114 wait complete. 2023-10-06T21:33:58.792 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:33:58.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:33:58 smithi026 ceph-mon[98490]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:59.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:33:58 smithi078 ceph-mon[134896]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:33:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:33:58 smithi167 ceph-mon[134494]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:00 smithi026 ceph-mon[98490]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:00 smithi078 ceph-mon[134896]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:00 smithi167 ceph-mon[134494]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:02 smithi026 ceph-mon[98490]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:02 smithi078 ceph-mon[134896]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:03.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:02 smithi167 ceph-mon[134494]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:04.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:04 smithi026 ceph-mon[98490]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:04.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:04 smithi078 ceph-mon[134896]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:04 smithi167 ceph-mon[134494]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:07.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:06 smithi026 ceph-mon[98490]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:34:07.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:06 smithi078 ceph-mon[134896]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:34:07.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:06 smithi167 ceph-mon[134494]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:34:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:08 smithi026 ceph-mon[98490]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:08 smithi078 ceph-mon[134896]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:09.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:08 smithi167 ceph-mon[134494]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:34:11.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:10 smithi026 ceph-mon[98490]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:11.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:10 smithi078 ceph-mon[134896]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:11.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:10 smithi167 ceph-mon[134494]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:34:12.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:11 smithi026 ceph-mon[98490]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:34:12.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:11 smithi078 ceph-mon[134896]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:34:12.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:11 smithi167 ceph-mon[134494]: osdmap e124: 12 total, 12 up, 12 in 2023-10-06T21:34:13.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:12 smithi026 ceph-mon[98490]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:12 smithi078 ceph-mon[134896]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:13.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:12 smithi167 ceph-mon[134494]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:14.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:14 smithi026 ceph-mon[98490]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:14 smithi078 ceph-mon[134896]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:15.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:14 smithi167 ceph-mon[134494]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 819 B/s wr, 6 op/s 2023-10-06T21:34:15.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:15 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:15.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:15 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:16.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:15 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:34:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:16 smithi026 ceph-mon[98490]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:16.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:16 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:16 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:16.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:16 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:34:16.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:16 smithi078 ceph-mon[134896]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:16 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:16 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:16.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:16 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:34:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:16 smithi167 ceph-mon[134494]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:16 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:34:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:16 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:34:17.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:16 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:34:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:18 smithi026 ceph-mon[98490]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:18 smithi026 ceph-mon[98490]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:34:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:18 smithi078 ceph-mon[134896]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:18 smithi078 ceph-mon[134896]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:34:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:18 smithi167 ceph-mon[134494]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:18 smithi167 ceph-mon[134494]: osdmap e125: 12 total, 12 up, 12 in 2023-10-06T21:34:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:20 smithi026 ceph-mon[98490]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 228 B/s wr, 6 op/s 2023-10-06T21:34:21.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:20 smithi078 ceph-mon[134896]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 228 B/s wr, 6 op/s 2023-10-06T21:34:21.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:20 smithi167 ceph-mon[134494]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 228 B/s wr, 6 op/s 2023-10-06T21:34:23.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:22 smithi026 ceph-mon[98490]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:23.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:22 smithi078 ceph-mon[134896]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:22 smithi167 ceph-mon[134494]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:24.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:24 smithi026 ceph-mon[98490]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:24.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:24 smithi078 ceph-mon[134896]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:25.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:24 smithi167 ceph-mon[134494]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-10-06T21:34:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:26 smithi026 ceph-mon[98490]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:27.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:26 smithi078 ceph-mon[134896]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:27.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:26 smithi167 ceph-mon[134494]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:34:28.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:34:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 -- 172.21.15.26:0/3225227821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c10d4f0 msgr2=0x7f639c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 --2- 172.21.15.26:0/3225227821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c10d4f0 0x7f639c10f960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f639400b0a0 tx=0x7f639400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 -- 172.21.15.26:0/3225227821 shutdown_connections 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 --2- 172.21.15.26:0/3225227821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c10d4f0 0x7f639c10f960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 --2- 172.21.15.26:0/3225227821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c0732e0 0x7f639c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 --2- 172.21.15.26:0/3225227821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 -- 172.21.15.26:0/3225227821 >> 172.21.15.26:0/3225227821 conn(0x7f639c0fc760 msgr2=0x7f639c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 -- 172.21.15.26:0/3225227821 shutdown_connections 2023-10-06T21:34:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.040+0000 7f639b59e700 1 -- 172.21.15.26:0/3225227821 wait complete. 2023-10-06T21:34:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.041+0000 7f639b59e700 1 Processor -- start 2023-10-06T21:34:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 -- start start 2023-10-06T21:34:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 0x7f639c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c10d4f0 0x7f639c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19e0b0 con 0x7f639c073c00 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19e220 con 0x7f639c0732e0 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639b59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19e3e0 con 0x7f639c10d4f0 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 0x7f639c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60498/0 (socket says 172.21.15.26:60498) 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f6399d9b700 1 -- 172.21.15.26:0/1132650329 learned_addr learned my addr 172.21.15.26:0/1132650329 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.042+0000 7f639ad9d700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c10d4f0 0x7f639c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 -- 172.21.15.26:0/1132650329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c10d4f0 msgr2=0x7f639c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c10d4f0 0x7f639c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 -- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 msgr2=0x7f639c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 -- 172.21.15.26:0/1132650329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009d00 con 0x7f639c0732e0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639a59c700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 0x7f639c1990d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9f0 tx=0x7f638c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f638b7fe700 1 -- 172.21.15.26:0/1132650329 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00c9c0 con 0x7f639c0732e0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f638b7fe700 1 -- 172.21.15.26:0/1132650329 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f638c010430 con 0x7f639c0732e0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f638b7fe700 1 -- 172.21.15.26:0/1132650329 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c018680 con 0x7f639c0732e0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.043+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c19e6c0 con 0x7f639c0732e0 2023-10-06T21:34:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.044+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c1a5f00 con 0x7f639c0732e0 2023-10-06T21:34:29.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.048+0000 7f638b7fe700 1 -- 172.21.15.26:0/1132650329 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f638c010860 con 0x7f639c0732e0 2023-10-06T21:34:29.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 Processor -- start 2023-10-06T21:34:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 -- start start 2023-10-06T21:34:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6abc0 0x55f01aa75530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a9d3460 con 0x55f01aa75a70 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a9c4c20 con 0x55f01aa6a2a0 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.049+0000 7f63b438db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a9c2540 con 0x55f01aa6abc0 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38610/0 (socket says 172.21.15.26:38610) 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 -- 172.21.15.26:0/4019187496 learned_addr learned my addr 172.21.15.26:0/4019187496 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f6399d9b700 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639a59c700 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6abc0 0x55f01aa75530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 -- 172.21.15.26:0/4019187496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6abc0 msgr2=0x55f01aa75530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6abc0 0x55f01aa75530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 -- 172.21.15.26:0/4019187496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 msgr2=0x55f01aa77e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa77e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 -- 172.21.15.26:0/4019187496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01a7e8de0 con 0x55f01aa6a2a0 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f638b7fe700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 0x7f63840a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.050+0000 7f639ad9d700 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6394015000 tx=0x7f6394009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397b6f66c7b0a709 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.051+0000 7f638b7fe700 1 -- 172.21.15.26:0/1132650329 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f638c014070 con 0x7f639c0732e0 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.051+0000 7f639a59c700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 0x7f63840a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.051+0000 7f6388ff9700 1 -- 172.21.15.26:0/4019187496 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e040 con 0x55f01aa6a2a0 2023-10-06T21:34:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.051+0000 7f6388ff9700 1 -- 172.21.15.26:0/4019187496 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f639400bcb0 con 0x55f01aa6a2a0 2023-10-06T21:34:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.051+0000 7f6388ff9700 1 -- 172.21.15.26:0/4019187496 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940079a0 con 0x55f01aa6a2a0 2023-10-06T21:34:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- 172.21.15.26:0/4019187496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 msgr2=0x55f01aa6a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f639a59c700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 0x7f63840a3b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f638c09a180 tx=0x7f638c010c60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6394015000 tx=0x7f6394009c20 comp rx=0 tx=0).stop 2023-10-06T21:34:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- 172.21.15.26:0/4019187496 shutdown_connections 2023-10-06T21:34:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6a680 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 --2- 172.21.15.26:0/4019187496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6abc0 0x55f01aa75530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- 172.21.15.26:0/4019187496 >> 172.21.15.26:0/4019187496 conn(0x55f01a9df260 msgr2=0x55f01aa66630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- 172.21.15.26:0/4019187496 shutdown_connections 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- 172.21.15.26:0/4019187496 wait complete. 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.052+0000 7f63b438db80 1 -- start start 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 Processor -- start 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6a2a0 0x55f01aa75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a9bde30 con 0x55f01aa6a2a0 2023-10-06T21:34:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a95a710 con 0x55f01aa6abc0 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a95a3b0 con 0x55f01aa75a70 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f63b438db80 10 client.0 _ll_register_callbacks cb 0x55f01a7b7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38614/0 (socket says 172.21.15.26:38614) 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.053+0000 7f6399d9b700 1 -- 172.21.15.26:0/1739601247 learned_addr learned my addr 172.21.15.26:0/1739601247 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f639ad9d700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6a2a0 0x55f01aa75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 -- 172.21.15.26:0/1739601247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 msgr2=0x55f01aa72d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 -- 172.21.15.26:0/1739601247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6a2a0 msgr2=0x55f01aa75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6a2a0 0x55f01aa75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 -- 172.21.15.26:0/1739601247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c110620 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6399d9b700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6390012860 tx=0x7f6390012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8200a46b685498d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6381ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.054+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390017070 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6390004050 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f01aa5e160 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6390004210 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 10 client.27878 fetch_fsmap learned FSMap version 67 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55f01a7d1c60 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f01aa5e160 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f639000f670 con 0x55f01aa6abc0 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 10 client.27878 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:34:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 10 client.27878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 20 client.27878 populate_metadata read hostname 'smithi026' 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aa854c0 0x55f01aa878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 4 client.27878 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.055+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f01a9b6e10 con 0x55f01aa854c0 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.056+0000 7f639ad9d700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aa854c0 0x55f01aa878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.056+0000 7f6381ffb700 10 client.27878 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.056+0000 7f639ad9d700 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aa854c0 0x55f01aa878e0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e70fb591d48dbd8 server_cookie=8f23094ecde67c34 in_seq=0 out_seq=0 2023-10-06T21:34:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f6381ffb700 1 -- 172.21.15.26:0/1739601247 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f01a9b6e10 con 0x55f01aa854c0 2023-10-06T21:34:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f6381ffb700 10 client.27878 handle_command_reply: tid=0 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 client.27878 shutdown 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 msgr2=0x55f01aa726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6390012860 tx=0x7f6390012b70 comp rx=0 tx=0).stop 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 shutdown_connections 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6a2a0 0x55f01aa75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aa854c0 0x55f01aa878e0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 --2- 172.21.15.26:0/1739601247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa726c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 >> 172.21.15.26:0/1739601247 conn(0x55f01a9df260 msgr2=0x55f01aa661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f6381ffb700 0 client.27878 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 shutdown_connections 2023-10-06T21:34:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.057+0000 7f63b438db80 1 -- 172.21.15.26:0/1739601247 wait complete. 2023-10-06T21:34:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.058+0000 7f63b438db80 20 client.27878 trim_cache size 0 max 16384 2023-10-06T21:34:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:28 smithi026 ceph-mon[98490]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:34:29.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.075+0000 7f63b438db80 1 Processor -- start 2023-10-06T21:34:29.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.075+0000 7f63b438db80 1 -- start start 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6abc0 0x55f01aa71ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01a83e920 con 0x55f01aa6abc0 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01aa6e570 con 0x55f01aa6a2a0 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f63b438db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01aa6ba90 con 0x55f01aa75a70 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6abc0 0x55f01aa71ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38630/0 (socket says 172.21.15.26:38630) 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 -- 172.21.15.26:0/422377066 learned_addr learned my addr 172.21.15.26:0/422377066 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 -- 172.21.15.26:0/422377066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 msgr2=0x55f01aa72400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 -- 172.21.15.26:0/422377066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6abc0 msgr2=0x55f01aa71ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6abc0 0x55f01aa71ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.076+0000 7f639ad9d700 1 -- 172.21.15.26:0/422377066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01a80f250 con 0x55f01aa6a2a0 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.077+0000 7f639ad9d700 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6394007770 tx=0x7f6394006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab96baffbb47d87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.077+0000 7f63897fa700 1 -- 172.21.15.26:0/422377066 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639402a070 con 0x55f01aa6a2a0 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.077+0000 7f63897fa700 1 -- 172.21.15.26:0/422377066 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6394007810 con 0x55f01aa6a2a0 2023-10-06T21:34:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.077+0000 7f63897fa700 1 -- 172.21.15.26:0/422377066 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e040 con 0x55f01aa6a2a0 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- 172.21.15.26:0/422377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 msgr2=0x55f01aa6d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6394007770 tx=0x7f6394006230 comp rx=0 tx=0).stop 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- 172.21.15.26:0/422377066 shutdown_connections 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa75a70 0x55f01aa72400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6a2a0 0x55f01aa6d140 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 --2- 172.21.15.26:0/422377066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa6abc0 0x55f01aa71ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- 172.21.15.26:0/422377066 >> 172.21.15.26:0/422377066 conn(0x55f01a9df260 msgr2=0x55f01aa65880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- 172.21.15.26:0/422377066 shutdown_connections 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- 172.21.15.26:0/422377066 wait complete. 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 -- start start 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.078+0000 7f63b438db80 1 Processor -- start 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 0x55f01aa7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01aa6f4e0 con 0x55f01aa75a70 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01aa6f650 con 0x55f01aa6abc0 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01aa6f810 con 0x55f01aa6a2a0 2023-10-06T21:34:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f63b438db80 10 client.0 _ll_register_callbacks cb 0x7f6390009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f6399d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:38634/0 (socket says 172.21.15.26:38634) 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.079+0000 7f6399d9b700 1 -- 172.21.15.26:0/568430782 learned_addr learned my addr 172.21.15.26:0/568430782 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f639ad9d700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f639ad9d700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 0x55f01aa7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 -- 172.21.15.26:0/568430782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 msgr2=0x55f01aa7c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 0x55f01aa7c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 -- 172.21.15.26:0/568430782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 msgr2=0x55f01aa6efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa6efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 -- 172.21.15.26:0/568430782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f01a7e8de0 con 0x55f01aa6abc0 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f639ad9d700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 0x55f01aa7c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:34:29.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6399d9b700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55f01aa62d50 tx=0x7f639000f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd6fa5be244429 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6381ffb700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390017070 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6390000e40 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.080+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f639400bcb0 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6390005400 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 10 client.27884 fetch_fsmap learned FSMap version 67 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55f01a7e72b0 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6390038080 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6390010430 con 0x55f01aa6abc0 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 10 client.27884 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:34:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 10 client.27884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 20 client.27884 populate_metadata read hostname 'smithi026' 2023-10-06T21:34:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aada7a0 0x55f01aadcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 4 client.27884 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:34:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f01a9b6b30 con 0x55f01aada7a0 2023-10-06T21:34:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.081+0000 7f639ad9d700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aada7a0 0x55f01aadcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f639ad9d700 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aada7a0 0x55f01aadcbc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deccd2ed3727f62e server_cookie=fa816af0a3260fc2 in_seq=0 out_seq=0 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f6381ffb700 10 client.27884 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f6381ffb700 1 -- 172.21.15.26:0/568430782 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f01a9b6b30 con 0x55f01aada7a0 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f6381ffb700 10 client.27884 handle_command_reply: tid=0 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 client.27884 shutdown 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 msgr2=0x55f01aa6ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55f01aa62d50 tx=0x7f639000f890 comp rx=0 tx=0).stop 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 shutdown_connections 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f01aa75a70 0x55f01aa6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f01aa6a2a0 0x55f01aa7c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55f01aada7a0 0x55f01aadcbc0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 --2- 172.21.15.26:0/568430782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f01aa6abc0 0x55f01aa6ea60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 >> 172.21.15.26:0/568430782 conn(0x55f01a9df260 msgr2=0x55f01aa66110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f6381ffb700 0 client.27884 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.082+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 shutdown_connections 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f63b438db80 1 -- 172.21.15.26:0/568430782 wait complete. 2023-10-06T21:34:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f63b438db80 20 client.27884 trim_cache size 0 max 16384 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:34:29.090 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:34:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 msgr2=0x7f63840a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 0x7f63840a3b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f638c09a180 tx=0x7f638c010c60 comp rx=0 tx=0).stop 2023-10-06T21:34:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 msgr2=0x7f639c1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 0x7f639c1990d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f638c00e9f0 tx=0x7f638c00ed00 comp rx=0 tx=0).stop 2023-10-06T21:34:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 shutdown_connections 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f639c10d4f0 0x7f639c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c0732e0 0x7f639c1990d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f63840a1690 0x7f63840a3b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 --2- 172.21.15.26:0/1132650329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f639c073c00 0x7f639c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.083+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 >> 172.21.15.26:0/1132650329 conn(0x7f639c0fc760 msgr2=0x7f639c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.084+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 shutdown_connections 2023-10-06T21:34:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:29.084+0000 7f639b59e700 1 -- 172.21.15.26:0/1132650329 wait complete. 2023-10-06T21:34:29.104 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:34:29.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:28 smithi078 ceph-mon[134896]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:34:29.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:28 smithi167 ceph-mon[134494]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:34:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:30 smithi026 ceph-mon[98490]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:34:31.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:30 smithi078 ceph-mon[134896]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:34:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:30 smithi167 ceph-mon[134494]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:34:33.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:32 smithi026 ceph-mon[98490]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:34:33.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:32 smithi078 ceph-mon[134896]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:34:33.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:32 smithi167 ceph-mon[134494]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:34:34.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:34 smithi026 ceph-mon[98490]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:34.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:34 smithi078 ceph-mon[134896]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:35.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:34 smithi167 ceph-mon[134494]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:36 smithi026 ceph-mon[98490]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:36 smithi078 ceph-mon[134896]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:37.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:36 smithi167 ceph-mon[134494]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:38 smithi026 ceph-mon[98490]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:38 smithi078 ceph-mon[134896]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:38 smithi167 ceph-mon[134494]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:41.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:40 smithi026 ceph-mon[98490]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:41.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:40 smithi078 ceph-mon[134896]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:41.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:40 smithi167 ceph-mon[134494]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:43.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:42 smithi026 ceph-mon[98490]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:42 smithi078 ceph-mon[134896]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:43.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:42 smithi167 ceph-mon[134494]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:44.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:44 smithi078 ceph-mon[134896]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:44 smithi167 ceph-mon[134494]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:45.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:44 smithi026 ceph-mon[98490]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:47.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:46 smithi026 ceph-mon[98490]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:47.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:46 smithi078 ceph-mon[134896]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:47.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:46 smithi167 ceph-mon[134494]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:49.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:48 smithi026 ceph-mon[98490]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:49.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:48 smithi078 ceph-mon[134896]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:49.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:48 smithi167 ceph-mon[134494]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:51.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:50 smithi026 ceph-mon[98490]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:51.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:50 smithi078 ceph-mon[134896]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:51.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:50 smithi167 ceph-mon[134494]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:53.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:52 smithi026 ceph-mon[98490]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:52 smithi078 ceph-mon[134896]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:53.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:52 smithi167 ceph-mon[134494]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:54.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:54 smithi078 ceph-mon[134896]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:55.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:54 smithi167 ceph-mon[134494]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:55.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:54 smithi026 ceph-mon[98490]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:34:57.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:56 smithi026 ceph-mon[98490]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:57.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:56 smithi078 ceph-mon[134896]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:56 smithi167 ceph-mon[134494]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:34:59.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:34:58 smithi026 ceph-mon[98490]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:59.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:34:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:34:58 smithi078 ceph-mon[134896]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:59.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:34:58 smithi167 ceph-mon[134494]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:34:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 -- 172.21.15.26:0/1673049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 msgr2=0x7f880c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 --2- 172.21.15.26:0/1673049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c101340 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009990 tx=0x7f87fc009ca0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 -- 172.21.15.26:0/1673049750 shutdown_connections 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 --2- 172.21.15.26:0/1673049750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 --2- 172.21.15.26:0/1673049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c101340 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 --2- 172.21.15.26:0/1673049750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c101910 0x7f880c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 -- 172.21.15.26:0/1673049750 >> 172.21.15.26:0/1673049750 conn(0x7f880c0fc7c0 msgr2=0x7f880c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 -- 172.21.15.26:0/1673049750 shutdown_connections 2023-10-06T21:34:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.322+0000 7f881179d700 1 -- 172.21.15.26:0/1673049750 wait complete. 2023-10-06T21:34:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.323+0000 7f881179d700 1 Processor -- start 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 -- start start 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c19d970 0x7f880c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c19e3f0 con 0x7f880c19d970 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c19e560 con 0x7f880c100f60 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f881179d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c19e720 con 0x7f880c106080 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f880bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f880bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57498/0 (socket says 172.21.15.26:57498) 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.324+0000 7f880bfff700 1 -- 172.21.15.26:0/245294740 learned_addr learned my addr 172.21.15.26:0/245294740 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f8810f9c700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c19d970 0x7f880c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880b7fe700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 -- 172.21.15.26:0/245294740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 msgr2=0x7f880c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 -- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c19d970 msgr2=0x7f880c19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c19d970 0x7f880c19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 -- 172.21.15.26:0/245294740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0095f0 con 0x7f880c100f60 2023-10-06T21:34:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880b7fe700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:34:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f880bfff700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0095c0 tx=0x7f87fc00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f88097fa700 1 -- 172.21.15.26:0/245294740 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc01d070 con 0x7f880c100f60 2023-10-06T21:34:59.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f88097fa700 1 -- 172.21.15.26:0/245294740 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87fc0054f0 con 0x7f880c100f60 2023-10-06T21:34:59.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f88097fa700 1 -- 172.21.15.26:0/245294740 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc004bd0 con 0x7f880c100f60 2023-10-06T21:34:59.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.325+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c1a5c20 con 0x7f880c100f60 2023-10-06T21:34:59.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.326+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c1a6050 con 0x7f880c100f60 2023-10-06T21:34:59.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.329+0000 7f88097fa700 1 -- 172.21.15.26:0/245294740 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f87fc00fc30 con 0x7f880c100f60 2023-10-06T21:34:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.331+0000 7f88097fa700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 0x7f87f40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.331+0000 7f88097fa700 1 -- 172.21.15.26:0/245294740 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f87fc0cdfc0 con 0x7f880c100f60 2023-10-06T21:34:59.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.331+0000 7f88262d1b80 1 Processor -- start 2023-10-06T21:34:59.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.331+0000 7f880b7fe700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 0x7f87f40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 -- start start 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f880b7fe700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 0x7f87f40a3bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8800005fd0 tx=0x7f8800005ee0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3dbc0 0x55d967e50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e53670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967da6460 con 0x55d967e3dbc0 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967d97c20 con 0x55d967e3d2a0 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f88262d1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967d95540 con 0x55d967e51280 2023-10-06T21:34:59.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f880bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f880bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57502/0 (socket says 172.21.15.26:57502) 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f880bfff700 1 -- 172.21.15.26:0/1447588070 learned_addr learned my addr 172.21.15.26:0/1447588070 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.332+0000 7f880b7fe700 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3dbc0 0x55d967e50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 -- 172.21.15.26:0/1447588070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 msgr2=0x55d967e53670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e53670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 -- 172.21.15.26:0/1447588070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3dbc0 msgr2=0x55d967e50d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3dbc0 0x55d967e50d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 -- 172.21.15.26:0/1447588070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d967bbbde0 con 0x55d967e3d2a0 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f880bfff700 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0a2d90 tx=0x7f87fc09aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968118bf10fd4a22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f87f27fc700 1 -- 172.21.15.26:0/1447588070 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc09f110 con 0x55d967e3d2a0 2023-10-06T21:34:59.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f87f27fc700 1 -- 172.21.15.26:0/1447588070 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87fc0a32c0 con 0x55d967e3d2a0 2023-10-06T21:34:59.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.333+0000 7f87f27fc700 1 -- 172.21.15.26:0/1447588070 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc09a4b0 con 0x55d967e3d2a0 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 -- 172.21.15.26:0/1447588070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 msgr2=0x55d967e3d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0a2d90 tx=0x7f87fc09aa10 comp rx=0 tx=0).stop 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 -- 172.21.15.26:0/1447588070 shutdown_connections 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e53670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3d2a0 0x55d967e3d680 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1447588070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3dbc0 0x55d967e50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 -- 172.21.15.26:0/1447588070 >> 172.21.15.26:0/1447588070 conn(0x55d967db2260 msgr2=0x55d967e39630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 -- 172.21.15.26:0/1447588070 shutdown_connections 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.334+0000 7f88262d1b80 1 -- 172.21.15.26:0/1447588070 wait complete. 2023-10-06T21:34:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 -- start start 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 Processor -- start 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e5c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967d90e30 con 0x55d967e3d2a0 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967d2d710 con 0x55d967e3dbc0 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.335+0000 7f88262d1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967d2d3b0 con 0x55d967e51280 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f88262d1b80 10 client.0 _ll_register_callbacks cb 0x55d967b8a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e51660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44422/0 (socket says 172.21.15.26:44422) 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880bfff700 1 -- 172.21.15.26:0/986901469 learned_addr learned my addr 172.21.15.26:0/986901469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f8810f9c700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e5c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 -- 172.21.15.26:0/986901469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 msgr2=0x55d967e5c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e5c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 -- 172.21.15.26:0/986901469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 msgr2=0x55d967e51660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e51660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 -- 172.21.15.26:0/986901469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c1a6050 con 0x55d967e3dbc0 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f880b7fe700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e51ba0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f880c19a5e0 tx=0x7f88000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd033719e83fd3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.336+0000 7f87eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800004a30 con 0x55d967e3dbc0 2023-10-06T21:34:59.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d967e31160 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mon.2 v2:172.21.15.78:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8800004b90 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8800004d50 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f88262d1b80 10 client.27896 fetch_fsmap learned FSMap version 67 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d967ba4c60 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mon.2 v2:172.21.15.78:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d967e31160 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.337+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mon.2 v2:172.21.15.78:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f880001f930 con 0x55d967e3dbc0 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 10 client.27896 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 10 client.27896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 20 client.27896 populate_metadata read hostname 'smithi026' 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967e60a40 0x55d967e62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 4 client.27896 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:34:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d967d89e10 con 0x55d967e60a40 2023-10-06T21:34:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f880bfff700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967e60a40 0x55d967e62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f880bfff700 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967e60a40 0x55d967e62e60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7445a35742a3ee server_cookie=6f0f5076f09be190 in_seq=0 out_seq=0 2023-10-06T21:34:59.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.338+0000 7f87eb7fe700 10 client.27896 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f87eb7fe700 1 -- 172.21.15.26:0/986901469 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d967d89e10 con 0x55d967e60a40 2023-10-06T21:34:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f87eb7fe700 10 client.27896 handle_command_reply: tid=0 2023-10-06T21:34:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 client.27896 shutdown 2023-10-06T21:34:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 msgr2=0x55d967e51ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e51ba0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f880c19a5e0 tx=0x7f88000047c0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 shutdown_connections 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e51280 0x55d967e5c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967e60a40 0x55d967e62e60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 --2- 172.21.15.26:0/986901469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e51ba0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f87eb7fe700 0 client.27896 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.339+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 >> 172.21.15.26:0/986901469 conn(0x55d967db2260 msgr2=0x55d967e391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.340+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 shutdown_connections 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.340+0000 7f88262d1b80 1 -- 172.21.15.26:0/986901469 wait complete. 2023-10-06T21:34:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.340+0000 7f88262d1b80 20 client.27896 trim_cache size 0 max 16384 2023-10-06T21:34:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.358+0000 7f88262d1b80 1 Processor -- start 2023-10-06T21:34:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.358+0000 7f88262d1b80 1 -- start start 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3d2a0 0x55d967e4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 0x55d967e5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967c11920 con 0x55d967e51280 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967e5e1a0 con 0x55d967e3dbc0 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f88262d1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967e51f40 con 0x55d967e3d2a0 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f8810f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 0x55d967e5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57534/0 (socket says 172.21.15.26:57534) 2023-10-06T21:34:59.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 -- 172.21.15.26:0/1889833044 learned_addr learned my addr 172.21.15.26:0/1889833044 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880bfff700 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3d2a0 0x55d967e4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 -- 172.21.15.26:0/1889833044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3d2a0 msgr2=0x55d967e4b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3d2a0 0x55d967e4b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 -- 172.21.15.26:0/1889833044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 msgr2=0x55d967e5bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 0x55d967e5bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.359+0000 7f880b7fe700 1 -- 172.21.15.26:0/1889833044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d967e67a90 con 0x55d967e3dbc0 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.360+0000 7f8810f9c700 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 0x55d967e5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.360+0000 7f880b7fe700 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8800009410 tx=0x7f880001ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c88a25caa94e4ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.360+0000 7f87f2ffd700 1 -- 172.21.15.26:0/1889833044 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800038070 con 0x55d967e3dbc0 2023-10-06T21:34:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.360+0000 7f87f2ffd700 1 -- 172.21.15.26:0/1889833044 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f880001faf0 con 0x55d967e3dbc0 2023-10-06T21:34:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.360+0000 7f87f2ffd700 1 -- 172.21.15.26:0/1889833044 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800010910 con 0x55d967e3dbc0 2023-10-06T21:34:59.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- 172.21.15.26:0/1889833044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 msgr2=0x55d967e4bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8800009410 tx=0x7f880001ff70 comp rx=0 tx=0).stop 2023-10-06T21:34:59.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- 172.21.15.26:0/1889833044 shutdown_connections 2023-10-06T21:34:59.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e51280 0x55d967e5bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3d2a0 0x55d967e4b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 --2- 172.21.15.26:0/1889833044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e3dbc0 0x55d967e4bd40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- 172.21.15.26:0/1889833044 >> 172.21.15.26:0/1889833044 conn(0x55d967db2260 msgr2=0x55d967e5cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- 172.21.15.26:0/1889833044 shutdown_connections 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- 172.21.15.26:0/1889833044 wait complete. 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.361+0000 7f88262d1b80 1 -- start start 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.362+0000 7f88262d1b80 1 Processor -- start 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3dbc0 0x55d967e546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967e55160 con 0x55d967e3d2a0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967e552d0 con 0x55d967e51280 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d967e55490 con 0x55d967e3dbc0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f88262d1b80 10 client.0 _ll_register_callbacks cb 0x7f880000ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f8810f9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f8810f9c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:57542/0 (socket says 172.21.15.26:57542) 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f8810f9c700 1 -- 172.21.15.26:0/3377580490 learned_addr learned my addr 172.21.15.26:0/3377580490 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44438/0 (socket says 172.21.15.26:44438) 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880b7fe700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3dbc0 0x55d967e546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 -- 172.21.15.26:0/3377580490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3dbc0 msgr2=0x55d967e546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3dbc0 0x55d967e546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 -- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 msgr2=0x55d967e54c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f880bfff700 1 -- 172.21.15.26:0/3377580490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88040097e0 con 0x55d967e3d2a0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.363+0000 7f8810f9c700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f880bfff700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f87fc005f50 tx=0x7f87fc09b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472ab615ed8733f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f87eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc09a920 con 0x55d967e3d2a0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f87fc0b9170 con 0x55d967e3d2a0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d967d89b30 con 0x55d967e3d2a0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87fc099360 con 0x55d967e3d2a0 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f88262d1b80 10 client.17520 fetch_fsmap learned FSMap version 67 2023-10-06T21:34:59.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.364+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d967ba4c60 con 0x55d967e3d2a0 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d967d89b30 con 0x55d967e3d2a0 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87fc09b220 con 0x55d967e3d2a0 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 10 client.17520 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 20 client.17520 populate_metadata read hostname 'smithi026' 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967eb5770 0x55d967eb7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 4 client.17520 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d967bdcf70 con 0x55d967eb5770 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.365+0000 7f880b7fe700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967eb5770 0x55d967eb7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f87eb7fe700 10 client.17520 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:59.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f880b7fe700 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967eb5770 0x55d967eb7b90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b372e2b8293c7672 server_cookie=d8d233303cf80c6a in_seq=0 out_seq=0 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f87eb7fe700 1 -- 172.21.15.26:0/3377580490 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d967bdcf70 con 0x55d967eb5770 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f87eb7fe700 10 client.17520 handle_command_reply: tid=0 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 client.17520 shutdown 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 msgr2=0x55d967e563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f87fc005f50 tx=0x7f87fc09b800 comp rx=0 tx=0).stop 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 shutdown_connections 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d967eb5770 0x55d967eb7b90 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d967e51280 0x55d967e54c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d967e3d2a0 0x55d967e563f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 --2- 172.21.15.26:0/3377580490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d967e3dbc0 0x55d967e546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f87eb7fe700 0 client.17520 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 >> 172.21.15.26:0/3377580490 conn(0x55d967db2260 msgr2=0x55d967e4b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 shutdown_connections 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.366+0000 7f88262d1b80 1 -- 172.21.15.26:0/3377580490 wait complete. 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f88262d1b80 20 client.17520 trim_cache size 0 max 16384 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:34:59.375 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:34:59.376 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 msgr2=0x7f87f40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 0x7f87f40a3bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8800005fd0 tx=0x7f8800005ee0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 msgr2=0x7f880c198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0095c0 tx=0x7f87fc00f740 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 shutdown_connections 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f87f40a1710 0x7f87f40a3bd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880c19d970 0x7f880c19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c100f60 0x7f880c198fd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 --2- 172.21.15.26:0/245294740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f880c106080 0x7f880c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:34:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.367+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 >> 172.21.15.26:0/245294740 conn(0x7f880c0fc7c0 msgr2=0x7f880c10ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:34:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.368+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 shutdown_connections 2023-10-06T21:34:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:34:59.368+0000 7f881179d700 1 -- 172.21.15.26:0/245294740 wait complete. 2023-10-06T21:34:59.392 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:35:01.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:00 smithi026 ceph-mon[98490]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:01.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:00 smithi078 ceph-mon[134896]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:01.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:00 smithi167 ceph-mon[134494]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:03.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:02 smithi026 ceph-mon[98490]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:03.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:02 smithi078 ceph-mon[134896]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:03.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:02 smithi167 ceph-mon[134494]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:04.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:04 smithi078 ceph-mon[134896]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:05.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:04 smithi167 ceph-mon[134494]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:05.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:04 smithi026 ceph-mon[98490]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:07.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:06 smithi026 ceph-mon[98490]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:35:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:06 smithi078 ceph-mon[134896]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:35:07.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:06 smithi167 ceph-mon[134494]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:35:09.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:08 smithi026 ceph-mon[98490]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:08 smithi078 ceph-mon[134896]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:08 smithi167 ceph-mon[134494]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:35:11.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:10 smithi078 ceph-mon[134896]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:11.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:10 smithi167 ceph-mon[134494]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:11.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:10 smithi026 ceph-mon[98490]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:35:12.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:11 smithi078 ceph-mon[134896]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:35:12.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:11 smithi167 ceph-mon[134494]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:35:12.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:11 smithi026 ceph-mon[98490]: osdmap e126: 12 total, 12 up, 12 in 2023-10-06T21:35:13.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:12 smithi078 ceph-mon[134896]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:13.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:12 smithi167 ceph-mon[134494]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:13.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:12 smithi026 ceph-mon[98490]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:14.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:14 smithi078 ceph-mon[134896]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:15.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:14 smithi167 ceph-mon[134494]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:15.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:14 smithi026 ceph-mon[98490]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 716 B/s wr, 6 op/s 2023-10-06T21:35:17.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:16 smithi026 ceph-mon[98490]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:17.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:16 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:16 smithi078 ceph-mon[134896]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:17.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:16 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:17.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:16 smithi167 ceph-mon[134494]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:17.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:16 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-06T21:35:18.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:17 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:18.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:17 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:18.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:17 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:35:18.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:17 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:17 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:18.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:17 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:35:18.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:17 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-06T21:35:18.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:17 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-06T21:35:18.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:17 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' 2023-10-06T21:35:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:18 smithi026 ceph-mon[98490]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:19.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:18 smithi026 ceph-mon[98490]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:35:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:18 smithi078 ceph-mon[134896]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:19.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:18 smithi078 ceph-mon[134896]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:35:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:18 smithi167 ceph-mon[134494]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:19.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:18 smithi167 ceph-mon[134494]: osdmap e127: 12 total, 12 up, 12 in 2023-10-06T21:35:21.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:20 smithi026 ceph-mon[98490]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:35:21.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:20 smithi078 ceph-mon[134896]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:35:21.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:20 smithi167 ceph-mon[134494]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 114 B/s wr, 6 op/s 2023-10-06T21:35:23.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:22 smithi078 ceph-mon[134896]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:23.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:22 smithi167 ceph-mon[134494]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:23.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:22 smithi026 ceph-mon[98490]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:24.871 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:24 smithi078 ceph-mon[134896]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:25.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:24 smithi167 ceph-mon[134494]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:25.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:24 smithi026 ceph-mon[98490]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-10-06T21:35:27.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:26 smithi026 ceph-mon[98490]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:27.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:26 smithi078 ceph-mon[134896]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:27.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:26 smithi167 ceph-mon[134494]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:29.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:28 smithi026 ceph-mon[98490]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:29.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:28 smithi078 ceph-mon[134896]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:29.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:28 smithi167 ceph-mon[134494]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-10-06T21:35:29.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-06T21:35:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 -- 172.21.15.26:0/813154019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf941020b0 msgr2=0x7faf94102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/813154019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf941020b0 0x7faf94102490 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faf90009990 tx=0x7faf90009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 -- 172.21.15.26:0/813154019 shutdown_connections 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/813154019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9410b480 0x7faf9410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/813154019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf941020b0 0x7faf94102490 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.625+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/813154019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941029d0 0x7faf9410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.626+0000 7faf9c3ff700 1 -- 172.21.15.26:0/813154019 >> 172.21.15.26:0/813154019 conn(0x7faf940fb890 msgr2=0x7faf940fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.626+0000 7faf9c3ff700 1 -- 172.21.15.26:0/813154019 shutdown_connections 2023-10-06T21:35:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.626+0000 7faf9c3ff700 1 -- 172.21.15.26:0/813154019 wait complete. 2023-10-06T21:35:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 Processor -- start 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 -- start start 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941020b0 0x7faf94199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941029d0 0x7faf94199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e2f0 con 0x7faf941029d0 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e460 con 0x7faf9410b480 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.627+0000 7faf9c3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e620 con 0x7faf941020b0 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941029d0 0x7faf94199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55310/0 (socket says 172.21.15.26:55310) 2023-10-06T21:35:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 -- 172.21.15.26:0/1546689173 learned_addr learned my addr 172.21.15.26:0/1546689173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a19b700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941020b0 0x7faf94199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 -- 172.21.15.26:0/1546689173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941020b0 msgr2=0x7faf94199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941020b0 0x7faf94199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 -- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941029d0 msgr2=0x7faf94199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941029d0 0x7faf94199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 -- 172.21.15.26:0/1546689173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900095f0 con 0x7faf9410b480 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.628+0000 7faf9a99c700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00ef30 tx=0x7faf8c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.629+0000 7faf8b7fe700 1 -- 172.21.15.26:0/1546689173 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c018070 con 0x7faf9410b480 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.629+0000 7faf8b7fe700 1 -- 172.21.15.26:0/1546689173 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf8c00f040 con 0x7faf9410b480 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.629+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9419e8a0 con 0x7faf9410b480 2023-10-06T21:35:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.629+0000 7faf8b7fe700 1 -- 172.21.15.26:0/1546689173 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00cd00 con 0x7faf9410b480 2023-10-06T21:35:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.629+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf941a6140 con 0x7faf9410b480 2023-10-06T21:35:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.634+0000 7faf8b7fe700 1 -- 172.21.15.26:0/1546689173 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7faf8c003a00 con 0x7faf9410b480 2023-10-06T21:35:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 Processor -- start 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 -- start start 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d124217460 con 0x55d1242ae9e0 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d124208c20 con 0x55d1242b9890 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.635+0000 7fafafda2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d124206540 con 0x55d1242ae0c0 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf8b7fe700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 0x7faf800a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55324/0 (socket says 172.21.15.26:55324) 2023-10-06T21:35:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9999a700 1 -- 172.21.15.26:0/129895349 learned_addr learned my addr 172.21.15.26:0/129895349 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9999a700 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf8b7fe700 1 -- 172.21.15.26:0/1546689173 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faf8c0ccbe0 con 0x7faf9410b480 2023-10-06T21:35:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a99c700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 0x7faf800a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 -- 172.21.15.26:0/129895349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 msgr2=0x55d1242ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 -- 172.21.15.26:0/129895349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 msgr2=0x55d1242bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9a19b700 1 -- 172.21.15.26:0/129895349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d12402cde0 con 0x55d1242ae9e0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.636+0000 7faf9999a700 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.637+0000 7faf9a99c700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 0x7faf800a3b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009f40 tx=0x7faf8c009ed0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.637+0000 7faf9a19b700 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b9350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf9000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dad14a685b633f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.637+0000 7faf88bfe700 1 -- 172.21.15.26:0/129895349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9001d070 con 0x55d1242ae9e0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.637+0000 7faf88bfe700 1 -- 172.21.15.26:0/129895349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf9001e040 con 0x55d1242ae9e0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.637+0000 7faf88bfe700 1 -- 172.21.15.26:0/129895349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900048d0 con 0x55d1242ae9e0 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 -- 172.21.15.26:0/129895349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 msgr2=0x55d1242b9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b9350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf9000b610 comp rx=0 tx=0).stop 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 -- 172.21.15.26:0/129895349 shutdown_connections 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 --2- 172.21.15.26:0/129895349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b9350 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 -- 172.21.15.26:0/129895349 >> 172.21.15.26:0/129895349 conn(0x55d124223260 msgr2=0x55d1242aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 -- 172.21.15.26:0/129895349 shutdown_connections 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.638+0000 7fafafda2b80 1 -- 172.21.15.26:0/129895349 wait complete. 2023-10-06T21:35:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.639+0000 7fafafda2b80 1 -- start start 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.639+0000 7fafafda2b80 1 Processor -- start 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d124201e30 con 0x55d1242ae9e0 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12419e710 con 0x55d1242b9890 2023-10-06T21:35:29.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12419e3b0 con 0x55d1242ae0c0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7fafafda2b80 10 client.0 _ll_register_callbacks cb 0x55d123ffb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7faf9a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55336/0 (socket says 172.21.15.26:55336) 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7faf9999a700 1 -- 172.21.15.26:0/4105445865 learned_addr learned my addr 172.21.15.26:0/4105445865 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.640+0000 7faf9999a700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 -- 172.21.15.26:0/4105445865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 msgr2=0x55d1242b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 -- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 msgr2=0x55d1242b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 -- 172.21.15.26:0/4105445865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8401a040 con 0x55d1242ae9e0 2023-10-06T21:35:29.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9999a700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf9a19b700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b64f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf900098e0 tx=0x7faf90005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f736cc0c805ed5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.641+0000 7faf7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9001d040 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1242a1f10 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faf9000f9f0 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf9000fbb0 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7fafafda2b80 10 client.17532 fetch_fsmap learned FSMap version 67 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7faf9410e5e0 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1242a1f10 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.642+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf9002b3f0 con 0x55d1242ae9e0 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 10 client.17532 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:35:29.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 20 client.17532 populate_metadata read hostname 'smithi026' 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d1242c8f80 0x55d1242cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 4 client.17532 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1241fae10 con 0x55d1242c8f80 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7faf9999a700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d1242c8f80 0x55d1242cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7faf9999a700 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d1242c8f80 0x55d1242cb3a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f64972e8ca73e5 server_cookie=13a8a21179ae2360 in_seq=0 out_seq=0 2023-10-06T21:35:29.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.643+0000 7faf7dffb700 10 client.17532 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:35:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7faf7dffb700 1 -- 172.21.15.26:0/4105445865 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1241fae10 con 0x55d1242c8f80 2023-10-06T21:35:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7faf7dffb700 10 client.17532 handle_command_reply: tid=0 2023-10-06T21:35:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 client.17532 shutdown 2023-10-06T21:35:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 msgr2=0x55d1242b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b64f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf900098e0 tx=0x7faf90005cb0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 shutdown_connections 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d1242c8f80 0x55d1242cb3a0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242b9890 0x55d1242b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae0c0 0x55d1242b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 --2- 172.21.15.26:0/4105445865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae9e0 0x55d1242b64f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 >> 172.21.15.26:0/4105445865 conn(0x55d124223260 msgr2=0x55d1242a9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.644+0000 7faf7dffb700 0 client.17532 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.645+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 shutdown_connections 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.645+0000 7fafafda2b80 1 -- 172.21.15.26:0/4105445865 wait complete. 2023-10-06T21:35:29.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.645+0000 7fafafda2b80 20 client.17532 trim_cache size 0 max 16384 2023-10-06T21:35:29.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 Processor -- start 2023-10-06T21:35:29.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 -- start start 2023-10-06T21:35:29.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 0x55d1242b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 0x55d1242bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d124082920 con 0x55d1242ae0c0 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1242c5c30 con 0x55d1242ae9e0 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.673+0000 7fafafda2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1242aedc0 con 0x55d1242b9890 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 0x55d1242b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55348/0 (socket says 172.21.15.26:55348) 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 0x55d1242b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50876/0 (socket says 172.21.15.26:50876) 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 -- 172.21.15.26:0/3719263635 learned_addr learned my addr 172.21.15.26:0/3719263635 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9999a700 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 0x55d1242bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 -- 172.21.15.26:0/3719263635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 msgr2=0x55d1242bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 0x55d1242bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 -- 172.21.15.26:0/3719263635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 msgr2=0x55d1242b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 0x55d1242b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 -- 172.21.15.26:0/3719263635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1242a6140 con 0x55d1242ae9e0 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9999a700 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 0x55d1242bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.674+0000 7faf9a19b700 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faf9002b800 tx=0x7faf9000b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c7ab6b84713e2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7faf893ff700 1 -- 172.21.15.26:0/3719263635 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9001d070 con 0x55d1242ae9e0 2023-10-06T21:35:29.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7faf893ff700 1 -- 172.21.15.26:0/3719263635 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf9000f460 con 0x55d1242ae9e0 2023-10-06T21:35:29.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7faf893ff700 1 -- 172.21.15.26:0/3719263635 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90040690 con 0x55d1242ae9e0 2023-10-06T21:35:29.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 -- 172.21.15.26:0/3719263635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 msgr2=0x55d1242b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faf9002b800 tx=0x7faf9000b610 comp rx=0 tx=0).stop 2023-10-06T21:35:29.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 -- 172.21.15.26:0/3719263635 shutdown_connections 2023-10-06T21:35:29.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242b9890 0x55d1242bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242ae0c0 0x55d1242b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 --2- 172.21.15.26:0/3719263635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae9e0 0x55d1242b1cf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 -- 172.21.15.26:0/3719263635 >> 172.21.15.26:0/3719263635 conn(0x55d124223260 msgr2=0x55d1242c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 -- 172.21.15.26:0/3719263635 shutdown_connections 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.675+0000 7fafafda2b80 1 -- 172.21.15.26:0/3719263635 wait complete. 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.676+0000 7fafafda2b80 1 -- start start 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.676+0000 7fafafda2b80 1 Processor -- start 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 0x55d1242bd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 0x55d1242bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 0x55d1242be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1242c0960 con 0x55d1242b9890 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12431bda0 con 0x55d1242ae0c0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12431bf60 con 0x55d1242ae9e0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7fafafda2b80 10 client.0 _ll_register_callbacks cb 0x7faf84007ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 0x55d1242bd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 0x55d1242bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9999a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 0x55d1242bd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55362/0 (socket says 172.21.15.26:55362) 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9999a700 1 -- 172.21.15.26:0/2593307443 learned_addr learned my addr 172.21.15.26:0/2593307443 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9999a700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 0x55d1242be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 -- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 msgr2=0x55d1242bd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 0x55d1242bd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 -- 172.21.15.26:0/2593307443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 msgr2=0x55d1242be3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 0x55d1242be3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9a19b700 1 -- 172.21.15.26:0/2593307443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9410e5e0 con 0x55d1242ae9e0 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.677+0000 7faf9999a700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 0x55d1242be3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf9a19b700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 0x55d1242bde80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55d1242afbb0 tx=0x7faf9002b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e669f534d3436a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9001d040 con 0x55d1242ae9e0 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1241fab30 con 0x55d1242ae9e0 2023-10-06T21:35:29.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7faf90000e40 con 0x55d1242ae9e0 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf9000f9f0 con 0x55d1242ae9e0 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7fafafda2b80 10 client.27733 fetch_fsmap learned FSMap version 67 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d124015c60 con 0x55d1242ae9e0 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.678+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1241fab30 con 0x55d1242ae9e0 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 67) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf9004ac90 con 0x55d1242ae9e0 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 10 client.27733 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 10 client.27733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 20 client.27733 populate_metadata read hostname 'smithi026' 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d12431d2a0 0x55d12431f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 4 client.27733 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 --> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d12404df70 con 0x55d12431d2a0 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7faf9999a700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d12431d2a0 0x55d12431f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7faf7dffb700 10 client.27733 ms_handle_connect on v2:172.21.15.78:6838/3790769886 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.679+0000 7faf9999a700 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d12431d2a0 0x55d12431f690 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9ece71ff8fd730 server_cookie=608e287027caaba9 in_seq=0 out_seq=0 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7faf7dffb700 1 -- 172.21.15.26:0/2593307443 <== mds.0 v2:172.21.15.78:6838/3790769886 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d12404df70 con 0x55d12431d2a0 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7faf7dffb700 10 client.27733 handle_command_reply: tid=0 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 client.27733 shutdown 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 msgr2=0x55d1242bde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 0x55d1242bde80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55d1242afbb0 tx=0x7faf9002b620 comp rx=0 tx=0).stop 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 shutdown_connections 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d1242b9890 0x55d1242be3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:6838/3790769886,v1:172.21.15.78:6839/3790769886] conn(0x55d12431d2a0 0x55d12431f690 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1242ae0c0 0x55d1242bd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 --2- 172.21.15.26:0/2593307443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1242ae9e0 0x55d1242bde80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 >> 172.21.15.26:0/2593307443 conn(0x55d124223260 msgr2=0x55d1242b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7faf7dffb700 0 client.27733 ms_handle_reset on v2:172.21.15.78:6838/3790769886 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 shutdown_connections 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 1 -- 172.21.15.26:0/2593307443 wait complete. 2023-10-06T21:35:29.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.680+0000 7fafafda2b80 20 client.27733 trim_cache size 0 max 16384 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "e19e9c1b-e47a-4787-9fed-7b02b4baa031": { 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e19e9c1b-e47a-4787-9fed-7b02b4baa031", 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-10-06T21:35:29.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:29.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:29.692 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:29.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 msgr2=0x7faf800a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 0x7faf800a3b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009f40 tx=0x7faf8c009ed0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 msgr2=0x7faf9419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00ef30 tx=0x7faf8c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 shutdown_connections 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf9410b480 0x7faf9419dcb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf941020b0 0x7faf94199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7faf800a1670 0x7faf800a3b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 --2- 172.21.15.26:0/1546689173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf941029d0 0x7faf94199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 >> 172.21.15.26:0/1546689173 conn(0x7faf940fb890 msgr2=0x7faf9410d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 shutdown_connections 2023-10-06T21:35:29.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.681+0000 7faf9c3ff700 1 -- 172.21.15.26:0/1546689173 wait complete. 2023-10-06T21:35:29.704 INFO:tasks.cephfs.filesystem:scrub status for tag:e19e9c1b-e47a-4787-9fed-7b02b4baa031 - {'path': '/', 'tag': 'e19e9c1b-e47a-4787-9fed-7b02b4baa031', 'options': 'recursive,force'} 2023-10-06T21:35:29.704 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-10-06T21:35:29.705 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-10-06T21:35:29.706 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-06T21:35:29.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:35:29.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 -- 172.21.15.26:0/647364966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81050b0 msgr2=0x7fbbe810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/647364966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81050b0 0x7fbbe810cc30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009f90 tx=0x7fbbe000b330 comp rx=0 tx=0).stop 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 -- 172.21.15.26:0/647364966 shutdown_connections 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/647364966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe81050b0 0x7fbbe810cc30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/647364966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe80fff90 0x7fbbe8100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/647364966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8100940 0x7fbbe8104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 -- 172.21.15.26:0/647364966 >> 172.21.15.26:0/647364966 conn(0x7fbbe80fb7f0 msgr2=0x7fbbe80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 -- 172.21.15.26:0/647364966 shutdown_connections 2023-10-06T21:35:29.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.945+0000 7fbbef1ff700 1 -- 172.21.15.26:0/647364966 wait complete. 2023-10-06T21:35:29.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.947+0000 7fbbef1ff700 1 Processor -- start 2023-10-06T21:35:29.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.947+0000 7fbbef1ff700 1 -- start start 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.947+0000 7fbbef1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 0x7fbbe81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.947+0000 7fbbef1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbef1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe819da80 0x7fbbe819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbef1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819e500 con 0x7fbbe81050b0 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbef1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819e670 con 0x7fbbe819da80 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbef1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819e830 con 0x7fbbe80fff90 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbed79c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe819da80 0x7fbbe819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50900/0 (socket says 172.21.15.26:50900) 2023-10-06T21:35:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 -- 172.21.15.26:0/2530970118 learned_addr learned my addr 172.21.15.26:0/2530970118 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbecf9b700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 0x7fbbe81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 -- 172.21.15.26:0/2530970118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 msgr2=0x7fbbe81990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 0x7fbbe81990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 -- 172.21.15.26:0/2530970118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe819da80 msgr2=0x7fbbe819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe819da80 0x7fbbe819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbe7fff700 1 -- 172.21.15.26:0/2530970118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009c40 con 0x7fbbe81050b0 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.948+0000 7fbbecf9b700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 0x7fbbe81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbe7fff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e970 tx=0x7fbbd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4009790 con 0x7fbbe81050b0 2023-10-06T21:35:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe81a5d90 con 0x7fbbe81050b0 2023-10-06T21:35:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd400c7c0 con 0x7fbbe81050b0 2023-10-06T21:35:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4005460 con 0x7fbbe81050b0 2023-10-06T21:35:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.949+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81a6210 con 0x7fbbe81050b0 2023-10-06T21:35:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.950+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8067cd0 con 0x7fbbe81050b0 2023-10-06T21:35:29.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.952+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fbbd400c930 con 0x7fbbe81050b0 2023-10-06T21:35:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.954+0000 7fbbe5ffb700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 0x7fbbd80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.954+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbbd4014070 con 0x7fbbe81050b0 2023-10-06T21:35:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.954+0000 7fbbecf9b700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 0x7fbbd80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.955+0000 7fbbecf9b700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 0x7fbbd80a3af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc005fd0 tx=0x7fbbdc009310 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:29.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:29.959+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbbd40964f0 con 0x7fbbe81050b0 2023-10-06T21:35:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.229+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbe819a430 con 0x7fbbe81050b0 2023-10-06T21:35:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.231+0000 7fbbe5ffb700 1 -- 172.21.15.26:0/2530970118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+23020 (secure 0 0 0) 0x7fbbd4099b10 con 0x7fbbe81050b0 2023-10-06T21:35:30.234 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:35:30.234 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":127,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:35:18.573659+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1157730691":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2246884211":"2023-10-07T21:10:19.013203+0000","172.21.15.26:6800/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3325517294":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3867812316":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:35:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 msgr2=0x7fbbd80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 0x7fbbd80a3af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc005fd0 tx=0x7fbbdc009310 comp rx=0 tx=0).stop 2023-10-06T21:35:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 msgr2=0x7fbbe8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e970 tx=0x7fbbd400ec80 comp rx=0 tx=0).stop 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 shutdown_connections 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbe819da80 0x7fbbe819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe80fff90 0x7fbbe81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fbbd80a1630 0x7fbbd80a3af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 --2- 172.21.15.26:0/2530970118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe81050b0 0x7fbbe8199620 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 >> 172.21.15.26:0/2530970118 conn(0x7fbbe80fb7f0 msgr2=0x7fbbe80fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.234+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 shutdown_connections 2023-10-06T21:35:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.235+0000 7fbbef1ff700 1 -- 172.21.15.26:0/2530970118 wait complete. 2023-10-06T21:35:30.261 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:35:30.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:35:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 -- 172.21.15.26:0/1931157596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c100940 msgr2=0x7f8c2c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 --2- 172.21.15.26:0/1931157596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c100940 0x7f8c2c104980 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009990 tx=0x7f8c14009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 -- 172.21.15.26:0/1931157596 shutdown_connections 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 --2- 172.21.15.26:0/1931157596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 --2- 172.21.15.26:0/1931157596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 0x7f8c2c100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 --2- 172.21.15.26:0/1931157596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c100940 0x7f8c2c104980 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 -- 172.21.15.26:0/1931157596 >> 172.21.15.26:0/1931157596 conn(0x7f8c2c0fb830 msgr2=0x7f8c2c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.509+0000 7f8c30dff700 1 -- 172.21.15.26:0/1931157596 shutdown_connections 2023-10-06T21:35:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.510+0000 7f8c30dff700 1 -- 172.21.15.26:0/1931157596 wait complete. 2023-10-06T21:35:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.511+0000 7f8c30dff700 1 Processor -- start 2023-10-06T21:35:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.511+0000 7f8c30dff700 1 -- start start 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 0x7f8c2c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 0x7f8c2c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19c540 con 0x7f8c2c19bac0 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19c6b0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c30dff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19c870 con 0x7f8c2c0fff90 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c2a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 0x7f8c2c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c23fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55388/0 (socket says 172.21.15.26:55388) 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c23fff700 1 -- 172.21.15.26:0/575847414 learned_addr learned my addr 172.21.15.26:0/575847414 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.512+0000 7f8c2ad9d700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 0x7f8c2c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 -- 172.21.15.26:0/575847414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 msgr2=0x7f8c2c197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 0x7f8c2c197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 -- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 msgr2=0x7f8c2c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 0x7f8c2c19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 -- 172.21.15.26:0/575847414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24009c40 con 0x7f8c2c1050b0 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c2ad9d700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 0x7f8c2c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:35:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c23fff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004740 tx=0x7f8c14009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1401d070 con 0x7f8c2c1050b0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c14004830 con 0x7f8c2c1050b0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14005310 con 0x7f8c2c1050b0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140095f0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.513+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c1a3fb0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.514+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c067c60 con 0x7f8c2c1050b0 2023-10-06T21:35:30.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.516+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f8c140054d0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.518+0000 7f8c237fe700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 0x7f8c180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.518+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8c140ce160 con 0x7f8c2c1050b0 2023-10-06T21:35:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.518+0000 7f8c2a59c700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 0x7f8c180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.519+0000 7f8c2a59c700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 0x7f8c180a3b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009c60 tx=0x7f8c1c009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.520+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8c140974a0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.751+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c2c100ac0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.752+0000 7f8c237fe700 1 -- 172.21.15.26:0/575847414 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c1409aac0 con 0x7f8c2c1050b0 2023-10-06T21:35:30.755 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:35:30.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 msgr2=0x7f8c180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 0x7f8c180a3b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009c60 tx=0x7f8c1c009260 comp rx=0 tx=0).stop 2023-10-06T21:35:30.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 msgr2=0x7f8c2c197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:30.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004740 tx=0x7f8c14009580 comp rx=0 tx=0).stop 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 shutdown_connections 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f8c180a1670 0x7f8c180a3b30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c2c19bac0 0x7f8c2c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c2c0fff90 0x7f8c2c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 --2- 172.21.15.26:0/575847414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c2c1050b0 0x7f8c2c197660 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 >> 172.21.15.26:0/575847414 conn(0x7f8c2c0fb830 msgr2=0x7f8c2c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 shutdown_connections 2023-10-06T21:35:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:30.755+0000 7f8c30dff700 1 -- 172.21.15.26:0/575847414 wait complete. 2023-10-06T21:35:30.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:35:31.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 -- 172.21.15.26:0/1968024298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 msgr2=0x7f614c10d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 --2- 172.21.15.26:0/1968024298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c10d1e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6140009990 tx=0x7f6140009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 -- 172.21.15.26:0/1968024298 shutdown_connections 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 --2- 172.21.15.26:0/1968024298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c10d7b0 0x7f614c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 --2- 172.21.15.26:0/1968024298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c0ffc00 0x7f614c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 --2- 172.21.15.26:0/1968024298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c10d1e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 -- 172.21.15.26:0/1968024298 >> 172.21.15.26:0/1968024298 conn(0x7f614c0fb830 msgr2=0x7f614c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 -- 172.21.15.26:0/1968024298 shutdown_connections 2023-10-06T21:35:31.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.016+0000 7f61515ff700 1 -- 172.21.15.26:0/1968024298 wait complete. 2023-10-06T21:35:31.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 Processor -- start 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 -- start start 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 0x7f614c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19e250 con 0x7f614c0ffc00 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19e3c0 con 0x7f614c1005b0 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.018+0000 7f61515ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19e580 con 0x7f614c10d7b0 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33584/0 (socket says 172.21.15.26:33584) 2023-10-06T21:35:31.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 -- 172.21.15.26:0/428357665 learned_addr learned my addr 172.21.15.26:0/428357665 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:31.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614a7fc700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614b7fe700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 0x7f614c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 -- 172.21.15.26:0/428357665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 msgr2=0x7f614c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 0x7f614c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 -- 172.21.15.26:0/428357665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 msgr2=0x7f614c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614affd700 1 -- 172.21.15.26:0/428357665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400095f0 con 0x7f614c0ffc00 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614b7fe700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 0x7f614c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.019+0000 7f614a7fc700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f614affd700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e9a0 tx=0x7f613c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c009790 con 0x7f614c0ffc00 2023-10-06T21:35:31.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c19e860 con 0x7f614c0ffc00 2023-10-06T21:35:31.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c010430 con 0x7f614c0ffc00 2023-10-06T21:35:31.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c018680 con 0x7f614c0ffc00 2023-10-06T21:35:31.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.020+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c1a60a0 con 0x7f614c0ffc00 2023-10-06T21:35:31.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.021+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c067c60 con 0x7f614c0ffc00 2023-10-06T21:35:31.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.024+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f613c0105a0 con 0x7f614c0ffc00 2023-10-06T21:35:31.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.025+0000 7f6133fff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 0x7f61340a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.026+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f613c014070 con 0x7f614c0ffc00 2023-10-06T21:35:31.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.026+0000 7f614a7fc700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 0x7f61340a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.027+0000 7f614a7fc700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 0x7f61340a3ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f614c19a880 tx=0x7f6140005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:31.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.031+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f613c052100 con 0x7f614c0ffc00 2023-10-06T21:35:31.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:30 smithi026 ceph-mon[98490]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:35:31.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:30 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2530970118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:35:31.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:30 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/575847414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:35:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:30 smithi078 ceph-mon[134896]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:35:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:30 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2530970118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:35:31.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:30 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/575847414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:35:31.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:30 smithi167 ceph-mon[134494]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 91 B/s wr, 5 op/s 2023-10-06T21:35:31.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:30 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2530970118' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:35:31.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:30 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/575847414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:35:31.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.338+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f614c1130f0 con 0x7f614c0ffc00 2023-10-06T21:35:31.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.339+0000 7f6133fff700 1 -- 172.21.15.26:0/428357665 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f613c0965d0 con 0x7f614c0ffc00 2023-10-06T21:35:31.342 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:35:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 msgr2=0x7f61340a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 0x7f61340a3ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f614c19a880 tx=0x7f6140005fb0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 msgr2=0x7f614c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e9a0 tx=0x7f613c00ecb0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 shutdown_connections 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f614c10d7b0 0x7f614c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f61340a15f0 0x7f61340a3ab0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f614c0ffc00 0x7f614c199270 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 --2- 172.21.15.26:0/428357665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f614c1005b0 0x7f614c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 >> 172.21.15.26:0/428357665 conn(0x7f614c0fb830 msgr2=0x7f614c10eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 shutdown_connections 2023-10-06T21:35:31.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.342+0000 7f61515ff700 1 -- 172.21.15.26:0/428357665 wait complete. 2023-10-06T21:35:31.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:35:31.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 -- 172.21.15.26:0/71672005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd54103cb0 msgr2=0x7efd54107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/71672005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd54103cb0 0x7efd54107cf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efd44009990 tx=0x7efd44009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 -- 172.21.15.26:0/71672005 shutdown_connections 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/71672005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd5410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/71672005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/71672005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd54103cb0 0x7efd54107cf0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 -- 172.21.15.26:0/71672005 >> 172.21.15.26:0/71672005 conn(0x7efd540feb40 msgr2=0x7efd54100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 -- 172.21.15.26:0/71672005 shutdown_connections 2023-10-06T21:35:31.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.606+0000 7efd5c3ff700 1 -- 172.21.15.26:0/71672005 wait complete. 2023-10-06T21:35:31.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.607+0000 7efd5c3ff700 1 Processor -- start 2023-10-06T21:35:31.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.607+0000 7efd5c3ff700 1 -- start start 2023-10-06T21:35:31.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd54199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5419db10 0x7efd5419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419e590 con 0x7efd54103300 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419e700 con 0x7efd5419db10 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5c3ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5419e8c0 con 0x7efd54108420 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33594/0 (socket says 172.21.15.26:33594) 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5999a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd54199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5a19b700 1 -- 172.21.15.26:0/4025105516 learned_addr learned my addr 172.21.15.26:0/4025105516 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5a99c700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5419db10 0x7efd5419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.608+0000 7efd5999a700 1 -- 172.21.15.26:0/4025105516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5419db10 msgr2=0x7efd5419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5999a700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5419db10 0x7efd5419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5999a700 1 -- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 msgr2=0x7efd541991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5999a700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5999a700 1 -- 172.21.15.26:0/4025105516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd440095f0 con 0x7efd54108420 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5a19b700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5999a700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd54199700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efd44000c00 tx=0x7efd44005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4401d070 con 0x7efd54108420 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd44021410 con 0x7efd54108420 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44017650 con 0x7efd54108420 2023-10-06T21:35:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.609+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541a5dc0 con 0x7efd54108420 2023-10-06T21:35:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.610+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd541a6190 con 0x7efd54108420 2023-10-06T21:35:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.611+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd54067c60 con 0x7efd54108420 2023-10-06T21:35:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.613+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efd44021580 con 0x7efd54108420 2023-10-06T21:35:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.615+0000 7efd4b7fe700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 0x7efd400a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.616+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7efd440cea80 con 0x7efd54108420 2023-10-06T21:35:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.616+0000 7efd5a19b700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 0x7efd400a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.616+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd44097e20 con 0x7efd54108420 2023-10-06T21:35:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.616+0000 7efd5a19b700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 0x7efd400a3ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd50011550 tx=0x7efd50009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.941+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd54103e00 con 0x7efd54108420 2023-10-06T21:35:31.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.941+0000 7efd4b7fe700 1 -- 172.21.15.26:0/4025105516 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd4409b440 con 0x7efd54108420 2023-10-06T21:35:31.944 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:35:31.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 msgr2=0x7efd400a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 0x7efd400a3ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd50011550 tx=0x7efd50009260 comp rx=0 tx=0).stop 2023-10-06T21:35:31.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 msgr2=0x7efd54199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:31.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd54199700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efd44000c00 tx=0x7efd44005740 comp rx=0 tx=0).stop 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 shutdown_connections 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd5419db10 0x7efd5419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efd400a15f0 0x7efd400a3ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54103300 0x7efd541991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 --2- 172.21.15.26:0/4025105516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd54108420 0x7efd54199700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 >> 172.21.15.26:0/4025105516 conn(0x7efd540feb40 msgr2=0x7efd54106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 shutdown_connections 2023-10-06T21:35:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:31.944+0000 7efd5c3ff700 1 -- 172.21.15.26:0/4025105516 wait complete. 2023-10-06T21:35:31.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T21:35:32.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:31 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/428357665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:35:32.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.210+0000 7f7d5fdff700 1 -- 172.21.15.26:0/25164621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58107030 msgr2=0x7f7d58107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:32.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.210+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/25164621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58107030 0x7f7d58107410 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009990 tx=0x7f7d54009ca0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.210+0000 7f7d5fdff700 1 -- 172.21.15.26:0/25164621 shutdown_connections 2023-10-06T21:35:32.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.210+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/25164621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58069990 0x7f7d5810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.210+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/25164621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58107030 0x7f7d58107410 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.211+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/25164621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58068fd0 0x7f7d58069450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.211+0000 7f7d5fdff700 1 -- 172.21.15.26:0/25164621 >> 172.21.15.26:0/25164621 conn(0x7f7d58077d60 msgr2=0x7f7d58078170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:32.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.211+0000 7f7d5fdff700 1 -- 172.21.15.26:0/25164621 shutdown_connections 2023-10-06T21:35:32.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.211+0000 7f7d5fdff700 1 -- 172.21.15.26:0/25164621 wait complete. 2023-10-06T21:35:32.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.212+0000 7f7d5fdff700 1 Processor -- start 2023-10-06T21:35:32.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.213+0000 7f7d5fdff700 1 -- start start 2023-10-06T21:35:32.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.213+0000 7f7d5fdff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 0x7f7d58196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:32.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5fdff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 0x7f7d581974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:32.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5fdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58107030 0x7f7d5819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:32.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5fdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5819bf80 con 0x7f7d58107030 2023-10-06T21:35:32.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5fdff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5819c0f0 con 0x7f7d58069990 2023-10-06T21:35:32.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5fdff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5819c2b0 con 0x7f7d58068fd0 2023-10-06T21:35:32.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 0x7f7d581974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:32.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 0x7f7d58196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:32.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5e39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58107030 0x7f7d5819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:32.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5db9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 0x7f7d58196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:46278/0 (socket says 172.21.15.26:46278) 2023-10-06T21:35:32.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5db9b700 1 -- 172.21.15.26:0/1580014778 learned_addr learned my addr 172.21.15.26:0/1580014778 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 -- 172.21.15.26:0/1580014778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 msgr2=0x7f7d58196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 0x7f7d58196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 -- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58107030 msgr2=0x7f7d5819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58107030 0x7f7d5819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.214+0000 7f7d5d39a700 1 -- 172.21.15.26:0/1580014778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540095f0 con 0x7f7d58069990 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d5d39a700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 0x7f7d581974e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e9f0 tx=0x7f7d4800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800c9c0 con 0x7f7d58069990 2023-10-06T21:35:32.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d48010430 con 0x7f7d58069990 2023-10-06T21:35:32.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5819c590 con 0x7f7d58069990 2023-10-06T21:35:32.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48018680 con 0x7f7d58069990 2023-10-06T21:35:32.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.215+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581a3dd0 con 0x7f7d58069990 2023-10-06T21:35:32.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.216+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58067440 con 0x7f7d58069990 2023-10-06T21:35:32.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.219+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7d48010860 con 0x7f7d58069990 2023-10-06T21:35:32.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.220+0000 7f7d4effd700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 0x7f7d440a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:35:32.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.220+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d48014070 con 0x7f7d58069990 2023-10-06T21:35:32.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.220+0000 7f7d5db9b700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 0x7f7d440a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:35:32.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.221+0000 7f7d5db9b700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 0x7f7d440a3b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:35:32.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.225+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7d48052100 con 0x7f7d58069990 2023-10-06T21:35:32.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:31 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/428357665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:35:32.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:31 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/428357665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:35:32.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.602+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f7d581982f0 con 0x7f7d58069990 2023-10-06T21:35:32.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.611+0000 7f7d4effd700 1 -- 172.21.15.26:0/1580014778 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+734409 (secure 0 0 0) 0x7f7d480964d0 con 0x7f7d58069990 2023-10-06T21:35:32.624 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:35:32.624 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:32.624 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2023-10-06T21:35:32.624 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 15306, 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3248613920 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2023-10-06T21:35:32.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3248613920 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/3248613920", 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2023-10-06T21:10:19.013490+0000", 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540138322906710015, 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.626 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 25336, 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540138322906710015, 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.628 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.629 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.632 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-10-06T21:35:32.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-10-06T21:35:32.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-10-06T21:35:32.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-10-06T21:35:32.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:35:32.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-10-06T21:35:32.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:35:32.649 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.650 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:35:32.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.653 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:32.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.656 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.659 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:32.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:35:32.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:32.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-10-06T21:35:32.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-10-06T21:35:32.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:35:32.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-10-06T21:35:32.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:35:32.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:35:32.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-10-06T21:35:32.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.687 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:35:32.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.691 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:35:32.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:32.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-10-06T21:35:32.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:35:32.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:32.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2023-10-06T21:35:32.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.710 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.711 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-10-06T21:35:32.712 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.715 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.717 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-10-06T21:35:32.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.725 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:35:32.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-10-06T21:35:32.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:35:32.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-10-06T21:35:32.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.737 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:35:32.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.741 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.745 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-10-06T21:35:32.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.753 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.755 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:35:32.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.762 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:35:32.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.770 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:35:32.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:35:32.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:35:32.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:35:32.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:35:32.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:35:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.793 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:35:32.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:35:32.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.803 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.804 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.809 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.813 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.814 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:35:32.815 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:35:32.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.825 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:32.826 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.828 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-10-06T21:35:32.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.833 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.834 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.835 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.837 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:35:32.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:32.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.848 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.855 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.856 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-10-06T21:35:32.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:35:32.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-10-06T21:35:32.866 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.869 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:32.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-10-06T21:35:32.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-10-06T21:35:32.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.880 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.886 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:32.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-10-06T21:35:32.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-10-06T21:35:32.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.897 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.899 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.900 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.907 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-10-06T21:35:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.916 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-10-06T21:35:32.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.920 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-10-06T21:35:32.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.928 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.929 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:35:32.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.932 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.933 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.935 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.936 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.940 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.944 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.947 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.948 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.953 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.954 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-10-06T21:35:32.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:35:32.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.961 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.964 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.966 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-10-06T21:35:32.970 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.974 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.977 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.982 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.985 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.987 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.990 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.991 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:32.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:32.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:32.994 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-10-06T21:35:32.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:32.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.002 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.008 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.009 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:35:33.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.013 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.015 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.017 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-10-06T21:35:33.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.028 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.032 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.035 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-10-06T21:35:33.036 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.042 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.047 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.048 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.051 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-10-06T21:35:33.053 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.055 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.058 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.061 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-10-06T21:35:33.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-10-06T21:35:33.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.075 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.091 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.092 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.097 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.100 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.102 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.105 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.108 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.110 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-10-06T21:35:33.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.114 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.120 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-10-06T21:35:33.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.123 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.126 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.128 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.129 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-10-06T21:35:33.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-10-06T21:35:33.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.134 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.137 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.139 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.140 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.141 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.142 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.144 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.147 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.150 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.152 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.154 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.158 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.159 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-10-06T21:35:33.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-10-06T21:35:33.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 25358, 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540138322906710015, 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.165 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.167 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.170 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.171 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-10-06T21:35:33.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.173 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.174 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-10-06T21:35:33.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.180 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-10-06T21:35:33.181 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.182 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.186 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-10-06T21:35:33.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.188 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:33.190 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.191 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.193 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.194 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.195 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-10-06T21:35:33.198 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.200 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.204 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.205 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-10-06T21:35:33.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:35:33.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.210 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2023-10-06T21:35:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.212 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.214 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-10-06T21:35:33.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.216 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-10-06T21:35:33.220 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:35:33.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-10-06T21:35:33.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:35:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:35:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-10-06T21:35:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.245 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.246 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.248 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-10-06T21:35:33.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.251 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-10-06T21:35:33.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-10-06T21:35:33.255 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:35:33.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.258 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-10-06T21:35:33.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-10-06T21:35:33.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.261 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.262 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:35:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.267 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.268 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.271 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.274 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-10-06T21:35:33.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.278 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:35:33.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.281 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.284 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.286 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.287 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.288 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:35:33.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:35:33.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.292 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.294 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.298 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:35:33.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:35:33.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.304 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.305 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:35:33.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:35:33.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.311 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:35:33.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.317 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.319 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.324 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:35:33.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.327 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.332 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.339 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.340 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:33.342 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.343 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-10-06T21:35:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.346 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:33.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.350 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.352 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.354 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-10-06T21:35:33.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.360 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:35:33.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.363 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.365 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.366 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.368 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.369 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-10-06T21:35:33.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.372 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-10-06T21:35:33.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.376 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.378 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.379 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.380 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.382 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.384 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-10-06T21:35:33.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.386 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.390 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.392 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.393 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.398 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.399 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.402 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-10-06T21:35:33.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.410 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.424 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.429 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.435 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.436 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.439 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.444 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.446 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.449 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.453 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.463 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.466 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.467 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.469 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-10-06T21:35:33.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-10-06T21:35:33.484 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.490 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:33.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:33.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.500 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.502 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.506 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-10-06T21:35:33.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.512 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.515 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.520 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.526 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.530 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.531 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:35:33.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.538 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.542 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.545 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-10-06T21:35:33.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-10-06T21:35:33.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.551 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.553 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-10-06T21:35:33.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.559 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.566 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.569 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.571 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.572 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.575 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.577 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.578 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.583 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-10-06T21:35:33.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.589 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:35:33.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.592 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:35:33.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.595 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-10-06T21:35:33.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-10-06T21:35:33.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.601 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.603 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.604 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-10-06T21:35:33.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-10-06T21:35:33.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.608 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.609 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.610 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.611 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.615 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.617 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.622 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.625 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2023-10-06T21:35:33.638 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "restful", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "snap_schedule" 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.639 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.641 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.642 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.644 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.645 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.649 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.655 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T21:35:33.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:33.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-10-06T21:35:33.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-10-06T21:35:33.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.663 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.665 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.667 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.668 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-10-06T21:35:33.672 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-10-06T21:35:33.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-10-06T21:35:33.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-10-06T21:35:33.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.682 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T21:35:33.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-10-06T21:35:33.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.687 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.689 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.691 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-06T21:35:33.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-10-06T21:35:33.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.696 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-10-06T21:35:33.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-10-06T21:35:33.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.706 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T21:35:33.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2023-10-06T21:35:33.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.716 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.717 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.721 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-10-06T21:35:33.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-10-06T21:35:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.729 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.730 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-10-06T21:35:33.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-10-06T21:35:33.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-10-06T21:35:33.737 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-10-06T21:35:33.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-10-06T21:35:33.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T21:35:33.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.745 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.746 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.748 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.750 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-10-06T21:35:33.753 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.755 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T21:35:33.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.761 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.766 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T21:35:33.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-10-06T21:35:33.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T21:35:33.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.778 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.781 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T21:35:33.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.788 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-10-06T21:35:33.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T21:35:33.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T21:35:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.800 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-10-06T21:35:33.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.809 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.810 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.811 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.813 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.815 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.819 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-10-06T21:35:33.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:33.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-10-06T21:35:33.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.828 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.830 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.832 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.835 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-10-06T21:35:33.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.837 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T21:35:33.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.842 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-10-06T21:35:33.847 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-10-06T21:35:33.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.856 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.857 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.861 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-10-06T21:35:33.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-10-06T21:35:33.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-10-06T21:35:33.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.869 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.874 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.875 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.878 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.880 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-06T21:35:33.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.882 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.887 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.890 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.891 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-10-06T21:35:33.897 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-10-06T21:35:33.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-10-06T21:35:33.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.903 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.907 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.909 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.914 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.915 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.918 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.924 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.925 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-10-06T21:35:33.928 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.929 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.930 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.935 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.936 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.939 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-10-06T21:35:33.941 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.942 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.945 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-10-06T21:35:33.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.956 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.960 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-06T21:35:33.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.963 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.966 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-10-06T21:35:33.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.969 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T21:35:33.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-10-06T21:35:33.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-10-06T21:35:33.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.977 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.991 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.993 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:33.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-10-06T21:35:33.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.996 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:33.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:33.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:33.999 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-10-06T21:35:34.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.006 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.009 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.016 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-10-06T21:35:34.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-10-06T21:35:34.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.027 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.030 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.032 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T21:35:34.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.036 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.039 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.048 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.049 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-10-06T21:35:34.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.056 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.059 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-10-06T21:35:34.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.066 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-10-06T21:35:34.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.075 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.077 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.080 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-10-06T21:35:34.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.089 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.091 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T21:35:34.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-10-06T21:35:34.101 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.102 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.103 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-10-06T21:35:34.108 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.109 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-10-06T21:35:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2023-10-06T21:35:34.114 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2023-10-06T21:35:34.115 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-10-06T21:35:34.116 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-10-06T21:35:34.117 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 88, 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2023-10-06T21:35:34.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2991291123 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1592297392 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.120 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1069856805 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1832861977 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 209204160 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-10-06T21:35:34.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3451651414 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-10-06T21:35:34.123 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:34.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:32 smithi167 ceph-mon[134494]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:35:34.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:32 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4025105516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:35:34.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:32 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1580014778' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:35:34.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:32 smithi078 ceph-mon[134896]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:35:34.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:32 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4025105516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:35:34.125 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:32 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1580014778' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:35:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 msgr2=0x7f7d440a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 0x7f7d440a3b70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006010 tx=0x7f7d5400b260 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 msgr2=0x7f7d581974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 0x7f7d581974e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e9f0 tx=0x7f7d4800ed00 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 shutdown_connections 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7d440a16b0 0x7f7d440a3b70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d58107030 0x7f7d5819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58068fd0 0x7f7d58196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 --2- 172.21.15.26:0/1580014778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d58069990 0x7f7d581974e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 >> 172.21.15.26:0/1580014778 conn(0x7f7d58077d60 msgr2=0x7f7d580febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 shutdown_connections 2023-10-06T21:35:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:35:32.618+0000 7f7d5fdff700 1 -- 172.21.15.26:0/1580014778 wait complete. 2023-10-06T21:35:34.132 DEBUG:tasks.check_counter:Getting stats from a 2023-10-06T21:35:34.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-06T21:35:34.133 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:32 smithi026 ceph-mon[98490]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-10-06T21:35:34.133 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:32 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4025105516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:35:34.133 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:32 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1580014778' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-06T21:35:34.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:34 smithi078 ceph-mon[134896]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:35:35.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:34 smithi167 ceph-mon[134494]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:35.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:34 smithi026 ceph-mon[98490]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:36.772 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4743673, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4747850, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 774878845, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 770244477, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 14, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 254.092108598, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 113.418703214, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 171.420676454, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.101769427, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4747850, 2023-10-06T21:35:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 110.914536887, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000023361 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 9473903, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.106675333, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000116 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1875536, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 287904 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 481, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 815, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 228214, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 1129909, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 25, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.088213165, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.039998841, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.024858211, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.014255831, 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 812, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.261678332, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000322263 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 35, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000005426, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000155 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3360, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 758688 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 17964, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 19433, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 6170534, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 9404467, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 10, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 1.519260472, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.674642362, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 1.093858705, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.018953136, 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 19431, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.006532501, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000206192 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 33892, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.013558494, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000400 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 14976 2023-10-06T21:35:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4969, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2.829692582, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000569469 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 749, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.037851154, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000050535 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 4866, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 4648, 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-10-06T21:35:36.780 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4648, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 123.835069421, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.026642656 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 218, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 9, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 27, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 2, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 1771, 2023-10-06T21:35:36.781 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 576, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 1159, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 36, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 912, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 1203, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 9, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 9, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 26, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 29217, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 21062, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 193, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 162, 2023-10-06T21:35:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 5, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 5155, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 19, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 818, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 24, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 1180, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 306, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 14, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 1008, 2023-10-06T21:35:36.783 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 1192328189, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 6, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 306, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 5, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 5, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 301, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 228, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 1218, 2023-10-06T21:35:36.784 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 2168, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 8, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 2541, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 866, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 236, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 1704 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-10-06T21:35:36.785 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 16, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 398, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 237, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 466, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 390, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 14, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 448, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 150, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 1330, 2023-10-06T21:35:36.786 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 62, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 19, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 2, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-10-06T21:35:36.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 4553, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 4553, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 8, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-10-06T21:35:36.788 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 8, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 2, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 18565610, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2304, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 12.238360010, 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005311788 2023-10-06T21:35:36.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 490, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 1157, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 667, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 107, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 225, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 118, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 1783, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 3206, 2023-10-06T21:35:36.790 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 1423, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 9, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 788, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 779, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 63256, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 198960 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 19049, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 3524, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 4866, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 560, 2023-10-06T21:35:36.791 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 5637, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 172, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.292453439, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001700310 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 134, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.011307266, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.007547069 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.792 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 290, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2.429411516, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.008377281 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1964, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 74.192907643, 2023-10-06T21:35:36.793 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.037776429 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000092553, 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000092553 2023-10-06T21:35:36.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.795 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.796 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.796 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-10-06T21:35:36.796 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.796 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 161, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.697316108, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004331155 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-10-06T21:35:36.798 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 93, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.377818338, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004062562 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1527, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 43.206141791, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.028294788 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.800 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-10-06T21:35:36.801 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.803 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 306, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.627620767, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005319022 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:35:36.804 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 2, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 4, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-10-06T21:35:36.805 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 271, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 135, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 1491, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:36.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:36.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:36.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 832071, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 972, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 88, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 1, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-10-06T21:35:36.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8128, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 169, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 3013601, 2023-10-06T21:35:36.810 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 110192, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 3374, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 14630563, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 27, 2023-10-06T21:35:36.811 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 3374, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3374, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 24.630650515, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.007300133 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3374, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3712 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 3374, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 423, 2023-10-06T21:35:36.812 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 2951, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 10, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 108, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 2412, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 153, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 1, 2023-10-06T21:35:36.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 166, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 306, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 20, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-10-06T21:35:36.814 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-10-06T21:35:36.815 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 127, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-10-06T21:35:36.816 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 42, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 365, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 18, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 153 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:35:36.817 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 9, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 9, 2023-10-06T21:35:36.818 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 150, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 72, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 4762116, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 352773846, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 4762116, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:36.819 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 4762115, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 352773774, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-10-06T21:35:36.820 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 3374, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 14645889, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 3374, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 14645889, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.821 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-10-06T21:35:36.822 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 3374, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 3374, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 3374, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 3374, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.823 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 150, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 15470, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 150, 2023-10-06T21:35:36.824 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 108, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 15470, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x5595e1f906e0": { 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:36.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 4553, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 14371306, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 4553, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 2304, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 14371306, 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:36.826 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:36.827 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:36.827 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:36.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:36.827 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:37.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:36 smithi026 ceph-mon[98490]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:37.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:36 smithi078 ceph-mon[134896]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:37.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:36 smithi167 ceph-mon[134494]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:37.298 INFO:tasks.check_counter:Daemon mds.a mds.exported=19 2023-10-06T21:35:37.298 INFO:tasks.check_counter:Daemon mds.a mds.imported=24 2023-10-06T21:35:37.299 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=16 2023-10-06T21:35:37.299 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=398 2023-10-06T21:35:37.299 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2023-10-06T21:35:37.299 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:35:37.299 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:35:37.299 DEBUG:tasks.check_counter:Getting stats from d 2023-10-06T21:35:37.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-06T21:35:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:35:39.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:38 smithi026 ceph-mon[98490]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:39.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:38 smithi078 ceph-mon[134896]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:39.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:38 smithi167 ceph-mon[134494]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:40.132 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:40.132 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.003571893, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000671422, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:40.133 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.134 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 458, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 445, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1630336, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 267936, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.082728113, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.032089511, 2023-10-06T21:35:40.135 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.046679980, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.011226613, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 445, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.077515516, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000174192 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:40.136 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1630336, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 267936 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 3, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 339, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 480, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 289776, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:40.137 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.031195816, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.023029159, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.003423802, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 336, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.051254694, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000152543 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:40.138 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 289776 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:40.139 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:40.140 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:40.142 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148714, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-10-06T21:35:40.143 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:40.144 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 457, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1563152, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 457, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:40.145 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 457, 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1563152, 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:40.146 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:40.613 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-10-06T21:35:40.614 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-10-06T21:35:40.614 DEBUG:tasks.check_counter:Getting stats from g 2023-10-06T21:35:40.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-06T21:35:41.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:40 smithi078 ceph-mon[134896]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:41.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:40 smithi167 ceph-mon[134494]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:41.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:40 smithi026 ceph-mon[98490]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:35:42.997 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 451, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 440, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1626496, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 265616, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.085282443, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.031784417, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.047636539, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.011097033, 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 439, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.080509836, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000183393 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1626496, 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 265616 2023-10-06T21:35:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 335, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 212192, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.030052006, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.022121622, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.003298437, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 332, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.049871387, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000150215 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 212192 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3024, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-10-06T21:35:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001386732, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000085125, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001245286, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001239703, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000413234 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3024, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-10-06T21:35:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-10-06T21:35:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 150008, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 31, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 453, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1562735, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 453, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 453, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1562735, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:43.011 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:43.018 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:42 smithi026 ceph-mon[98490]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:43.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:42 smithi078 ceph-mon[134896]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:43.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:42 smithi167 ceph-mon[134494]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-10-06T21:35:43.482 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-10-06T21:35:43.483 DEBUG:tasks.check_counter:Getting stats from j 2023-10-06T21:35:43.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-06T21:35:44.287 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-06T21:35:44.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:44 smithi026 ceph-mon[98490]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:44.870 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:44 smithi078 ceph-mon[134896]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:45.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:44 smithi167 ceph-mon[134494]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 460, 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 447, 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1630880, 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 269184, 2023-10-06T21:35:45.962 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.086460101, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.031746639, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.048901211, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.011234127, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 447, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.070859995, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000158523 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:45.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1630880, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 269184 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 340, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2023-10-06T21:35:45.964 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 213072, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.030250675, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.022724337, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002702544, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 337, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.065294975, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000193753 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.965 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 213072 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2023-10-06T21:35:45.966 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.000799741, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000636187, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.967 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.968 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:45.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:45.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:45.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 150025, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 32, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-10-06T21:35:45.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-10-06T21:35:45.973 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 458, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1563385, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 458, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 458, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1563385, 2023-10-06T21:35:45.974 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-10-06T21:35:45.975 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-10-06T21:35:46.446 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-10-06T21:35:46.447 DEBUG:tasks.check_counter:Getting stats from b 2023-10-06T21:35:46.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-06T21:35:46.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:46 smithi078 ceph-mon[134896]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:47.092 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:35:47.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:46 smithi167 ceph-mon[134494]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:47.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:46 smithi026 ceph-mon[98490]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:48.638 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-06T21:35:48.638 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4813708, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4811816, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 764629081, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 799250223, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 280, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 120.945411452, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 45.852328732, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 84.046312260, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.133485598, 2023-10-06T21:35:48.639 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4811819, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 66.672877242, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000013856 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9615105, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.050790759, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000109 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 45232, 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 1408 2023-10-06T21:35:48.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3532415, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 3533351, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 561640695, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 592872056, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 11, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 131.515980539, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 57.117515383, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 92.277242168, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.132942118, 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:48.641 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3533350, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 64.623221024, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000018289 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7057942, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.892465548, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000126 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 43456, 2023-10-06T21:35:48.642 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 770864 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 12245, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 12292, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 5950909, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 12492343, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 27, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 9, 2023-10-06T21:35:48.643 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.973497715, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.241012455, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.374724506, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.415905204, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 12285, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.047570869, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000166672 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 9931, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002273261, 2023-10-06T21:35:48.644 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000228 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 2733520, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 588256 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:48.645 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7264, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 3.547979243, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000488433 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 283, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.016600846, 2023-10-06T21:35:48.646 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000058660 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 3469, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 3096, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3096, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 140.149962925, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.045268075 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2023-10-06T21:35:48.647 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 373, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 14, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 913, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 3, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 414, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 141, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 260, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 13, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 193, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 3007, 2023-10-06T21:35:48.648 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 44, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 44, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 16, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 9871, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 7354, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 351, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 155, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 5, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 1131, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2023-10-06T21:35:48.649 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 51, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 2585, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 50, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 1525, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 2348, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 26, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 1008, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 1192328189, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 6, 2023-10-06T21:35:48.650 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 2348, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 228, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 8, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 220, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 2120, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 32, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 573, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 2063, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:35:48.651 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 5, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 2459, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 1395, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 107, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 477 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.652 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 95, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 91, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 47, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 213, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 379, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 6, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2023-10-06T21:35:48.653 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 131, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 63, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 300, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 16, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:35:48.654 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 8, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 51, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 3, 2023-10-06T21:35:48.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 5408, 2023-10-06T21:35:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 5155, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 5155, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 253, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 33, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 79, 2023-10-06T21:35:48.657 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 78, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 78, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 1, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 1, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 1, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 34553225, 2023-10-06T21:35:48.658 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 36430772, 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2452, 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 40.143304051, 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.016371657 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.659 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 234, 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 2799, 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 2565, 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 109, 2023-10-06T21:35:48.660 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 166, 2023-10-06T21:35:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 57, 2023-10-06T21:35:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 417, 2023-10-06T21:35:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 3992, 2023-10-06T21:35:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 3575, 2023-10-06T21:35:48.661 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 44, 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 1385, 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 1341, 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 72172, 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 207152 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 7041, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 1314, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 3469, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 629, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 2140, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1083, 2023-10-06T21:35:48.663 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 44.646372257, 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.041224720 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 337, 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 5.054985952, 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.014999958 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2023-10-06T21:35:48.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 69, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.040011999, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.015072637 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 375, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6.784390520, 2023-10-06T21:35:48.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.018091708 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000224248, 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000224248 2023-10-06T21:35:48.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.667 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 25, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004792597, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000191703 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 35, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.232470282, 2023-10-06T21:35:48.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.006642008 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 25, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.263721582, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.010548863 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.669 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 593, 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.632396982, 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001066436 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 68, 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 6.248180384, 2023-10-06T21:35:48.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.091885005 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 367, 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 74.459457291, 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.202886804 2023-10-06T21:35:48.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.672 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2023-10-06T21:35:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 19, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.543654960, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.028613418 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2023-10-06T21:35:48.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001100052, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001100052 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.675 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.676 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.018612449, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003102074 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 92, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.219591370, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002386862 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 5, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 3, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 2, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 28, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 14, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1514, 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2023-10-06T21:35:48.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:48.680 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:48.681 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:48.682 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 656748, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 595, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 616, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 7, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:48.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8128, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 169, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:48.684 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2023-10-06T21:35:48.685 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 1335243, 2023-10-06T21:35:48.685 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 44441, 2023-10-06T21:35:48.685 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:48.685 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2023-10-06T21:35:48.685 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2023-10-06T21:35:48.686 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 8198, 2023-10-06T21:35:48.687 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 37880771, 2023-10-06T21:35:48.687 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 37, 2023-10-06T21:35:48.687 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 8198, 2023-10-06T21:35:48.687 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2023-10-06T21:35:48.687 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8198, 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 156.608489879, 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.019103255 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2023-10-06T21:35:48.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8198, 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 12552 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 8198, 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 2412, 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 5786, 2023-10-06T21:35:48.689 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2023-10-06T21:35:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2023-10-06T21:35:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 842, 2023-10-06T21:35:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 1208, 2023-10-06T21:35:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 42, 2023-10-06T21:35:48.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 2486, 2023-10-06T21:35:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 318, 2023-10-06T21:35:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2023-10-06T21:35:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2023-10-06T21:35:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2023-10-06T21:35:48.691 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 3, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 604, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 2364, 2023-10-06T21:35:48.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 2416, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2023-10-06T21:35:48.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2023-10-06T21:35:48.694 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2023-10-06T21:35:48.695 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2023-10-06T21:35:48.696 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 127, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 43, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2023-10-06T21:35:48.697 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 2053, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 28, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 188 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.698 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2023-10-06T21:35:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2023-10-06T21:35:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2023-10-06T21:35:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2023-10-06T21:35:48.700 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 22, 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 10, 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 63, 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.701 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 68, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 8358367, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 579958068, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:48.702 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 8358367, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 8358366, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 579958000, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.703 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:48.704 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 8198, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 37887126, 2023-10-06T21:35:48.705 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 8198, 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 37887126, 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:48.707 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 8198, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 8198, 2023-10-06T21:35:48.708 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 8198, 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 8198, 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.710 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:35:48.710 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:48.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:48.710 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 63, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 6579, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 63, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 34, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6579, 2023-10-06T21:35:48.711 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x559955bce6e0": { 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 5408, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 32236468, 2023-10-06T21:35:48.712 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 5408, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2452, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 32236468, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:48.713 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-06T21:35:49.058 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:48 smithi078 ceph-mon[134896]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:49.059 INFO:tasks.check_counter:Daemon mds.b mds.exported=51 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds.imported=50 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=95 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=91 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=25 2023-10-06T21:35:49.060 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2023-10-06T21:35:49.060 DEBUG:tasks.check_counter:Getting stats from e 2023-10-06T21:35:49.060 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-06T21:35:49.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:48 smithi167 ceph-mon[134494]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:49.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:48 smithi026 ceph-mon[98490]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:49.698 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:35:51.158 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:50 smithi078 ceph-mon[134896]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:51.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:50 smithi167 ceph-mon[134494]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:51.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:50 smithi026 ceph-mon[98490]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1295, 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1297, 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 5938697, 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 402809, 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2023-10-06T21:35:52.435 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.093376560, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.027174330, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.025340941, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.031853199, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1295, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.135707103, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000104793 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.436 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2263, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2208, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 10688432, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 810625, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.197334844, 2023-10-06T21:35:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.049872145, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.080990846, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.064914982, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2207, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.342555273, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000155213 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1859264, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 276944 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2172, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2508, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4881343, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1426266, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2023-10-06T21:35:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.195978921, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.057772739, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.048090123, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.072686060, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2506, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.541650297, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000216141 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3344, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 752640 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:52.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 8742, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.214339167, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000024518 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 1643, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 893, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 656, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 94, 2023-10-06T21:35:52.444 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 366, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 15, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2023-10-06T21:35:52.445 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:35:52.446 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2023-10-06T21:35:52.447 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2023-10-06T21:35:52.448 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:35:52.449 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2023-10-06T21:35:52.450 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 4065, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 8, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2023-10-06T21:35:52.451 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 22380347, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 22379773, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 4065 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1475, 2023-10-06T21:35:52.452 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 2797, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 1322, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 70, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 128, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 58, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 1643, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 3355, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 1712, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 58676, 2023-10-06T21:35:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 207152 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.455 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.456 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.457 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.458 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2023-10-06T21:35:52.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2023-10-06T21:35:52.463 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.464 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2023-10-06T21:35:52.465 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2023-10-06T21:35:52.467 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 2, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 3383, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:52.468 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:52.469 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:52.470 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 264970, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 1627, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:52.471 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8128, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 169, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 4640956, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 46998, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2023-10-06T21:35:52.472 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 5177, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 5177, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5177, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 25.930591372, 2023-10-06T21:35:52.473 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.005008806 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5177, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 5178 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 5177, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 5177, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2023-10-06T21:35:52.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 3169, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 2006, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2023-10-06T21:35:52.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2023-10-06T21:35:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2023-10-06T21:35:52.477 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 127, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 42, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 7, 2023-10-06T21:35:52.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 8, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 1, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2023-10-06T21:35:52.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.480 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 5726, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 20962271, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 5726, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 5726, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 20962271, 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:52.481 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:52.482 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 5177, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 18199022, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 5177, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 18199022, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2023-10-06T21:35:52.483 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 5177, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 5177, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 5177, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 5177, 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:52.484 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:52.485 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x5648752e6960": { 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:52.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:52.487 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:52.488 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-06T21:35:52.800 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:35:52.801 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:35:52.801 DEBUG:tasks.check_counter:Getting stats from h 2023-10-06T21:35:52.801 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-06T21:35:53.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:52 smithi078 ceph-mon[134896]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:53.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:52 smithi167 ceph-mon[134494]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:53.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:52 smithi026 ceph-mon[98490]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:53.453 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:35:54.736 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:54 smithi078 ceph-mon[134896]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:55.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:54 smithi167 ceph-mon[134494]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:55.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:54 smithi026 ceph-mon[98490]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:35:55.492 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-06T21:35:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2640, 2023-10-06T21:35:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2949, 2023-10-06T21:35:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1406037, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 7214601, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 18, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.156285830, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.054976951, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.036268855, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.049650574, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2947, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.233757776, 2023-10-06T21:35:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000079320 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 17, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000001966, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000115 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 5264, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 724896 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 8859, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 11025, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4727479, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 13693137, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 20, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 7, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.519494960, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.204053063, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.283054401, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.085872478, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 11022, 2023-10-06T21:35:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.665634800, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000151119 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 6219, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002044375, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000328 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1870560, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 290368 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3550857, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 3548815, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 583531385, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 571519673, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 11, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 132.698521203, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 56.533457921, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 95.317904759, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.119395934, 2023-10-06T21:35:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3548815, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 46.144555316, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000013002 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7090583, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.694635201, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000097 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 140624 2023-10-06T21:35:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7918, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.141908255, 2023-10-06T21:35:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000270511 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 242, 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.011552310, 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000047736 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2023-10-06T21:35:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 2993, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 2373, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2373, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 80.135613853, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.033769748 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 620, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 14, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 1, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 43, 2023-10-06T21:35:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 4, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 1598, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 390, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 1169, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 39, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 864, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 2347, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 599, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 599, 2023-10-06T21:35:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 26, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 19455, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 14736, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 602, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 317, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 3, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 2993, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 49, 2023-10-06T21:35:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 1621, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 45, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 2319, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 4754, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 12, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 1008, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 1192328189, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 6, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 4754, 2023-10-06T21:35:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 24, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 24, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 4730, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 123, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 1090, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 1785, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 1, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:35:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 2536, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 930, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 169, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 676 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 151, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 134, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 77, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 403, 2023-10-06T21:35:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 314, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 11, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 184, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 61, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 667, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 58, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:35:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 50, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 4, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 1, 2023-10-06T21:35:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 4065, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 4065, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 8, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 8, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2023-10-06T21:35:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 22380347, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 0, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1931, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 21.718576194, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.011247320 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 0 2023-10-06T21:35:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 1459, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 3015, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 1556, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 108, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 178, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 70, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 1614, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 4050, 2023-10-06T21:35:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 2436, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 599, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 1783, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 1184, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 68612, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 198960 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 12349, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 3722, 2023-10-06T21:35:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 2993, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 261, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 5705, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 60, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.093937639, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001565627 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 152, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.512933868, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003374564 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 130, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.958349961, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.015064230 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 869, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 18.068887885, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.020792736 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 134, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.718194387, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.005359659 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 109, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2.087221310, 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.019148819 2023-10-06T21:35:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 781, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 54.718992939, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.070062731 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2023-10-06T21:35:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004073777, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004073777 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2023-10-06T21:35:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 137, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.973022087, 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.014401621 2023-10-06T21:35:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2023-10-06T21:35:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 2, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 225, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 112, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1510, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 876738, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 1840, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8128, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 169, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 5198482, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 92472, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2023-10-06T21:35:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 7158, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 18579855, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 27, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 7158, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7158, 2023-10-06T21:35:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 35.551439726, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.004966672 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7158, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 7491 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 7158, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 4800, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 2358, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2023-10-06T21:35:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 10, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 31, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 1962, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 93, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 1, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 80, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2023-10-06T21:35:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 4757, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 20, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2023-10-06T21:35:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 2, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2023-10-06T21:35:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 127, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 42, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 12, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 12, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2023-10-06T21:35:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 375, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 30, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 130 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 1, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 2 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 10, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 10, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 61, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 3562352, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 269110189, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 3562352, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3562352, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 269110189, 2023-10-06T21:35:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 7158, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 18585980, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 7158, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 18585980, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 7158, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 7158, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 7158, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 7158, 2023-10-06T21:35:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 61, 2023-10-06T21:35:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 6353, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 61, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 31, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 6353, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x557f463086e0": { 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 4065, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 18186043, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 4065, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 1931, 2023-10-06T21:35:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 18186043, 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:55.549 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds.exported=49 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds.imported=45 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=151 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=134 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:35:56.262 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:35:56.263 DEBUG:tasks.check_counter:Getting stats from k 2023-10-06T21:35:56.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-06T21:35:56.888 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-06T21:35:57.098 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:56 smithi078 ceph-mon[134896]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:57.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:56 smithi167 ceph-mon[134494]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:57.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:56 smithi026 ceph-mon[98490]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 464, 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 451, 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1631968, 2023-10-06T21:35:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 271664, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.062392550, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.019688881, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.037540128, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.010644157, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 451, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.101605318, 2023-10-06T21:35:58.212 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000225288 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1631968, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 271664 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.213 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 3, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 344, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 480, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 290656, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 6, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.014206834, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.010005547, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001502953, 2023-10-06T21:35:58.214 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 341, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.058053643, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000170245 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:58.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 290656 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2023-10-06T21:35:58.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000386761, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000314390, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2023-10-06T21:35:58.218 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:35:58.219 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2023-10-06T21:35:58.220 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2023-10-06T21:35:58.221 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:35:58.222 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148714, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2023-10-06T21:35:58.223 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2023-10-06T21:35:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 463, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1563932, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 463, 2023-10-06T21:35:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 463, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1563932, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2023-10-06T21:35:58.226 INFO:teuthology.orchestra.run.smithi078.stdout:} 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-10-06T21:35:58.543 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-10-06T21:35:58.544 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-10-06T21:35:58.544 DEBUG:tasks.check_counter:Getting stats from c 2023-10-06T21:35:58.544 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-06T21:35:59.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:35:58 smithi167 ceph-mon[134494]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:59.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:35:58 smithi078 ceph-mon[134896]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:35:59.124 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:35:59.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:35:58 smithi026 ceph-mon[98490]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-10-06T21:36:00.361 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T21:36:00.361 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:36:00.361 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4297, 2023-10-06T21:36:00.361 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 4297, 2023-10-06T21:36:00.361 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 13807849, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1335578, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.388997998, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.110436478, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.100592030, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.146597446, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4296, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.812144312, 2023-10-06T21:36:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000189046 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3024, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1482, 2023-10-06T21:36:00.363 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1427, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3887678, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 568104, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.162139846, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.050602608, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.067952491, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.042248474, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1426, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.264669870, 2023-10-06T21:36:00.364 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000185602 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1859536, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 277568 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4, 2023-10-06T21:36:00.365 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 343, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 748, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 841895, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.018239927, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012060336, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002483531, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000027883, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 340, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.088730370, 2023-10-06T21:36:00.366 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000260971 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 841584 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-10-06T21:36:00.367 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8717, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.238485605, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000027358 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-10-06T21:36:00.368 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.369 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 1364, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 177, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 1158, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 29, 2023-10-06T21:36:00.370 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 1111, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 9, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 21, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-10-06T21:36:00.371 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2023-10-06T21:36:00.372 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:36:00.373 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-10-06T21:36:00.374 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-10-06T21:36:00.375 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-10-06T21:36:00.376 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 4553, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 8, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2023-10-06T21:36:00.377 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 18565610, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 18565421, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 4553 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-10-06T21:36:00.378 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 450, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 1159, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 709, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 36, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 57, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 21, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 1364, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 2249, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 885, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 51448, 2023-10-06T21:36:00.379 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207152 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.380 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.381 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.382 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.383 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.384 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.385 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.386 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.387 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.388 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.389 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-10-06T21:36:00.390 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.391 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.392 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 2, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 4, 2023-10-06T21:36:00.393 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 2, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 3, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 4299, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:36:00.394 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:36:00.395 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:36:00.396 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:36:00.397 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 216176, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 737, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8128, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 169, 2023-10-06T21:36:00.398 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 2296655, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 67434, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-10-06T21:36:00.399 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 5228, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 5228, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5228, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 13.199763415, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002524820 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-10-06T21:36:00.400 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5228, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 5229 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 5228, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 5228, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-10-06T21:36:00.401 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 3209, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 2017, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-10-06T21:36:00.402 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:36:00.403 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-10-06T21:36:00.404 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 127, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 42, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 5, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 6, 2023-10-06T21:36:00.405 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 1, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.406 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:36:00.407 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 5779, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 17145209, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 5779, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 5779, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 17145209, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.408 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:00.409 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 5228, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 14371311, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 5228, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 14371311, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:00.410 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 5228, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 5228, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 5228, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 5228, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.411 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:00.412 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55a1f43ec960": { 2023-10-06T21:36:00.413 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:00.414 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:00.415 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:00.415 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.415 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:00.415 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:36:00.769 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:36:00.769 DEBUG:tasks.check_counter:Getting stats from f 2023-10-06T21:36:00.770 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-06T21:36:01.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:00 smithi167 ceph-mon[134494]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:36:01.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:00 smithi078 ceph-mon[134896]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:36:01.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:00 smithi026 ceph-mon[98490]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-10-06T21:36:01.333 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 4, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3024, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 448, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001516809, 2023-10-06T21:36:02.564 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000035781, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000570369, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000796038, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000265346 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:02.565 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3024, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 458, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 447, 2023-10-06T21:36:02.566 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1628400, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 269968, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.072175804, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.024281387, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.039580261, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.012547588, 2023-10-06T21:36:02.567 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 446, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.082922486, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000185924 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.568 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1628400, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 269968 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 342, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 480, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 290304, 2023-10-06T21:36:02.569 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.018468857, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013033355, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.001460897, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 339, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.083855262, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000247360 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.570 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 290304 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.571 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:36:02.572 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:36:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-10-06T21:36:02.574 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148692, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 30, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-10-06T21:36:02.575 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:36:02.576 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:36:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-10-06T21:36:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-10-06T21:36:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2023-10-06T21:36:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2023-10-06T21:36:02.577 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-10-06T21:36:02.578 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-10-06T21:36:02.578 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-10-06T21:36:02.578 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-10-06T21:36:02.578 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:02.579 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:36:02.579 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:02.579 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:02.579 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:02.579 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 461, 2023-10-06T21:36:02.580 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1563672, 2023-10-06T21:36:02.580 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:02.580 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 461, 2023-10-06T21:36:02.580 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:02.580 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 461, 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1563672, 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:02.581 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T21:36:02.886 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:02 smithi167 ceph-mon[134494]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:02.888 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2023-10-06T21:36:02.888 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2023-10-06T21:36:02.888 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2023-10-06T21:36:02.888 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2023-10-06T21:36:02.888 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2023-10-06T21:36:02.889 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2023-10-06T21:36:02.889 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2023-10-06T21:36:02.889 DEBUG:tasks.check_counter:Getting stats from i 2023-10-06T21:36:02.889 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-06T21:36:03.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:02 smithi078 ceph-mon[134896]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:03.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:02 smithi026 ceph-mon[98490]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:03.448 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:36:04.696 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2367, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2369, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4902197, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 736147, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.235534464, 2023-10-06T21:36:04.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.062072585, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.060305705, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.091093883, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2367, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.457654517, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000193347 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.698 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1732, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2007, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 14296328, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1104800, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2023-10-06T21:36:04.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.164855012, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.055850529, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.044620031, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.050470565, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2005, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.333472783, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000166320 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.700 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 566704 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 807, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 767, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 4728143, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 338843, 2023-10-06T21:36:04.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.094424570, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.029452113, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.035852533, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.022838323, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 767, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.224284723, 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000292418 2023-10-06T21:36:04.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 442288, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 227504 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-10-06T21:36:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7430, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.219054839, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000029482 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2023-10-06T21:36:04.704 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.705 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 423, 2023-10-06T21:36:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 177, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 229, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 17, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 170, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 14, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 14, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2023-10-06T21:36:04.707 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2023-10-06T21:36:04.708 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 1008, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 1192328189, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 6, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2023-10-06T21:36:04.709 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2023-10-06T21:36:04.710 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2023-10-06T21:36:04.711 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2023-10-06T21:36:04.712 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 1876, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2023-10-06T21:36:04.713 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 57, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 34553225, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 36433886, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 36432565, 2023-10-06T21:36:04.714 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 1876 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 210, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 1094, 2023-10-06T21:36:04.715 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 884, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 46, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 59, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 13, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 423, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 1630, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 1207, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 52952, 2023-10-06T21:36:04.716 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 207152 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.717 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.718 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.719 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2023-10-06T21:36:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.722 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.723 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.724 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.725 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2023-10-06T21:36:04.726 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.727 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.728 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.729 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.730 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 2, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 0, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 3381, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-10-06T21:36:04.731 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:36:04.732 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:36:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:36:04.734 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 223669, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 464, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 8128, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 169, 2023-10-06T21:36:04.735 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 1060302, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 34182, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2023-10-06T21:36:04.736 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 4455, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 4455, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4455, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 27.976505723, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006279799 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2023-10-06T21:36:04.737 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4455, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 4456 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 4455, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 4455, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 2690, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1762, 2023-10-06T21:36:04.738 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2023-10-06T21:36:04.739 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2023-10-06T21:36:04.741 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2023-10-06T21:36:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 127, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 38, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 7, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 8, 2023-10-06T21:36:04.743 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 1, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.744 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2023-10-06T21:36:04.745 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4902, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 23459874, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4902, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4902, 2023-10-06T21:36:04.746 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 23459874, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:04.747 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4455, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 22239598, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4455, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 22239598, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.748 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4455, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 4455, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4455, 2023-10-06T21:36:04.749 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4455, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:04.751 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x561d6569a960": { 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2023-10-06T21:36:04.752 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:04.753 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T21:36:04.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:04 smithi167 ceph-mon[134494]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:04.885 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:04 smithi078 ceph-mon[134896]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:05.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:04 smithi026 ceph-mon[98490]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-10-06T21:36:05.089 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-10-06T21:36:05.089 DEBUG:tasks.check_counter:Getting stats from l 2023-10-06T21:36:05.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:14d8d032507701c76ca224e769ee65a96b660a8c shell --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-06T21:36:05.647 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-06T21:36:06.783 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001224769, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000230966, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:36:06.784 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.785 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 6, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 347, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 3344, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 214400, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 5, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.018010950, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012530833, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.001754247, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:06.786 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 345, 2023-10-06T21:36:06.787 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.071810156, 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000208145 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:06.789 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3344, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 214400 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 463, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 452, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1629760, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 273088, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2023-10-06T21:36:06.790 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.073356690, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.024674438, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.040762423, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.012741361, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 451, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.070028867, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000155274 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2023-10-06T21:36:06.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1629760, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 273088 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2023-10-06T21:36:06.792 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-06T21:36:06.793 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2023-10-06T21:36:06.794 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148977, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 30, 2023-10-06T21:36:06.795 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2023-10-06T21:36:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2023-10-06T21:36:06.797 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 465, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1564295, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 465, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 465, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1564295, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: } 2023-10-06T21:36:06.799 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T21:36:07.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:06 smithi078 ceph-mon[134896]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:36:07.141 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:06 smithi167 ceph-mon[134494]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:36:07.190 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-10-06T21:36:07.190 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-10-06T21:36:07.191 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-10-06T21:36:07.191 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-10-06T21:36:07.191 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-10-06T21:36:07.191 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-10-06T21:36:07.191 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-10-06T21:36:07.191 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T21:36:07.207 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T21:36:07.208 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:36:07.208 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:36:07.261 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:36:07.261 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:36:07.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:36:07.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:06 smithi026 ceph-mon[98490]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 20 op/s 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 -- 172.21.15.26:0/1867121270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6630101ff0 msgr2=0x7f66301023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 --2- 172.21.15.26:0/1867121270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6630101ff0 0x7f66301023d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6624009990 tx=0x7f6624009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 -- 172.21.15.26:0/1867121270 shutdown_connections 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 --2- 172.21.15.26:0/1867121270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f663010b410 0x7f663010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 --2- 172.21.15.26:0/1867121270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6630101ff0 0x7f66301023d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 --2- 172.21.15.26:0/1867121270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f663010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 -- 172.21.15.26:0/1867121270 >> 172.21.15.26:0/1867121270 conn(0x7f66300fb7f0 msgr2=0x7f66300fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 -- 172.21.15.26:0/1867121270 shutdown_connections 2023-10-06T21:36:07.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.593+0000 7f6636bff700 1 -- 172.21.15.26:0/1867121270 wait complete. 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.594+0000 7f6636bff700 1 Processor -- start 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.594+0000 7f6636bff700 1 -- start start 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f66301996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 0x7f663019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e150 con 0x7f6630102910 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e2c0 con 0x7f663010b410 2023-10-06T21:36:07.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f6636bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663019e480 con 0x7f6630101ff0 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:45548/0 (socket says 172.21.15.26:45548) 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 -- 172.21.15.26:0/562463974 learned_addr learned my addr 172.21.15.26:0/562463974 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663519c700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 0x7f663019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f662ffff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f66301996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 -- 172.21.15.26:0/562463974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 msgr2=0x7f663019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 0x7f663019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 -- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 msgr2=0x7f66301996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f66301996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f663499b700 1 -- 172.21.15.26:0/562463974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240095f0 con 0x7f6630101ff0 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.595+0000 7f662ffff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f66301996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:07.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f663519c700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 0x7f663019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f663499b700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6624005910 tx=0x7f6624005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662401d070 con 0x7f6630101ff0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6624022470 con 0x7f6630101ff0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663019e700 con 0x7f6630101ff0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624017650 con 0x7f6630101ff0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.596+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301a5fa0 con 0x7f6630101ff0 2023-10-06T21:36:07.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.597+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6630108940 con 0x7f6630101ff0 2023-10-06T21:36:07.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.600+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f66240225e0 con 0x7f6630101ff0 2023-10-06T21:36:07.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.601+0000 7f662dffb700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 0x7f66200a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:07.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.601+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f66240cea90 con 0x7f6630101ff0 2023-10-06T21:36:07.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.602+0000 7f662ffff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 0x7f66200a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:07.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.602+0000 7f662ffff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 0x7f66200a3b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f663019a780 tx=0x7f661c009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:07.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.603+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6624097ef0 con 0x7f6630101ff0 2023-10-06T21:36:07.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.848+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6630067cd0 con 0x7f6630101ff0 2023-10-06T21:36:07.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.850+0000 7f662dffb700 1 -- 172.21.15.26:0/562463974 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+23020 (secure 0 0 0) 0x7f662409b510 con 0x7f6630101ff0 2023-10-06T21:36:07.853 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:36:07.853 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":127,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:35:18.573659+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1157730691":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2246884211":"2023-10-07T21:10:19.013203+0000","172.21.15.26:6800/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3325517294":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3867812316":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:36:07.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.852+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 msgr2=0x7f66200a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.852+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 0x7f66200a3b70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f663019a780 tx=0x7f661c009260 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.852+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 msgr2=0x7f6630199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.852+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6624005910 tx=0x7f6624005720 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 shutdown_connections 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f663010b410 0x7f663019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6630101ff0 0x7f6630199170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f66200a16b0 0x7f66200a3b70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 --2- 172.21.15.26:0/562463974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6630102910 0x7f66301996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 >> 172.21.15.26:0/562463974 conn(0x7f66300fb7f0 msgr2=0x7f663010d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 shutdown_connections 2023-10-06T21:36:07.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:07.853+0000 7f6636bff700 1 -- 172.21.15.26:0/562463974 wait complete. 2023-10-06T21:36:07.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T21:04:02.882358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T21:08:14.443239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T21:08:15.493688+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-06T21:36:07.876 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T21:36:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2486392041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 msgr2=0x7f27c810ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2486392041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c810ff60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009f90 tx=0x7f27c000b330 comp rx=0 tx=0).stop 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2486392041 shutdown_connections 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2486392041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c810ff60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2486392041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c81036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2486392041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c8103c70 0x7f27c8107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2486392041 >> 172.21.15.26:0/2486392041 conn(0x7f27c80feb60 msgr2=0x7f27c8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2486392041 shutdown_connections 2023-10-06T21:36:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.113+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2486392041 wait complete. 2023-10-06T21:36:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 Processor -- start 2023-10-06T21:36:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 -- start start 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819e820 con 0x7f27c81083e0 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819e990 con 0x7f27c819dda0 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.115+0000 7f27cf1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c819eb50 con 0x7f27c81032c0 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:54994/0 (socket says 172.21.15.26:54994) 2023-10-06T21:36:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 -- 172.21.15.26:0/2476663074 learned_addr learned my addr 172.21.15.26:0/2476663074 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37240/0 (socket says 172.21.15.26:37240) 2023-10-06T21:36:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27ccf9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 -- 172.21.15.26:0/2476663074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 msgr2=0x7f27c8199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c8199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 -- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 msgr2=0x7f27c8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 -- 172.21.15.26:0/2476663074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009c40 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27c7fff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27ccf9b700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c8199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.116+0000 7f27cd79c700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009f60 tx=0x7f27c0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.117+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000e040 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.117+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c0008e80 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.117+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00052a0 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.117+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81a6050 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.117+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81a6480 con 0x7f27c819dda0 2023-10-06T21:36:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.118+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c819a880 con 0x7f27c819dda0 2023-10-06T21:36:08.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.121+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f27c001d470 con 0x7f27c819dda0 2023-10-06T21:36:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.122+0000 7f27c5ffb700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.122+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f27c00ce6b0 con 0x7f27c819dda0 2023-10-06T21:36:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.122+0000 7f27ccf9b700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.123+0000 7f27ccf9b700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 0x7f27b80a3ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27bc011490 tx=0x7f27bc009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.123+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27c0097a50 con 0x7f27c819dda0 2023-10-06T21:36:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.383+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27c8067470 con 0x7f27c819dda0 2023-10-06T21:36:08.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.383+0000 7f27c5ffb700 1 -- 172.21.15.26:0/2476663074 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27c009b070 con 0x7f27c819dda0 2023-10-06T21:36:08.386 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-10-06T21:36:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 msgr2=0x7f27b80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 0x7f27b80a3ab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27bc011490 tx=0x7f27bc009260 comp rx=0 tx=0).stop 2023-10-06T21:36:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 msgr2=0x7f27c819e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009f60 tx=0x7f27c0004c80 comp rx=0 tx=0).stop 2023-10-06T21:36:08.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 shutdown_connections 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27c819dda0 0x7f27c819e220 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27c81032c0 0x7f27c8199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f27b80a15f0 0x7f27b80a3ab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 --2- 172.21.15.26:0/2476663074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27c81083e0 0x7f27c8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 >> 172.21.15.26:0/2476663074 conn(0x7f27c80feb60 msgr2=0x7f27c80ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 shutdown_connections 2023-10-06T21:36:08.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.386+0000 7f27cf1ff700 1 -- 172.21.15.26:0/2476663074 wait complete. 2023-10-06T21:36:08.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.647+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/636397603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28101ff0 msgr2=0x7f4e281023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.647+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/636397603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28101ff0 0x7f4e281023d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/636397603 shutdown_connections 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/636397603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/636397603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28101ff0 0x7f4e281023d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/636397603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28102910 0x7f4e2810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/636397603 >> 172.21.15.26:0/636397603 conn(0x7f4e280fb830 msgr2=0x7f4e280fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/636397603 shutdown_connections 2023-10-06T21:36:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.648+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/636397603 wait complete. 2023-10-06T21:36:08.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.650+0000 7f4e2e1ff700 1 Processor -- start 2023-10-06T21:36:08.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.650+0000 7f4e2e1ff700 1 -- start start 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 0x7f4e28196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 0x7f4e28197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819bfc0 con 0x7f4e28101ff0 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819c130 con 0x7f4e28102910 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e2e1ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819c2f0 con 0x7f4e2810b410 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 0x7f4e28196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 0x7f4e28196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37264/0 (socket says 172.21.15.26:37264) 2023-10-06T21:36:08.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e26ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 0x7f4e28197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e277fe700 1 -- 172.21.15.26:0/698640678 learned_addr learned my addr 172.21.15.26:0/698640678 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.651+0000 7f4e27fff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 -- 172.21.15.26:0/698640678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 msgr2=0x7f4e2819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 -- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 msgr2=0x7f4e28196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 0x7f4e28196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 -- 172.21.15.26:0/698640678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180095f0 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e27fff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e26ffd700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 0x7f4e28197520 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d6b0 tx=0x7f4e1c00d9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00f890 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c010460 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2819c5d0 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00e610 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.652+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e281a3e10 con 0x7f4e28102910 2023-10-06T21:36:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.653+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28067c60 con 0x7f4e28102910 2023-10-06T21:36:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.657+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f4e1c00f9f0 con 0x7f4e28102910 2023-10-06T21:36:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.658+0000 7f4e24ff9700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 0x7f4e100a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.658+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4e1c0ccdc0 con 0x7f4e28102910 2023-10-06T21:36:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.658+0000 7f4e277fe700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 0x7f4e100a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.659+0000 7f4e277fe700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 0x7f4e100a3ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006010 tx=0x7f4e1800b540 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.660+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e1c0960d0 con 0x7f4e28102910 2023-10-06T21:36:08.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.911+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e280008d0 con 0x7f4e28102910 2023-10-06T21:36:08.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.911+0000 7f4e24ff9700 1 -- 172.21.15.26:0/698640678 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e1c0996f0 con 0x7f4e28102910 2023-10-06T21:36:08.914 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-10-06T21:36:08.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 msgr2=0x7f4e100a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 0x7f4e100a3ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006010 tx=0x7f4e1800b540 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 msgr2=0x7f4e28197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 0x7f4e28197520 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d6b0 tx=0x7f4e1c00d9c0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 shutdown_connections 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2810b410 0x7f4e2819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f4e100a15f0 0x7f4e100a3ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e28101ff0 0x7f4e28196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 --2- 172.21.15.26:0/698640678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e28102910 0x7f4e28197520 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.915+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 >> 172.21.15.26:0/698640678 conn(0x7f4e280fb830 msgr2=0x7f4e2810d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.916+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 shutdown_connections 2023-10-06T21:36:08.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:08.916+0000 7f4e2e1ff700 1 -- 172.21.15.26:0/698640678 wait complete. 2023-10-06T21:36:08.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T21:36:09.120 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:08 smithi078 ceph-mon[134896]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:08 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/562463974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:09.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:08 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2476663074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 -- 172.21.15.26:0/427143850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 msgr2=0x7fb164073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 --2- 172.21.15.26:0/427143850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb164073640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb154009990 tx=0x7fb154009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 -- 172.21.15.26:0/427143850 shutdown_connections 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 --2- 172.21.15.26:0/427143850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb16410d710 0x7fb16410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 --2- 172.21.15.26:0/427143850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb164073640 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 --2- 172.21.15.26:0/427143850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb164073b80 0x7fb16410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 -- 172.21.15.26:0/427143850 >> 172.21.15.26:0/427143850 conn(0x7fb1640fc960 msgr2=0x7fb1640fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 -- 172.21.15.26:0/427143850 shutdown_connections 2023-10-06T21:36:09.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.181+0000 7fb168bff700 1 -- 172.21.15.26:0/427143850 wait complete. 2023-10-06T21:36:09.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.182+0000 7fb168bff700 1 Processor -- start 2023-10-06T21:36:09.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.182+0000 7fb168bff700 1 -- start start 2023-10-06T21:36:09.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb164073b80 0x7fb164199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 0x7fb16419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16419dfb0 con 0x7fb16410d710 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16419e120 con 0x7fb164073260 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb168bff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16419e2e0 con 0x7fb164073b80 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55018/0 (socket says 172.21.15.26:55018) 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 -- 172.21.15.26:0/316094263 learned_addr learned my addr 172.21.15.26:0/316094263 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb162d9d700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 0x7fb16419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb161d9b700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb164073b80 0x7fb164199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 -- 172.21.15.26:0/316094263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb164073b80 msgr2=0x7fb164199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb164073b80 0x7fb164199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 -- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 msgr2=0x7fb16419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 0x7fb16419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.183+0000 7fb16259c700 1 -- 172.21.15.26:0/316094263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb158009520 con 0x7fb164073260 2023-10-06T21:36:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb162d9d700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 0x7fb16419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb16259c700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb15400f690 tx=0x7fb15400f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15401d070 con 0x7fb164073260 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1540218d0 con 0x7fb164073260 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1540095f0 con 0x7fb164073260 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540176e0 con 0x7fb164073260 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.184+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1641a5c20 con 0x7fb164073260 2023-10-06T21:36:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.185+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1640673d0 con 0x7fb164073260 2023-10-06T21:36:09.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.188+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fb154021a40 con 0x7fb164073260 2023-10-06T21:36:09.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.190+0000 7fb1537fe700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 0x7fb14c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.190+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb1540ce550 con 0x7fb164073260 2023-10-06T21:36:09.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.190+0000 7fb161d9b700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 0x7fb14c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.191+0000 7fb161d9b700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 0x7fb14c0a3b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb16419a5e0 tx=0x7fb158009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:09.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.194+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb154097890 con 0x7fb164073260 2023-10-06T21:36:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:08 smithi167 ceph-mon[134494]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:08 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/562463974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:09.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:08 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2476663074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:36:09.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:08 smithi026 ceph-mon[98490]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-10-06T21:36:09.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/562463974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:09.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:08 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2476663074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-06T21:36:09.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.516+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb164113180 con 0x7fb164073260 2023-10-06T21:36:09.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.517+0000 7fb1537fe700 1 -- 172.21.15.26:0/316094263 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb15409aeb0 con 0x7fb164073260 2023-10-06T21:36:09.520 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-10-06T21:36:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 msgr2=0x7fb14c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 0x7fb14c0a3b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb16419a5e0 tx=0x7fb158009260 comp rx=0 tx=0).stop 2023-10-06T21:36:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 msgr2=0x7fb1641991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb15400f690 tx=0x7fb15400f6c0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 shutdown_connections 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fb14c0a1670 0x7fb14c0a3b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb16410d710 0x7fb16419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb164073260 0x7fb1641991e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 --2- 172.21.15.26:0/316094263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb164073b80 0x7fb164199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 >> 172.21.15.26:0/316094263 conn(0x7fb1640fc960 msgr2=0x7fb16410f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 shutdown_connections 2023-10-06T21:36:09.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.520+0000 7fb168bff700 1 -- 172.21.15.26:0/316094263 wait complete. 2023-10-06T21:36:09.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-06T21:36:09.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 -- 172.21.15.26:0/4249354693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3524103cb0 msgr2=0x7f3524107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 --2- 172.21.15.26:0/4249354693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3524103cb0 0x7f3524107cf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3518009990 tx=0x7f3518009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 -- 172.21.15.26:0/4249354693 shutdown_connections 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 --2- 172.21.15.26:0/4249354693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 0x7f352410ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 --2- 172.21.15.26:0/4249354693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 0x7f35241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 --2- 172.21.15.26:0/4249354693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3524103cb0 0x7f3524107cf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 -- 172.21.15.26:0/4249354693 >> 172.21.15.26:0/4249354693 conn(0x7f35240feb40 msgr2=0x7f3524100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 -- 172.21.15.26:0/4249354693 shutdown_connections 2023-10-06T21:36:09.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.757+0000 7f3529fff700 1 -- 172.21.15.26:0/4249354693 wait complete. 2023-10-06T21:36:09.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.758+0000 7f3529fff700 1 Processor -- start 2023-10-06T21:36:09.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 -- start start 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 0x7f3524199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 0x7f3524199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352419e610 con 0x7f3524103300 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352419e780 con 0x7f352419dc20 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3529fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352419e940 con 0x7f3524108420 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3523fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f35237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 0x7f3524199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3523fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:55044/0 (socket says 172.21.15.26:55044) 2023-10-06T21:36:09.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.759+0000 7f3523fff700 1 -- 172.21.15.26:0/1413958078 learned_addr learned my addr 172.21.15.26:0/1413958078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3522ffd700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 0x7f3524199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 -- 172.21.15.26:0/1413958078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 msgr2=0x7f3524199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 0x7f3524199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 -- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 msgr2=0x7f3524199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 0x7f3524199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 -- 172.21.15.26:0/1413958078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180095f0 con 0x7f352419dc20 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3523fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f351c00ef30 tx=0x7f351c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c018070 con 0x7f352419dc20 2023-10-06T21:36:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351c00f040 con 0x7f352419dc20 2023-10-06T21:36:09.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35241a5ed0 con 0x7f352419dc20 2023-10-06T21:36:09.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.760+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c00cdb0 con 0x7f352419dc20 2023-10-06T21:36:09.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.761+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35241a62f0 con 0x7f352419dc20 2023-10-06T21:36:09.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.762+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3524067c60 con 0x7f352419dc20 2023-10-06T21:36:09.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.764+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f351c003a00 con 0x7f352419dc20 2023-10-06T21:36:09.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.766+0000 7f3520ff9700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 0x7f350c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:09.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.766+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f351c0ccd00 con 0x7f352419dc20 2023-10-06T21:36:09.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.766+0000 7f35237fe700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 0x7f350c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:09.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.767+0000 7f35237fe700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 0x7f350c0a3b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c60 tx=0x7f3514009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:09.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:09.767+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f351c096040 con 0x7f352419dc20 2023-10-06T21:36:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.057+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3524103e90 con 0x7f352419dc20 2023-10-06T21:36:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.057+0000 7f3520ff9700 1 -- 172.21.15.26:0/1413958078 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v127) v1 ==== 81+0+942 (secure 0 0 0) 0x7f351c0b9080 con 0x7f352419dc20 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1157730691 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2246884211 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2830651730 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1380403953 2023-10-07T21:01:44.284577+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/10926064 2023-10-07T21:01:44.284577+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/118366491 2023-10-07T21:01:44.284577+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2830651730 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/4033583135 2023-10-07T21:01:17.251649+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3325517294 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.061 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2078185111 2023-10-07T21:01:17.251649+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3098538275 2023-10-07T21:01:17.251649+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1222842153 2023-10-07T21:01:17.251649+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3867812316 2023-10-07T21:10:19.013203+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2655877248 2023-10-07T21:01:44.284577+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3098538275 2023-10-07T21:01:17.251649+0000 2023-10-06T21:36:10.062 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/118366491 2023-10-07T21:01:44.284577+0000 2023-10-06T21:36:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 msgr2=0x7f350c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 0x7f350c0a3b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3514009c60 tx=0x7f3514009260 comp rx=0 tx=0).stop 2023-10-06T21:36:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 msgr2=0x7f352419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f351c00ef30 tx=0x7f351c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 shutdown_connections 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f352419dc20 0x7f352419e0d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f350c0a1670 0x7f350c0a3b30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3524103300 0x7f3524199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 --2- 172.21.15.26:0/1413958078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3524108420 0x7f3524199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 >> 172.21.15.26:0/1413958078 conn(0x7f35240feb40 msgr2=0x7f35240ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 shutdown_connections 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:10.060+0000 7f3529fff700 1 -- 172.21.15.26:0/1413958078 wait complete. 2023-10-06T21:36:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:listed 16 entries 2023-10-06T21:36:10.084 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T21:36:10.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:10.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /tmp/cephfs 2023-10-06T21:36:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/698640678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:09 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/316094263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/698640678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:09 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/316094263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/698640678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:09 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/316094263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-06T21:36:10.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:36:10.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:10.289 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-10-06T21:36:10.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:10.506 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-10-06T21:36:10.730 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-10-06T21:36:10.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:10.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-10-06T21:36:10.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:10.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-10-06T21:36:11.069 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T21:36:11.069 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-10-06T21:36:11.069 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-06T21:36:11.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348692k,nr_inodes=4087173,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/f1516414062d3bc771cf064b0ba0f074bc19debb2c7210dbe1a90d1e040d52bb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/f1516414062d3bc771cf064b0ba0f074bc19debb2c7210dbe1a90d1e040d52bb/diff,workdir=/var/lib/containers/storage/overlay/f1516414062d3bc771cf064b0ba0f074bc19debb2c7210dbe1a90d1e040d52bb/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b78b7e6b030f823c73b6ee9f966d3100c3ce5e4aaabab861f37aed0b711b62ee/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/b78b7e6b030f823c73b6ee9f966d3100c3ce5e4aaabab861f37aed0b711b62ee/diff,workdir=/var/lib/containers/storage/overlay/b78b7e6b030f823c73b6ee9f966d3100c3ce5e4aaabab861f37aed0b711b62ee/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/69ac03d0855e681d178f46f33f272ae25113d1bd4295c5d40ab24361ff018076/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/69ac03d0855e681d178f46f33f272ae25113d1bd4295c5d40ab24361ff018076/diff,workdir=/var/lib/containers/storage/overlay/69ac03d0855e681d178f46f33f272ae25113d1bd4295c5d40ab24361ff018076/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d04d080e8923a053426291fab06301281323adc3b99e90216fe5fe90d4e83773/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/d04d080e8923a053426291fab06301281323adc3b99e90216fe5fe90d4e83773/diff,workdir=/var/lib/containers/storage/overlay/d04d080e8923a053426291fab06301281323adc3b99e90216fe5fe90d4e83773/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7eb91bc6f89f94cf1570a5992e81625180153861f1eebbc37c7b7ab54cc894ab/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/7eb91bc6f89f94cf1570a5992e81625180153861f1eebbc37c7b7ab54cc894ab/diff,workdir=/var/lib/containers/storage/overlay/7eb91bc6f89f94cf1570a5992e81625180153861f1eebbc37c7b7ab54cc894ab/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1b271981af66a0324b347f6d97bdcab2a3f674bc24eb9715494d45e716e66bcc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/1b271981af66a0324b347f6d97bdcab2a3f674bc24eb9715494d45e716e66bcc/diff,workdir=/var/lib/containers/storage/overlay/1b271981af66a0324b347f6d97bdcab2a3f674bc24eb9715494d45e716e66bcc/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/33c34cd995a5c61f7ac631fb6096131915f35553a3181fa4b6c72b4e813c7140/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/33c34cd995a5c61f7ac631fb6096131915f35553a3181fa4b6c72b4e813c7140/diff,workdir=/var/lib/containers/storage/overlay/33c34cd995a5c61f7ac631fb6096131915f35553a3181fa4b6c72b4e813c7140/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/347a225ca1d71f0ac9afee517e19ba637af1e65bf0ef5e01346d4d4bfd19841e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/347a225ca1d71f0ac9afee517e19ba637af1e65bf0ef5e01346d4d4bfd19841e/diff,workdir=/var/lib/containers/storage/overlay/347a225ca1d71f0ac9afee517e19ba637af1e65bf0ef5e01346d4d4bfd19841e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/08b1d5929e06d745e543fdd6a103edb159e15e97baa3a9d90f54f46a5a7c88e1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/08b1d5929e06d745e543fdd6a103edb159e15e97baa3a9d90f54f46a5a7c88e1/diff,workdir=/var/lib/containers/storage/overlay/08b1d5929e06d745e543fdd6a103edb159e15e97baa3a9d90f54f46a5a7c88e1/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6698cb36f2b77ef8d5460015244c484f1df173766247a9c2bc89170f0cea659c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/Y6HKBUVURRPNJWHWIKDR4LCIFO:/var/lib/containers/storage/overlay/l/24JHY4DLV7H43L44ZZBQRKYQ52,upperdir=/var/lib/containers/storage/overlay/6698cb36f2b77ef8d5460015244c484f1df173766247a9c2bc89170f0cea659c/diff,workdir=/var/lib/containers/storage/overlay/6698cb36f2b77ef8d5460015244c484f1df173766247a9c2bc89170f0cea659c/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-06T21:36:11.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:11.127 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-10-06T21:36:11.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T21:36:11.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-10-06T21:36:11.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:10 smithi026 ceph-mon[98490]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:11.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:10 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1413958078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:36:11.263 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-10-06T21:36:11.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:36:11.267 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-06T21:36:11.278 INFO:tasks.cephadm:Teardown begin 2023-10-06T21:36:11.278 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T21:36:11.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:36:11.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:10 smithi167 ceph-mon[134494]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:11.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:10 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1413958078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:36:11.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:10 smithi078 ceph-mon[134896]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-10-06T21:36:11.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:10 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1413958078' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-06T21:36:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 -- 172.21.15.26:0/11636540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 msgr2=0x7f1834104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 --2- 172.21.15.26:0/11636540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f1834104a60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1828009990 tx=0x7f1828009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 -- 172.21.15.26:0/11636540 shutdown_connections 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 --2- 172.21.15.26:0/11636540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f1834105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 --2- 172.21.15.26:0/11636540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f1834100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 --2- 172.21.15.26:0/11636540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f1834104a60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 -- 172.21.15.26:0/11636540 >> 172.21.15.26:0/11636540 conn(0x7f18340fb6d0 msgr2=0x7f18340fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 -- 172.21.15.26:0/11636540 shutdown_connections 2023-10-06T21:36:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.512+0000 7f1833fff700 1 -- 172.21.15.26:0/11636540 wait complete. 2023-10-06T21:36:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.513+0000 7f1833fff700 1 Processor -- start 2023-10-06T21:36:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 -- start start 2023-10-06T21:36:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f183419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f183419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18341a22c0 con 0x7f18340ffc80 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18341a2430 con 0x7f18341005a0 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1833fff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18341a25f0 con 0x7f1834105080 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f18337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f1832ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f183419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f18337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.26:52828/0 (socket says 172.21.15.26:52828) 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f18337fe700 1 -- 172.21.15.26:0/2243529326 learned_addr learned my addr 172.21.15.26:0/2243529326 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.514+0000 7f18327fc700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f183419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 -- 172.21.15.26:0/2243529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 msgr2=0x7f183419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f183419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 -- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 msgr2=0x7f183419d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f183419d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 -- 172.21.15.26:0/2243529326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280095f0 con 0x7f1834105080 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f1832ffd700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f183419d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f18337fe700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f182c00ef30 tx=0x7f182c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c018070 con 0x7f1834105080 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18341a2870 con 0x7f1834105080 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182c00f040 con 0x7f1834105080 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cd00 con 0x7f1834105080 2023-10-06T21:36:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.515+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18341aa110 con 0x7f1834105080 2023-10-06T21:36:11.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.516+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1834067360 con 0x7f1834105080 2023-10-06T21:36:11.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.519+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f182c003a00 con 0x7f1834105080 2023-10-06T21:36:11.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.520+0000 7f181bfff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 0x7f181c0a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.520+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f182c0ccdd0 con 0x7f1834105080 2023-10-06T21:36:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.520+0000 7f1832ffd700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 0x7f181c0a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.521+0000 7f1832ffd700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 0x7f181c0a3ba0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1824011490 tx=0x7f1824009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.522+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f182c0960e0 con 0x7f1834105080 2023-10-06T21:36:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.874+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f183419e630 con 0x7f1834105080 2023-10-06T21:36:11.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.875+0000 7f181bfff700 1 -- 172.21.15.26:0/2243529326 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f182c099700 con 0x7f1834105080 2023-10-06T21:36:11.878 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:36:11.878 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":67,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:28:54.512601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 msgr2=0x7f181c0a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 0x7f181c0a3ba0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1824011490 tx=0x7f1824009260 comp rx=0 tx=0).stop 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 msgr2=0x7f18341a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f182c00ef30 tx=0x7f182c00c3a0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 shutdown_connections 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1834105080 0x7f18341a1c80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f181c0a16e0 0x7f181c0a3ba0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18340ffc80 0x7f183419d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 --2- 172.21.15.26:0/2243529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18341005a0 0x7f183419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 >> 172.21.15.26:0/2243529326 conn(0x7f18340fb6d0 msgr2=0x7f18340fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:11.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 shutdown_connections 2023-10-06T21:36:11.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:11.878+0000 7f1833fff700 1 -- 172.21.15.26:0/2243529326 wait complete. 2023-10-06T21:36:11.883 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 67 2023-10-06T21:36:11.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T21:36:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 -- 172.21.15.26:0/3306632832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc10b480 msgr2=0x7f62dc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 --2- 172.21.15.26:0/3306632832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc10b480 0x7f62dc10d8d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b0a0 tx=0x7f62d400b3b0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 -- 172.21.15.26:0/3306632832 shutdown_connections 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 --2- 172.21.15.26:0/3306632832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc10b480 0x7f62dc10d8d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 --2- 172.21.15.26:0/3306632832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1020b0 0x7f62dc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 --2- 172.21.15.26:0/3306632832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc1029d0 0x7f62dc10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 -- 172.21.15.26:0/3306632832 >> 172.21.15.26:0/3306632832 conn(0x7f62dc0fb7f0 msgr2=0x7f62dc0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 -- 172.21.15.26:0/3306632832 shutdown_connections 2023-10-06T21:36:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.135+0000 7f62e25ff700 1 -- 172.21.15.26:0/3306632832 wait complete. 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.136+0000 7f62e25ff700 1 Processor -- start 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 -- start start 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1029d0 0x7f62dc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 0x7f62dc19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19e190 con 0x7f62dc10b480 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19e300 con 0x7f62dc1020b0 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e25ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19e4c0 con 0x7f62dc1029d0 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62dbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51918/0 (socket says 172.21.15.26:51918) 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62dbfff700 1 -- 172.21.15.26:0/1772878043 learned_addr learned my addr 172.21.15.26:0/1772878043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.137+0000 7f62e0b9c700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 0x7f62dc19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62db7fe700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1029d0 0x7f62dc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 -- 172.21.15.26:0/1772878043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1029d0 msgr2=0x7f62dc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1029d0 0x7f62dc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 -- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 msgr2=0x7f62dc19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 0x7f62dc19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 -- 172.21.15.26:0/1772878043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4009d00 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62e0b9c700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 0x7f62dc19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62dbfff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9a0 tx=0x7f62cc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009790 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc00cbd0 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc19e7a0 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.138+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc018680 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.139+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc1a5fe0 con 0x7f62dc1020b0 2023-10-06T21:36:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.140+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc067cd0 con 0x7f62dc1020b0 2023-10-06T21:36:12.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.142+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f62cc00cd40 con 0x7f62dc1020b0 2023-10-06T21:36:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.143+0000 7f62d97fa700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 0x7f62c40a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.144+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f62cc014070 con 0x7f62dc1020b0 2023-10-06T21:36:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.144+0000 7f62db7fe700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 0x7f62c40a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.144+0000 7f62db7fe700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 0x7f62c40a3b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62dc19a7c0 tx=0x7f62d0009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.145+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f62cc096660 con 0x7f62dc1020b0 2023-10-06T21:36:12.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:11 smithi167 ceph-mon[134494]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:36:12.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:11 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/2243529326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:12.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:11 smithi026 ceph-mon[98490]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:36:12.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:11 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/2243529326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:12.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:11 smithi078 ceph-mon[134896]: osdmap e128: 12 total, 12 up, 12 in 2023-10-06T21:36:12.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:11 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/2243529326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:12.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.493+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62dc19a570 con 0x7f62dc1020b0 2023-10-06T21:36:12.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.494+0000 7f62d97fa700 1 -- 172.21.15.26:0/1772878043 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+11308 (secure 0 0 0) 0x7f62cc099c80 con 0x7f62dc1020b0 2023-10-06T21:36:12.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:36:12.497 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4105384407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":4105384407},{"type":"v1","addr":"172.21.15.26:6835","nonce":4105384407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2894864282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2894864282},{"type":"v1","addr":"172.21.15.26:6837","nonce":2894864282}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14511,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/4229413867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":4229413867},{"type":"v1","addr":"172.21.15.26:6839","nonce":4229413867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/4048987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":4048987},{"type":"v1","addr":"172.21.15.167:6833","nonce":4048987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3813856521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3813856521},{"type":"v1","addr":"172.21.15.78:6833","nonce":3813856521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3142312308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3142312308},{"type":"v1","addr":"172.21.15.167:6835","nonce":3142312308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":67,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-06T21:08:16.065653+0000","modified":"2023-10-06T21:28:54.512601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24587,"mds_1":24559,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6835/4281425672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4281425672},{"type":"v1","addr":"172.21.15.78:6835","nonce":4281425672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24514":{"gid":24514,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2946984451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2946984451},{"type":"v1","addr":"172.21.15.167:6837","nonce":2946984451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.78:6837/1931966907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1931966907},{"type":"v1","addr":"172.21.15.78:6837","nonce":1931966907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2891556508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2891556508},{"type":"v1","addr":"172.21.15.26:6841","nonce":2891556508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24587":{"gid":24587,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6839/3790769886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3790769886},{"type":"v1","addr":"172.21.15.78:6839","nonce":3790769886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25876":{"gid":25876,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/3154348902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3154348902},{"type":"v1","addr":"172.21.15.167:6839","nonce":3154348902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-06T21:36:12.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.496+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 msgr2=0x7f62c40a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.496+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 0x7f62c40a3b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62dc19a7c0 tx=0x7f62d0009260 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.496+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 msgr2=0x7f62dc199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.496+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9a0 tx=0x7f62cc00ed60 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 shutdown_connections 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f62c40a16b0 0x7f62c40a3b70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62dc10b480 0x7f62dc19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62dc1020b0 0x7f62dc199220 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 --2- 172.21.15.26:0/1772878043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62dc1029d0 0x7f62dc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 >> 172.21.15.26:0/1772878043 conn(0x7f62dc0fb7f0 msgr2=0x7f62dc0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:12.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 shutdown_connections 2023-10-06T21:36:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.497+0000 7f62e25ff700 1 -- 172.21.15.26:0/1772878043 wait complete. 2023-10-06T21:36:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 67 2023-10-06T21:36:12.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/1260295234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc0103130 msgr2=0x7fdbc0103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/1260295234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc0103130 0x7fdbc0103510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009990 tx=0x7fdbb4009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/1260295234 shutdown_connections 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/1260295234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0108250 0x7fdbc010fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/1260295234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc0103130 0x7fdbc0103510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/1260295234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0103ae0 0x7fdbc0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/1260295234 >> 172.21.15.26:0/1260295234 conn(0x7fdbc00fe950 msgr2=0x7fdbc0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:12.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/1260295234 shutdown_connections 2023-10-06T21:36:12.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.748+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/1260295234 wait complete. 2023-10-06T21:36:12.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.749+0000 7fdbc5f9d700 1 Processor -- start 2023-10-06T21:36:12.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 -- start start 2023-10-06T21:36:12.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0103130 0x7fdbc0198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0108250 0x7fdbc01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc019e2a0 con 0x7fdbc0103130 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc019e410 con 0x7fdbc019d820 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc5f9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc019e5d0 con 0x7fdbc0108250 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc579c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc579c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51924/0 (socket says 172.21.15.26:51924) 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.750+0000 7fdbc579c700 1 -- 172.21.15.26:0/3614562963 learned_addr learned my addr 172.21.15.26:0/3614562963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc4f9b700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0103130 0x7fdbc0198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbbffff700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0108250 0x7fdbc01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 -- 172.21.15.26:0/3614562963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0108250 msgr2=0x7fdbc01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0108250 0x7fdbc01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 -- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0103130 msgr2=0x7fdbc0198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:12.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0103130 0x7fdbc0198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:12.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 -- 172.21.15.26:0/3614562963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40095f0 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc579c700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8007ee0 tx=0x7fdbb800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8017070 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb800f040 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc01a5b30 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.751+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8005740 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.752+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc01a6010 con 0x7fdbc019d820 2023-10-06T21:36:12.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.753+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc0067360 con 0x7fdbc019d820 2023-10-06T21:36:12.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.755+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7fdbb800dc60 con 0x7fdbc019d820 2023-10-06T21:36:12.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.757+0000 7fdbbdffb700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 0x7fdbb00a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:12.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.757+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fdbb80cc8b0 con 0x7fdbc019d820 2023-10-06T21:36:12.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.757+0000 7fdbc4f9b700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 0x7fdbb00a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:12.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.758+0000 7fdbc4f9b700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 0x7fdbb00a3c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005b00 tx=0x7fdbb4005a90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:12.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:12.761+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdbb8095b30 con 0x7fdbc019d820 2023-10-06T21:36:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.070+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbc0103cc0 con 0x7fdbc019d820 2023-10-06T21:36:13.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.072+0000 7fdbbdffb700 1 -- 172.21.15.26:0/3614562963 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+23088 (secure 0 0 0) 0x7fdbb8099150 con 0x7fdbc019d820 2023-10-06T21:36:13.075 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-10-06T21:36:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"fsid":"4dcc50a6-648b-11ee-8db5-212e2dc638e7","created":"2023-10-06T21:00:45.549995+0000","modified":"2023-10-06T21:36:10.835095+0000","last_up_change":"2023-10-06T21:07:28.342861+0000","last_in_change":"2023-10-06T21:07:10.699804+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T21:04:02.882358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-06T21:08:14.443239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-06T21:08:15.493688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":128,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e106834-15f2-43e2-86a4-4256e7169c93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6803","nonce":3569177052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6805","nonce":3569177052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6809","nonce":3569177052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3569177052},{"type":"v1","addr":"172.21.15.26:6807","nonce":3569177052}]},"public_addr":"172.21.15.26:6803/3569177052","cluster_addr":"172.21.15.26:6805/3569177052","heartbeat_back_addr":"172.21.15.26:6809/3569177052","heartbeat_front_addr":"172.21.15.26:6807/3569177052","state":["exists","up"]},{"osd":1,"uuid":"cc8d8aba-6316-4996-bb52-6f32af502c53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6801","nonce":4119914658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6803","nonce":4119914658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6807","nonce":4119914658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4119914658},{"type":"v1","addr":"172.21.15.78:6805","nonce":4119914658}]},"public_addr":"172.21.15.78:6801/4119914658","cluster_addr":"172.21.15.78:6803/4119914658","heartbeat_back_addr":"172.21.15.78:6807/4119914658","heartbeat_front_addr":"172.21.15.78:6805/4119914658","state":["exists","up"]},{"osd":2,"uuid":"30a5233e-8442-48f1-af3e-3a0752ff274e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6801","nonce":2625559471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6803","nonce":2625559471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6807","nonce":2625559471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2625559471},{"type":"v1","addr":"172.21.15.167:6805","nonce":2625559471}]},"public_addr":"172.21.15.167:6801/2625559471","cluster_addr":"172.21.15.167:6803/2625559471","heartbeat_back_addr":"172.21.15.167:6807/2625559471","heartbeat_front_addr":"172.21.15.167:6805/2625559471","state":["exists","up"]},{"osd":3,"uuid":"50d5d2ee-19a4-407b-bec4-92dd3e7c684a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6811","nonce":1210213318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6813","nonce":1210213318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6817","nonce":1210213318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1210213318},{"type":"v1","addr":"172.21.15.26:6815","nonce":1210213318}]},"public_addr":"172.21.15.26:6811/1210213318","cluster_addr":"172.21.15.26:6813/1210213318","heartbeat_back_addr":"172.21.15.26:6817/1210213318","heartbeat_front_addr":"172.21.15.26:6815/1210213318","state":["exists","up"]},{"osd":4,"uuid":"22b765a7-0ed8-452b-a897-6b5a7c720e59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6809","nonce":3086176217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6811","nonce":3086176217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6815","nonce":3086176217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3086176217},{"type":"v1","addr":"172.21.15.78:6813","nonce":3086176217}]},"public_addr":"172.21.15.78:6809/3086176217","cluster_addr":"172.21.15.78:6811/3086176217","heartbeat_back_addr":"172.21.15.78:6815/3086176217","heartbeat_front_addr":"172.21.15.78:6813/3086176217","state":["exists","up"]},{"osd":5,"uuid":"4c5d6f6e-9881-41e0-859d-2e8702b70d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6809","nonce":644360584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6811","nonce":644360584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6815","nonce":644360584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":644360584},{"type":"v1","addr":"172.21.15.167:6813","nonce":644360584}]},"public_addr":"172.21.15.167:6809/644360584","cluster_addr":"172.21.15.167:6811/644360584","heartbeat_back_addr":"172.21.15.167:6815/644360584","heartbeat_front_addr":"172.21.15.167:6813/644360584","state":["exists","up"]},{"osd":6,"uuid":"cba40219-6497-43c6-a864-79afb928c52a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6819","nonce":1485072774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6821","nonce":1485072774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6825","nonce":1485072774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1485072774},{"type":"v1","addr":"172.21.15.26:6823","nonce":1485072774}]},"public_addr":"172.21.15.26:6819/1485072774","cluster_addr":"172.21.15.26:6821/1485072774","heartbeat_back_addr":"172.21.15.26:6825/1485072774","heartbeat_front_addr":"172.21.15.26:6823/1485072774","state":["exists","up"]},{"osd":7,"uuid":"28a181f7-1d70-4d23-a95c-3fd0309f720d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6817","nonce":2278627403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6819","nonce":2278627403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6823","nonce":2278627403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":2278627403},{"type":"v1","addr":"172.21.15.78:6821","nonce":2278627403}]},"public_addr":"172.21.15.78:6817/2278627403","cluster_addr":"172.21.15.78:6819/2278627403","heartbeat_back_addr":"172.21.15.78:6823/2278627403","heartbeat_front_addr":"172.21.15.78:6821/2278627403","state":["exists","up"]},{"osd":8,"uuid":"cd2b8aa3-49f9-4263-825a-bd720b77e281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6817","nonce":263711994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6819","nonce":263711994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6823","nonce":263711994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":263711994},{"type":"v1","addr":"172.21.15.167:6821","nonce":263711994}]},"public_addr":"172.21.15.167:6817/263711994","cluster_addr":"172.21.15.167:6819/263711994","heartbeat_back_addr":"172.21.15.167:6823/263711994","heartbeat_front_addr":"172.21.15.167:6821/263711994","state":["exists","up"]},{"osd":9,"uuid":"fc3a0395-6144-480c-aaf7-d2f8f8f22f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6827","nonce":157001209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6829","nonce":157001209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6833","nonce":157001209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":157001209},{"type":"v1","addr":"172.21.15.26:6831","nonce":157001209}]},"public_addr":"172.21.15.26:6827/157001209","cluster_addr":"172.21.15.26:6829/157001209","heartbeat_back_addr":"172.21.15.26:6833/157001209","heartbeat_front_addr":"172.21.15.26:6831/157001209","state":["exists","up"]},{"osd":10,"uuid":"2bda1e71-4772-49f7-8d68-cc56c7f174ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6825","nonce":1966991917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6827","nonce":1966991917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6831","nonce":1966991917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1966991917},{"type":"v1","addr":"172.21.15.78:6829","nonce":1966991917}]},"public_addr":"172.21.15.78:6825/1966991917","cluster_addr":"172.21.15.78:6827/1966991917","heartbeat_back_addr":"172.21.15.78:6831/1966991917","heartbeat_front_addr":"172.21.15.78:6829/1966991917","state":["exists","up"]},{"osd":11,"uuid":"d023d04f-16f7-4e4c-8fef-e2ac8e7db619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6825","nonce":4274564614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6827","nonce":4274564614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6831","nonce":4274564614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":4274564614},{"type":"v1","addr":"172.21.15.167:6829","nonce":4274564614}]},"public_addr":"172.21.15.167:6825/4274564614","cluster_addr":"172.21.15.167:6827/4274564614","heartbeat_back_addr":"172.21.15.167:6831/4274564614","heartbeat_front_addr":"172.21.15.167:6829/4274564614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:18.165430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:03:40.038267+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:01.103445+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:27.629361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:04:50.078626+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:10.848980+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:36.454883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:05:58.765633+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:18.599950+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:06:43.709099+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:05.519248+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T21:07:26.636917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1157730691":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2246884211":"2023-10-07T21:10:19.013203+0000","172.21.15.26:6800/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/1380403953":"2023-10-07T21:01:44.284577+0000","172.21.15.26:0/10926064":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/118366491":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6801/2830651730":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/4033583135":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3325517294":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2078185111":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6801/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/1222842153":"2023-10-07T21:01:17.251649+0000","172.21.15.26:0/3867812316":"2023-10-07T21:10:19.013203+0000","172.21.15.26:0/2655877248":"2023-10-07T21:01:44.284577+0000","172.21.15.26:6800/3098538275":"2023-10-07T21:01:17.251649+0000","172.21.15.26:6800/118366491":"2023-10-07T21:01:44.284577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T21:36:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 msgr2=0x7fdbb00a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 0x7fdbb00a3c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005b00 tx=0x7fdbb4005a90 comp rx=0 tx=0).stop 2023-10-06T21:36:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 msgr2=0x7fdbc019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8007ee0 tx=0x7fdbb800d3b0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 shutdown_connections 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdbc019d820 0x7fdbc019dca0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7fdbb00a1760 0x7fdbb00a3c20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0103130 0x7fdbc0198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 --2- 172.21.15.26:0/3614562963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc0108250 0x7fdbc01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 >> 172.21.15.26:0/3614562963 conn(0x7fdbc00fe950 msgr2=0x7fdbc0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 shutdown_connections 2023-10-06T21:36:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.075+0000 7fdbc5f9d700 1 -- 172.21.15.26:0/3614562963 wait complete. 2023-10-06T21:36:13.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T21:36:13.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:12 smithi026 ceph-mon[98490]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-10-06T21:36:13.167 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:12 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/1772878043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:13.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:12 smithi167 ceph-mon[134494]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-10-06T21:36:13.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:12 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/1772878043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 -- 172.21.15.26:0/3722482573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16480ffbd0 msgr2=0x7f16480fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 --2- 172.21.15.26:0/3722482573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16480ffbd0 0x7f16480fffb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1644009990 tx=0x7f1644009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 -- 172.21.15.26:0/3722482573 shutdown_connections 2023-10-06T21:36:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 --2- 172.21.15.26:0/3722482573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 --2- 172.21.15.26:0/3722482573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16480ffbd0 0x7f16480fffb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 --2- 172.21.15.26:0/3722482573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1648100580 0x7f164810d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 -- 172.21.15.26:0/3722482573 >> 172.21.15.26:0/3722482573 conn(0x7f16480fb830 msgr2=0x7f16480fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 -- 172.21.15.26:0/3722482573 shutdown_connections 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.316+0000 7f164f5ff700 1 -- 172.21.15.26:0/3722482573 wait complete. 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.317+0000 7f164f5ff700 1 Processor -- start 2023-10-06T21:36:13.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 -- start start 2023-10-06T21:36:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 0x7f16481991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e1b0 con 0x7f16480ffbd0 2023-10-06T21:36:13.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e320 con 0x7f1648100580 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164f5ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e4e0 con 0x7f164810d780 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164cb9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 0x7f16481991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164cb9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.26:51950/0 (socket says 172.21.15.26:51950) 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.318+0000 7f164cb9a700 1 -- 172.21.15.26:0/4019475068 learned_addr learned my addr 172.21.15.26:0/4019475068 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164db9c700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 -- 172.21.15.26:0/4019475068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 msgr2=0x7f164819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 -- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 msgr2=0x7f16481991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 0x7f16481991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 -- 172.21.15.26:0/4019475068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440095f0 con 0x7f1648100580 2023-10-06T21:36:13.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164db9c700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164d39b700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 0x7f16481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164cb9a700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f163800d980 tx=0x7f163800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638004220 con 0x7f1648100580 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1638007640 con 0x7f1648100580 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164819e7c0 con 0x7f1648100580 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.319+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638005610 con 0x7f1648100580 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.320+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16481a6000 con 0x7f1648100580 2023-10-06T21:36:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.321+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648067c60 con 0x7f1648100580 2023-10-06T21:36:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.323+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f16380447c0 con 0x7f1648100580 2023-10-06T21:36:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.324+0000 7f163e7fc700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 0x7f16340a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.324+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f16380cd2f0 con 0x7f1648100580 2023-10-06T21:36:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.324+0000 7f164d39b700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 0x7f16340a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.325+0000 7f164d39b700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 0x7f16340a3ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f164400b5c0 tx=0x7f1644005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.327+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1638096570 con 0x7f1648100580 2023-10-06T21:36:13.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:12 smithi078 ceph-mon[134896]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-10-06T21:36:13.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:12 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/1772878043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-06T21:36:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.676+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f164819a520 con 0x7f1648100580 2023-10-06T21:36:13.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.684+0000 7f163e7fc700 1 -- 172.21.15.26:0/4019475068 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1638016090 con 0x7f1648100580 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 msgr2=0x7f16340a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 0x7f16340a3ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f164400b5c0 tx=0x7f1644005fd0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 msgr2=0x7f1648199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f163800d980 tx=0x7f163800dc90 comp rx=0 tx=0).stop 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 shutdown_connections 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f164810d780 0x7f164819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f16340a15f0 0x7f16340a3ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16480ffbd0 0x7f16481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 --2- 172.21.15.26:0/4019475068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1648100580 0x7f1648199710 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 >> 172.21.15.26:0/4019475068 conn(0x7f16480fb830 msgr2=0x7f164810eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 shutdown_connections 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.687+0000 7f164f5ff700 1 -- 172.21.15.26:0/4019475068 wait complete. 2023-10-06T21:36:13.691 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T21:36:13.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T21:36:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 -- 172.21.15.26:0/4261757351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a90073b80 msgr2=0x7f7a9010d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 --2- 172.21.15.26:0/4261757351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a90073b80 0x7f7a9010d1c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009990 tx=0x7f7a84009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 -- 172.21.15.26:0/4261757351 shutdown_connections 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 --2- 172.21.15.26:0/4261757351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a9010d700 0x7f7a9010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 --2- 172.21.15.26:0/4261757351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 0x7f7a90073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 --2- 172.21.15.26:0/4261757351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a90073b80 0x7f7a9010d1c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 -- 172.21.15.26:0/4261757351 >> 172.21.15.26:0/4261757351 conn(0x7f7a900fc930 msgr2=0x7f7a900fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 -- 172.21.15.26:0/4261757351 shutdown_connections 2023-10-06T21:36:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.923+0000 7f7a969ff700 1 -- 172.21.15.26:0/4261757351 wait complete. 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.924+0000 7f7a969ff700 1 Processor -- start 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.924+0000 7f7a969ff700 1 -- start start 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 0x7f7a90199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 0x7f7a901996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 0x7f7a9019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9019e160 con 0x7f7a9010d700 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9019e2d0 con 0x7f7a90073b80 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a969ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9019e490 con 0x7f7a90073260 2023-10-06T21:36:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a94f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 0x7f7a9019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 0x7f7a901996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a94f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 0x7f7a9019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41236/0 (socket says 172.21.15.26:41236) 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a94f9c700 1 -- 172.21.15.26:0/16638217 learned_addr learned my addr 172.21.15.26:0/16638217 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8ffff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 0x7f7a90199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 -- 172.21.15.26:0/16638217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 msgr2=0x7f7a90199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 0x7f7a90199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 -- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 msgr2=0x7f7a9019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 0x7f7a9019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.925+0000 7f7a8f7fe700 1 -- 172.21.15.26:0/16638217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840095f0 con 0x7f7a90073b80 2023-10-06T21:36:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a8f7fe700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 0x7f7a901996c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7a84000c00 tx=0x7f7a84005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8401d070 con 0x7f7a90073b80 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a84021410 con 0x7f7a90073b80 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84017650 con 0x7f7a90073b80 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9019e710 con 0x7f7a90073b80 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.926+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a901a5fb0 con 0x7f7a90073b80 2023-10-06T21:36:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.927+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90067c60 con 0x7f7a90073b80 2023-10-06T21:36:13.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.929+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7f7a84021580 con 0x7f7a90073b80 2023-10-06T21:36:13.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.930+0000 7f7a8d7fa700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 0x7f7a780a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:13.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.930+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8800+0+0 (secure 0 0 0) 0x7f7a840ce510 con 0x7f7a90073b80 2023-10-06T21:36:13.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.930+0000 7f7a8ffff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 0x7f7a780a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:13.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.931+0000 7f7a8ffff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 0x7f7a780a3b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009c60 tx=0x7f7a80009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:13.934+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7a84097770 con 0x7f7a90073b80 2023-10-06T21:36:14.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.033 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: starting mds.i at 2023-10-06T21:36:14.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3614562963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:14.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/4019475068' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mon[134494]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.034 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-mds[170785]: -- 172.21.15.167:0/2083706313 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564bad2711e0 con 0x564bad246800 2023-10-06T21:36:14.035 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:36:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: starting mds.c at 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a[98466]: 2023-10-06T21:36:13.677+0000 7f5579efb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3614562963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/4019475068' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T21:36:14.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mon[98490]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-mds[149961]: -- 172.21.15.26:0/2700937876 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555c68bd1e0 con 0x5555c6892c00 2023-10-06T21:36:14.075 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:36:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: starting mds.a at 2023-10-06T21:36:14.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mds[171216]: -- 172.21.15.78:0/1577733227 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4315771e0 con 0x55f43154cc00 2023-10-06T21:36:14.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: starting mds.e at 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3614562963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/4019475068' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: osdmap e129: 12 total, 12 up, 12 in 2023-10-06T21:36:14.122 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mon[134896]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mds[171895]: -- 172.21.15.78:0/3309078742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561367e9f1e0 con 0x561367e74800 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: starting mds.b at 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: ignoring --setuser ceph since I am not root 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: ignoring --setgroup ceph since I am not root 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-mds[171555]: -- 172.21.15.78:0/2664943467 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3c48511e0 con 0x55d3c4826c00 2023-10-06T21:36:14.123 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:36:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: starting mds.h at 2023-10-06T21:36:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.195+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7a90113090 con 0x7f7a90073b80 2023-10-06T21:36:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.688+0000 7f7a8d7fa700 1 -- 172.21.15.26:0/16638217 <== mon.2 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a8409ad90 con 0x7f7a90073b80 2023-10-06T21:36:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 msgr2=0x7f7a780a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 0x7f7a780a3b30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009c60 tx=0x7f7a80009260 comp rx=0 tx=0).stop 2023-10-06T21:36:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 msgr2=0x7f7a901996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 0x7f7a901996c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7a84000c00 tx=0x7f7a84005740 comp rx=0 tx=0).stop 2023-10-06T21:36:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 shutdown_connections 2023-10-06T21:36:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7f7a780a1670 0x7f7a780a3b30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a9010d700 0x7f7a9019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a90073260 0x7f7a90199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 --2- 172.21.15.26:0/16638217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90073b80 0x7f7a901996c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 >> 172.21.15.26:0/16638217 conn(0x7f7a900fc930 msgr2=0x7f7a9010f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 shutdown_connections 2023-10-06T21:36:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.692+0000 7f7a969ff700 1 -- 172.21.15.26:0/16638217 wait complete. 2023-10-06T21:36:14.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T21:36:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 -- 172.21.15.26:0/2266041751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1010b0 msgr2=0x7efccc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 --2- 172.21.15.26:0/2266041751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1010b0 0x7efccc101490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0098f0 tx=0x7efcbc009c00 comp rx=0 tx=0).stop 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 -- 172.21.15.26:0/2266041751 shutdown_connections 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 --2- 172.21.15.26:0/2266041751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1061d0 0x7efccc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 --2- 172.21.15.26:0/2266041751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1010b0 0x7efccc101490 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 --2- 172.21.15.26:0/2266041751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc101a60 0x7efccc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 -- 172.21.15.26:0/2266041751 >> 172.21.15.26:0/2266041751 conn(0x7efccc0fc930 msgr2=0x7efccc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 -- 172.21.15.26:0/2266041751 shutdown_connections 2023-10-06T21:36:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.925+0000 7efcd09ff700 1 -- 172.21.15.26:0/2266041751 wait complete. 2023-10-06T21:36:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.926+0000 7efcd09ff700 1 Processor -- start 2023-10-06T21:36:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 -- start start 2023-10-06T21:36:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1010b0 0x7efccc197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc19b9c0 0x7efccc19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc19c440 con 0x7efccc1061d0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc19c5b0 con 0x7efccc19b9c0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcd09ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc19c770 con 0x7efccc1010b0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcc3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41262/0 (socket says 172.21.15.26:41262) 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcc3fff700 1 -- 172.21.15.26:0/80900929 learned_addr learned my addr 172.21.15.26:0/80900929 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efccad9d700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc19b9c0 0x7efccc19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.927+0000 7efcca59c700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1010b0 0x7efccc197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 -- 172.21.15.26:0/80900929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1010b0 msgr2=0x7efccc197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1010b0 0x7efccc197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 -- 172.21.15.26:0/80900929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc19b9c0 msgr2=0x7efccc19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc19b9c0 0x7efccc19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 -- 172.21.15.26:0/80900929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc009520 con 0x7efccc1061d0 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc3fff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4009690 con 0x7efccc1061d0 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb400ce80 con 0x7efccc1061d0 2023-10-06T21:36:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efccc1a3cd0 con 0x7efccc1061d0 2023-10-06T21:36:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4018420 con 0x7efccc1061d0 2023-10-06T21:36:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.928+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc1a41b0 con 0x7efccc1061d0 2023-10-06T21:36:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.929+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc067c60 con 0x7efccc1061d0 2023-10-06T21:36:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.931+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7efcb400c9c0 con 0x7efccc1061d0 2023-10-06T21:36:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.933+0000 7efcc37fe700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 0x7efcb80a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.933+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8800+0+0 (secure 0 0 0) 0x7efcb4014070 con 0x7efccc1061d0 2023-10-06T21:36:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.933+0000 7efcca59c700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 0x7efcb80a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.934+0000 7efcca59c700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 0x7efcb80a3ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005e90 tx=0x7efcbc019040 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:14.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:14.937+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efcb4052100 con 0x7efccc1061d0 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/16638217' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: Cluster is now healthy 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6837/485194142,v1:172.21.15.167:6839/485194142] up:boot 2023-10-06T21:36:15.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.167:6836/1396489343,v1:172.21.15.167:6838/1396489343] up:boot 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6836/1508446965,v1:172.21.15.78:6838/1508446965] up:boot 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.26:6840/3047778359,v1:172.21.15.26:6841/3047778359] up:boot 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6834/4188412087,v1:172.21.15.78:6835/4188412087] up:boot 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: mds.? [v2:172.21.15.78:6837/3799094590,v1:172.21.15.78:6839/3799094590] up:boot 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: fsmap 12 up:standby 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:36:15.075 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:14 smithi026 ceph-mon[98490]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:36:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.152+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efccc198370 con 0x7efccc1061d0 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/16638217' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T21:36:15.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: Cluster is now healthy 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6837/485194142,v1:172.21.15.167:6839/485194142] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.167:6836/1396489343,v1:172.21.15.167:6838/1396489343] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6836/1508446965,v1:172.21.15.78:6838/1508446965] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.26:6840/3047778359,v1:172.21.15.26:6841/3047778359] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6834/4188412087,v1:172.21.15.78:6835/4188412087] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: mds.? [v2:172.21.15.78:6837/3799094590,v1:172.21.15.78:6839/3799094590] up:boot 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: fsmap 12 up:standby 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:36:15.284 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:36:15.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:36:15.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:36:15.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:36:15.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:14 smithi167 ceph-mon[134494]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:36:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-06T21:36:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/16638217' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-06T21:36:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-06T21:36:15.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: Cluster is now healthy 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6837/485194142,v1:172.21.15.167:6839/485194142] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.167:6836/1396489343,v1:172.21.15.167:6838/1396489343] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6836/1508446965,v1:172.21.15.78:6838/1508446965] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.26:6840/3047778359,v1:172.21.15.26:6841/3047778359] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6834/4188412087,v1:172.21.15.78:6835/4188412087] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: mds.? [v2:172.21.15.78:6837/3799094590,v1:172.21.15.78:6839/3799094590] up:boot 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: fsmap 12 up:standby 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-06T21:36:15.372 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-06T21:36:15.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-06T21:36:15.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-06T21:36:15.373 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:14 smithi078 ceph-mon[134896]: from='mgr.15306 172.21.15.26:0/2971792937' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-06T21:36:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.902+0000 7efcc37fe700 1 -- 172.21.15.26:0/80900929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7efcb4099f20 con 0x7efccc1061d0 2023-10-06T21:36:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 msgr2=0x7efcb80a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 0x7efcb80a3ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005e90 tx=0x7efcbc019040 comp rx=0 tx=0).stop 2023-10-06T21:36:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 msgr2=0x7efccc197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400ed00 comp rx=0 tx=0).stop 2023-10-06T21:36:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 shutdown_connections 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efccc19b9c0 0x7efccc19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efccc1010b0 0x7efccc197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7efcb80a15f0 0x7efcb80a3ab0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efcbc005e90 tx=0x7efcbc019040 comp rx=0 tx=0).stop 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 --2- 172.21.15.26:0/80900929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efccc1061d0 0x7efccc197560 secure :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e9f0 tx=0x7efcb400ed00 comp rx=0 tx=0).stop 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.909+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 >> 172.21.15.26:0/80900929 conn(0x7efccc0fc930 msgr2=0x7efccc0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.910+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 shutdown_connections 2023-10-06T21:36:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:15.910+0000 7efcd09ff700 1 -- 172.21.15.26:0/80900929 wait complete. 2023-10-06T21:36:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-10-06T21:36:15.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T21:36:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/2715306053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb84101ff0 msgr2=0x7ffb841023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/2715306053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb84101ff0 0x7ffb841023d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009990 tx=0x7ffb74009ca0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/2715306053 shutdown_connections 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/2715306053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb8410b410 0x7ffb8410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/2715306053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb84101ff0 0x7ffb841023d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/2715306053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb8410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/2715306053 >> 172.21.15.26:0/2715306053 conn(0x7ffb840fb830 msgr2=0x7ffb840fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/2715306053 shutdown_connections 2023-10-06T21:36:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.146+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/2715306053 wait complete. 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.147+0000 7ffb8a9ff700 1 Processor -- start 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.147+0000 7ffb8a9ff700 1 -- start start 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb84101ff0 0x7ffb84197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 0x7ffb8419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8419c020 con 0x7ffb84102910 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8419c190 con 0x7ffb84101ff0 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb8a9ff700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8419c350 con 0x7ffb8410b410 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41280/0 (socket says 172.21.15.26:41280) 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 -- 172.21.15.26:0/3797552118 learned_addr learned my addr 172.21.15.26:0/3797552118 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb83fff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb84101ff0 0x7ffb84197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb88f9c700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 0x7ffb8419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 -- 172.21.15.26:0/3797552118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 msgr2=0x7ffb8419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 0x7ffb8419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 -- 172.21.15.26:0/3797552118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb84101ff0 msgr2=0x7ffb84197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.148+0000 7ffb837fe700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb84101ff0 0x7ffb84197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb837fe700 1 -- 172.21.15.26:0/3797552118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009520 con 0x7ffb84102910 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb88f9c700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 0x7ffb8419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb837fe700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9f0 tx=0x7ffb7800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800cbd0 con 0x7ffb84102910 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb740095f0 con 0x7ffb84102910 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb78009750 con 0x7ffb84102910 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003d60 con 0x7ffb84102910 2023-10-06T21:36:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.149+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8419c820 con 0x7ffb84102910 2023-10-06T21:36:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.150+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb84067c60 con 0x7ffb84102910 2023-10-06T21:36:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.152+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 136134+0+0 (secure 0 0 0) 0x7ffb78003f00 con 0x7ffb84102910 2023-10-06T21:36:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.154+0000 7ffb817fa700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 0x7ffb6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T21:36:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.154+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8275+0+0 (secure 0 0 0) 0x7ffb78014070 con 0x7ffb84102910 2023-10-06T21:36:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.154+0000 7ffb83fff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 0x7ffb6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T21:36:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.155+0000 7ffb83fff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 0x7ffb6c0a3ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b2e0 tx=0x7ffb74005fb0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T21:36:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.158+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffb780967d0 con 0x7ffb84102910 2023-10-06T21:36:16.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:15 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:16.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:15 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:16.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.367+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb84198390 con 0x7ffb84102910 2023-10-06T21:36:16.370 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:15 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.905+0000 7ffb817fa700 1 -- 172.21.15.26:0/3797552118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffb78099df0 con 0x7ffb84102910 2023-10-06T21:36:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.909+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 msgr2=0x7ffb6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.909+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 0x7ffb6c0a3ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b2e0 tx=0x7ffb74005fb0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.909+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 msgr2=0x7ffb84197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T21:36:16.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.909+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9f0 tx=0x7ffb7800ed00 comp rx=0 tx=0).stop 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 shutdown_connections 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb8410b410 0x7ffb8419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb84101ff0 0x7ffb84197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:6800/3248613920,v1:172.21.15.26:6801/3248613920] conn(0x7ffb6c0a15f0 0x7ffb6c0a3ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 --2- 172.21.15.26:0/3797552118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb84102910 0x7ffb84197580 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T21:36:16.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 >> 172.21.15.26:0/3797552118 conn(0x7ffb840fb830 msgr2=0x7ffb8410d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T21:36:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 shutdown_connections 2023-10-06T21:36:16.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-10-06T21:36:16.910+0000 7ffb8a9ff700 1 -- 172.21.15.26:0/3797552118 wait complete. 2023-10-06T21:36:16.923 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-10-06T21:36:16.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:36:17.082 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:36:17.150 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:36:17.182 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-06T21:36:17.182 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:36:17.226 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:36:17.243 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-06T21:36:17.260 INFO:tasks.cephadm:Stopping all daemons... 2023-10-06T21:36:17.261 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-06T21:36:17.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a 2023-10-06T21:36:17.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:16 smithi167 ceph-mon[134494]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-10-06T21:36:17.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:16 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T21:36:17.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:16 smithi167 ceph-mon[134494]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:36:17.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:16 smithi167 ceph-mon[134494]: from='client.? 172.21.15.26:0/3797552118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:17.323 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:16 smithi026 ceph-mon[98490]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-10-06T21:36:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:16 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T21:36:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:16 smithi026 ceph-mon[98490]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:36:17.324 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:16 smithi026 ceph-mon[98490]: from='client.? 172.21.15.26:0/3797552118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:16 smithi078 ceph-mon[134896]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-10-06T21:36:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:16 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/80900929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-06T21:36:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:16 smithi078 ceph-mon[134896]: osdmap e130: 12 total, 12 up, 12 in 2023-10-06T21:36:17.371 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:16 smithi078 ceph-mon[134896]: from='client.? 172.21.15.26:0/3797552118' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-06T21:36:17.823 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:17 smithi026 systemd[1]: Stopping Ceph mon.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:18.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a[98466]: 2023-10-06T21:36:17.934+0000 7f557f706700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:18.221 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a[98466]: 2023-10-06T21:36:17.934+0000 7f557f706700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-06T21:36:18.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Oct 06 21:36:18 smithi026 bash[182271]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-a 2023-10-06T21:36:19.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-06T21:36:19.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:19.106 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-06T21:36:19.106 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-06T21:36:19.106 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.b 2023-10-06T21:36:19.564 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:19 smithi078 systemd[1]: Stopping Ceph mon.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:19.848 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-b[134872]: 2023-10-06T21:36:19.560+0000 7fa79b62b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:19.848 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-b[134872]: 2023-10-06T21:36:19.560+0000 7fa79b62b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-06T21:36:20.121 INFO:journalctl@ceph.mon.b.smithi078.stdout:Oct 06 21:36:20 smithi078 bash[180391]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-b 2023-10-06T21:36:20.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-06T21:36:20.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:20.331 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-06T21:36:20.331 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-06T21:36:20.331 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.c 2023-10-06T21:36:20.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:20 smithi167 systemd[1]: Stopping Ceph mon.c for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:20.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-c[134467]: 2023-10-06T21:36:20.678+0000 7f300839a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:20.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-c[134467]: 2023-10-06T21:36:20.678+0000 7f300839a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-06T21:36:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Oct 06 21:36:20 smithi167 bash[180244]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mon-c 2023-10-06T21:36:21.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-06T21:36:21.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:21.273 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-06T21:36:21.273 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-06T21:36:21.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x 2023-10-06T21:36:21.573 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:36:21 smithi026 systemd[1]: Stopping Ceph mgr.x for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:22.065 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:36:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:36:21.801+0000 7fc9e3e00700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:22.065 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:36:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x[98734]: 2023-10-06T21:36:21.801+0000 7fc9e3e00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-06T21:36:22.324 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Oct 06 21:36:22 smithi026 bash[182509]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-x 2023-10-06T21:36:22.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-06T21:36:22.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:22.621 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-06T21:36:22.621 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-06T21:36:22.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.y 2023-10-06T21:36:23.028 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:36:22 smithi078 systemd[1]: Stopping Ceph mgr.y for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:23.323 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:36:23 smithi078 bash[180532]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-y 2023-10-06T21:36:23.508 INFO:journalctl@ceph.mgr.y.smithi078.stdout:Oct 06 21:36:23 smithi078 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-06T21:36:23.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-06T21:36:23.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:23.547 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-06T21:36:23.548 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-06T21:36:23.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.z 2023-10-06T21:36:23.985 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:36:23 smithi167 systemd[1]: Stopping Ceph mgr.z for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:24.247 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Oct 06 21:36:24 smithi167 bash[180387]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mgr-z 2023-10-06T21:36:24.438 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-06T21:36:24.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:24.474 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-06T21:36:24.475 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-06T21:36:24.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.0 2023-10-06T21:36:24.823 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:36:24 smithi026 systemd[1]: Stopping Ceph osd.0 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:25.323 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:36:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0[116687]: 2023-10-06T21:36:24.982+0000 7f9b8abf4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:25.324 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:36:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0[116687]: 2023-10-06T21:36:24.983+0000 7f9b8abf4700 -1 osd.0 131 *** Got signal Terminated *** 2023-10-06T21:36:25.324 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:36:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0[116687]: 2023-10-06T21:36:24.983+0000 7f9b8abf4700 -1 osd.0 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:36:31.323 INFO:journalctl@ceph.osd.0.smithi026.stdout:Oct 06 21:36:31 smithi026 bash[182655]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-0 2023-10-06T21:36:32.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-06T21:36:32.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:32.930 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-06T21:36:32.930 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-06T21:36:32.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.1 2023-10-06T21:36:33.288 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:36:32 smithi078 systemd[1]: Stopping Ceph osd.1 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:33.621 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:36:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:36:33.284+0000 7fd60e4bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:33.621 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:36:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:36:33.284+0000 7fd60e4bc700 -1 osd.1 131 *** Got signal Terminated *** 2023-10-06T21:36:33.621 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:36:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1[139985]: 2023-10-06T21:36:33.284+0000 7fd60e4bc700 -1 osd.1 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:36:39.621 INFO:journalctl@ceph.osd.1.smithi078.stdout:Oct 06 21:36:39 smithi078 bash[180679]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-1 2023-10-06T21:36:40.733 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-06T21:36:40.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:40.773 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-06T21:36:40.773 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-06T21:36:40.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.2 2023-10-06T21:36:41.138 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:36:40 smithi167 systemd[1]: Stopping Ceph osd.2 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:41.533 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:36:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:36:41.135+0000 7f13fc267700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:41.533 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:36:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:36:41.135+0000 7f13fc267700 -1 osd.2 131 *** Got signal Terminated *** 2023-10-06T21:36:41.533 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:36:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2[140344]: 2023-10-06T21:36:41.135+0000 7f13fc267700 -1 osd.2 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:36:47.283 INFO:journalctl@ceph.osd.2.smithi167.stdout:Oct 06 21:36:46 smithi167 bash[180533]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-2 2023-10-06T21:36:48.275 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-06T21:36:48.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:48.315 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-06T21:36:48.315 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-06T21:36:48.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.3 2023-10-06T21:36:48.823 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:36:48 smithi026 systemd[1]: Stopping Ceph osd.3 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:49.323 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:36:48 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3[125269]: 2023-10-06T21:36:48.822+0000 7f88e7bd6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:49.324 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:36:48 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3[125269]: 2023-10-06T21:36:48.822+0000 7f88e7bd6700 -1 osd.3 131 *** Got signal Terminated *** 2023-10-06T21:36:49.324 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:36:48 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3[125269]: 2023-10-06T21:36:48.822+0000 7f88e7bd6700 -1 osd.3 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:36:52.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:51 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:51.757+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:52.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:52 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:52.203+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:53.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:52 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:52.784+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:53.532 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:53.073+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:53.620 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:53 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:53.205+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:54.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:53 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:53.796+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:54.533 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:54.060+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:54.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:54 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:54.206+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:54.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:54 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:54.791+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:54.935 INFO:journalctl@ceph.osd.3.smithi026.stdout:Oct 06 21:36:54 smithi026 bash[182938]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-3 2023-10-06T21:36:55.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:55.085+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:55.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:55.013+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:36:55.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:54 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:54.933+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:36:55.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:55 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:55.243+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:55.963 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:55.767+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:56.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:56 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:56.071+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:55.969+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:36:56.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:55 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:55.961+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:36:56.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:56 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:56.212+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:56.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-06T21:36:56.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:36:56.811 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-06T21:36:56.811 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-06T21:36:56.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.4 2023-10-06T21:36:56.965 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:56 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:56.743+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:57.121 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:36:56 smithi078 systemd[1]: Stopping Ceph osd.4 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:36:57.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:57 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:57.054+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:57.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:56 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:56.960+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:36:57.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:56 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:56.963+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:36:57.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:57.203+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:57.621 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:36:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:36:57.163+0000 7fda86651700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:36:57.621 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:36:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:36:57.163+0000 7fda86651700 -1 osd.4 131 *** Got signal Terminated *** 2023-10-06T21:36:57.621 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:36:57 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:36:57.163+0000 7fda86651700 -1 osd.4 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:36:58.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:57 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:57.760+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:58.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:57 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:57.978+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:36:58.282 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:58 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:58.048+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:58.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:57 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:57.913+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:36:58.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:58 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:58.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:36:59.074 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:58 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:58.793+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:36:59.075 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:58 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:58.983+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:36:59.282 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:36:59 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:36:59.067+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:36:59.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:58 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:58.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:36:59.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:36:59 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:36:59.144+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:00.074 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:36:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:36:59.808+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:00.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:36:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:36:59.981+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:00.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:00.019+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:00.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:36:59 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:36:59.894+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:00.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:00 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:00.126+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:01.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:00.776+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:01.074 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:00.776+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:01.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:00.997+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:01.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:00 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:00.997+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:01.234 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:00.970+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:01.235 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:00.970+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:01.235 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:00.932+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:01.235 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:00 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:00.932+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:01.504 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:01 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:01.137+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:01.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:01.232+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:01.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:01 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:01.501+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:02.073 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:01.793+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:02.074 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:01 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:01.793+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:02.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:02.004+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:02.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:02.004+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:02.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:01.963+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:02.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:01.963+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:02.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:01.968+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:02.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:01 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:01.968+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:02.371 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:37:01 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4[148023]: 2023-10-06T21:37:01.978+0000 7fda7f440700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:36.341605+0000 front 2023-10-06T21:36:36.341634+0000 (oldest deadline 2023-10-06T21:37:01.041915+0000) 2023-10-06T21:37:02.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:02 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:02.142+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:02.770 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:02 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:02.515+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:02.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:02 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:02.281+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:03.050 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:37:02 smithi078 bash[180953]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-4 2023-10-06T21:37:03.239 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:02 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:02.946+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:03.240 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:02 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:02.946+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:03.240 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:03.013+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:03.240 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:03.013+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:03.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:02.837+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:03.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:02.837+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:03.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:02.998+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:03.325 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:02.998+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:03.507 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:03 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:03.157+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:03.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:03.236+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:03.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:03 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:03.504+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:04.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-06T21:37:04.235 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:37:04 smithi078 systemd[1]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.4.service: Succeeded. 2023-10-06T21:37:04.235 INFO:journalctl@ceph.osd.4.smithi078.stdout:Oct 06 21:37:04 smithi078 systemd[1]: Stopped Ceph osd.4 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7. 2023-10-06T21:37:04.236 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:04 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:04.195+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:04.236 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:04 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:04.195+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:04.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:04.249 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-06T21:37:04.249 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-06T21:37:04.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.5 2023-10-06T21:37:04.280 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:03.996+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:04.281 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:03.996+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:04.281 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:03.985+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:04.281 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:03 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:03.985+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:04.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:03 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:03.840+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:04.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:03 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:03.840+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:04.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:03 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:03.989+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:04.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:03 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:03.989+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:04.532 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 systemd[1]: Stopping Ceph osd.5 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:04.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:04.279+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:04.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:04 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:04.553+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:04.941 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:04.597+0000 7f6c27cc5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:04.941 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:04.597+0000 7f6c27cc5700 -1 osd.5 131 *** Got signal Terminated *** 2023-10-06T21:37:04.941 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:04.597+0000 7f6c27cc5700 -1 osd.5 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:04.941 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:04.938+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:05.282 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:04 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:04.938+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:05.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:05 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:05.035+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:05.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:05 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:05.035+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:05.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:04 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:04.872+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:05.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:04 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:04.872+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:05.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:04 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:04.947+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:05.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:04 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:04.947+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:05.575 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:05 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:05.182+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:05.575 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:05 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:05.182+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:05.782 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:05 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:05.292+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:05.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:05 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:05.572+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:06.252 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:06 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:06.016+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:06.253 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:06 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:06.016+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:06.253 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:05 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:05.972+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:06.253 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:05 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:05.972+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:06.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:05 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:05.901+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:06.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:05 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:05.901+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:06.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:05 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:05.936+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:06.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:05 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:05.936+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:06.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:06 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:06.250+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:06.542 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:06 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:06.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:06.542 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:06 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:06.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:06.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:06 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:06.539+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:07.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:06 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:06.947+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:07.283 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:06 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:06.947+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:07.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:07.042+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:07.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:07.042+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:07.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:07.042+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:07.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:06 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:06.880+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:07.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:06 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:06.880+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:07.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:06 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:06.912+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:07.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:06 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:06.912+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:07.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:07 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:07.116+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:07.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:07 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:07.116+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:07.782 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:07.282+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:07.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:07 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:07.511+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:08.262 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:07.959+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:08.263 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:07 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:07.959+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:08.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:08.075+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:08.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:08.075+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:08.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:08.075+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:08.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:07.889+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:08.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:07.889+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:08.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:07.889+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:08.325 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:07.962+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:08.325 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:07 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:07.962+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:08.496 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:08 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:08.145+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:08.496 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:08 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:08.145+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:08.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:08.260+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:08.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:08 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:08.494+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:09.271 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:08.970+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.661804+0000 front 2023-10-06T21:36:27.661791+0000 (oldest deadline 2023-10-06T21:36:52.361762+0000) 2023-10-06T21:37:09.272 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:08 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5[148450]: 2023-10-06T21:37:08.970+0000 7f6c242bb700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.763532+0000 front 2023-10-06T21:36:35.763586+0000 (oldest deadline 2023-10-06T21:37:00.463311+0000) 2023-10-06T21:37:09.272 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:09.070+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:09.272 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:09.070+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:09.272 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:09.070+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:09.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:08.914+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:09.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:08.914+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:09.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:08.881+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:09.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:08.881+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:09.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:08.881+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:09.532 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:09 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:09.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:09.533 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:09 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:09.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:09.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:09.269+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:09.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:09 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:09.269+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:09.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:09 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:09.529+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:09.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:09 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:09.529+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:10.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:09.956+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:10.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:09.956+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:10.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:09.901+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:10.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:09.901+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:10.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:09 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:09.901+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:10.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:10.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:10.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:10.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:10.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:10.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:10.473 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:10.081+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:10.474 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:10.081+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:10.474 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:10.081+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:10.474 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:10.252+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:10.474 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:10 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:10.252+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:10.785 INFO:journalctl@ceph.osd.5.smithi167.stdout:Oct 06 21:37:10 smithi167 bash[180809]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-5 2023-10-06T21:37:10.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:10.566+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:10.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:10 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:10.566+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:11.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:10.906+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:11.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:10.906+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:11.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:10.906+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:11.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:10.975+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:11.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:10 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:10.975+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:11.453 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:11.272+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:11.453 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:11.272+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:11.453 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:11.033+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:11.453 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:11.033+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:11.453 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:11 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:11.033+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:11.552 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:11.138+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:11.552 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:11.138+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:11.552 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:11.138+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:11.853 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-06T21:37:11.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:11.549+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:11.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:11 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:11.549+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:11.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:11.934 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-06T21:37:11.934 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-06T21:37:11.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.6 2023-10-06T21:37:12.273 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:12.270+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:12.273 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:12.052+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:12.273 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:12.052+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:12.273 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:12.052+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:12.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:11.974+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:11.974+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:11.974+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:11.882+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:11.882+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:11 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:11.882+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:12.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 systemd[1]: Stopping Ceph osd.6 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:12.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:12 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:12.270+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:12.599 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:12.178+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:12.599 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:12.178+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:12.600 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:12.179+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:12.823 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.443+0000 7fa0f9338700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:12.824 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.443+0000 7fa0f9338700 -1 osd.6 131 *** Got signal Terminated *** 2023-10-06T21:37:12.824 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.443+0000 7fa0f9338700 -1 osd.6 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:12.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:12.596+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:12.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:12 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:12.596+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.840+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.840+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:12.840+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:12.931+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:12.931+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:13.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:12 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:12.931+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:13.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:13.227+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:13.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:13.227+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:13.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:13.072+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:13.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:13.072+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:13.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:13 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:13.072+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:13.576 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:13.140+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:13.576 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:13.140+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:13.576 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:13.140+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:13.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:13.574+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:13.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:13 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:13.574+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:14.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:13.880+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:14.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:13.880+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:14.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:13.880+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:14.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:13.885+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:14.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:13.885+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:14.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:13.885+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:14.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:14.099+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:14.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:14.099+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:14.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:14.099+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:14.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:14.200+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:14.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:14 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:14.200+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:14.592 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:14.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:14.592 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:14.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:14.592 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:14.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:14.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:14.590+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:14.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:14 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:14.590+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:14.879+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:14.879+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:14.879+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:14.883+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:14.883+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:15.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:14 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:14.883+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:15.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:15.205+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:15.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:15.205+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:15.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:15.077+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:15.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:15.077+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:15.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:15.077+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:15.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:15 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:15.077+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:15.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:15.157+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:15.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:15.157+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:15.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:15.157+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:16.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:15.623+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:16.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:15 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:15.623+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:15.886+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:15.886+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:15.886+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:15.907+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:15.907+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:16.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:15 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:15.907+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:16.209+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:16.209+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:16.046+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:16.046+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:16.046+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:16.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:16 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:16.046+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:16.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:16.184+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:16.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:16.184+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:16.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:16.184+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:16.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:16.184+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:17.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:16.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:17.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:16.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:17.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:16.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:17.323 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:16.917+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.585973+0000 front 2023-10-06T21:36:27.586465+0000 (oldest deadline 2023-10-06T21:36:51.686150+0000) 2023-10-06T21:37:17.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:16.917+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.989003+0000 front 2023-10-06T21:36:37.988850+0000 (oldest deadline 2023-10-06T21:37:00.189644+0000) 2023-10-06T21:37:17.324 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6[133282]: 2023-10-06T21:37:16.917+0000 7fa0f3129700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.791669+0000 front 2023-10-06T21:36:44.791549+0000 (oldest deadline 2023-10-06T21:37:07.091540+0000) 2023-10-06T21:37:17.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:16.945+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:17.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:16.945+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:17.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:16 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:16.945+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:17.230+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:17.230+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:17.230+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:17.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:17.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:17.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:17.200+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:17.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:17.200+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:17.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:17.200+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:17.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:17.200+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:18.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:17.667+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:18.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:17.667+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:18.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:17 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:17.667+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:18.229 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:17.971+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:18.229 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:17.971+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:18.229 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:17.971+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:18.229 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:17 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:17.971+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:18.204+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:18.204+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:18.204+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.986+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.986+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.986+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:18.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:17 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:17.986+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:18.573 INFO:journalctl@ceph.osd.6.smithi026.stdout:Oct 06 21:37:18 smithi026 bash[183212]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-6 2023-10-06T21:37:18.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:18.172+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:18.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:18.172+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:18.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:18.172+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:18.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:18.172+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:19.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:18.682+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:19.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:18.682+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:19.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:18.682+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:19.263 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:18.930+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:19.263 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:18.930+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:19.263 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:18.930+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:19.263 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:18 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:18.930+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:19 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:19.164+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:19 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:19.164+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:19 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:19.164+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:18.989+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:18.989+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:19.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:18.989+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:19.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:18 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:18.989+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:19.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:19.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:19.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:19.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:19.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:19.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:19.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:19.153+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:20.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:19.939+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:20.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:19.939+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:20.075 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:19.939+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:20.075 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:19 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:19.939+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:20.120 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:19.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:20.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:19.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:20.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:19 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:19.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:20.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-06T21:37:20.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:20.403 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-06T21:37:20.403 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-06T21:37:20.403 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.7 2023-10-06T21:37:20.434 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:20.162+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:20.434 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:20.162+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:20.434 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:20.162+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:20.434 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:20.162+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:20.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:20.209+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:20.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:20.209+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:20.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:20.209+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:20.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:20.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:20.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:20.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:20.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:20.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:20.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:20 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:20.032+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:20.763 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 systemd[1]: Stopping Ceph osd.7 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:20.764 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.632+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:20.764 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.632+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:20.764 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.632+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:20.764 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.761+0000 7f1627413700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:20.764 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.761+0000 7f1627413700 -1 osd.7 131 *** Got signal Terminated *** 2023-10-06T21:37:21.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:20 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:20.761+0000 7f1627413700 -1 osd.7 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:21.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:20 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:20.942+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:21.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:20 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:20.942+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:21.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:20 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:20.942+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:21.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:20 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:20.942+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:21.190+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:21.190+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:21.190+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:21.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:21.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:21.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:21.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:21 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:21.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:21.598 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:21.151+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:21.598 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:21.151+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:21.598 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:21.151+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:21.598 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:21.151+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:21.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:21.596+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:21.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:21.596+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:21.871 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:21 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:21.596+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:22.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:21.895+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:22.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:21.895+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:22.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:21.895+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:22.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:21 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:21.895+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:22.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:22.212+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:22.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:22.212+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:22.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:22.212+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:22.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:22.038+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:22.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:22.038+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:22.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:22.038+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:22.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:22 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:22.038+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:22.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:22.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:22.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:22.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:22.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:22.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:22.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:22.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:23.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:22.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:23.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:22.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:23.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:22.643+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:23.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:23.017+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:23.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:23.017+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:23.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:23.017+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:23.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:23.017+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:23.283 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:23.233+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:23.284 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:23.233+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:23.284 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:23.233+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:23.284 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:23 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:23.233+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:23.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:22.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:23.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:22.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:23.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:22.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:23.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:22 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:22.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:23.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:23.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:23.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:23.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:23.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:23.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:23.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:23.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:24.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:23.686+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:24.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:23.686+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:24.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:23 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:23.686+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:24.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:24.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:24.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:24.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:24.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:24.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:24.264 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:24.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:24.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:23 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:23.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:24.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:23 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:23.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:24.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:23 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:23.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:24.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:23 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:23.871+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:24.532 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:24.261+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:24.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:24.261+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:24.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:24.261+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:24.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:24 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:24.261+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:24.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:24.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:24.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:24.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:24.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:24.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:24.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:24.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:25.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:24.658+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:25.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:24.658+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:25.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:24.658+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:25.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:24 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:24.658+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:58.451548+0000 front 2023-10-06T21:36:58.451569+0000 (oldest deadline 2023-10-06T21:37:23.751740+0000) 2023-10-06T21:37:25.278 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:25.020+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:25.278 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:25.020+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:25.279 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:25.020+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:25.279 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:25.020+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:25.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:24.866+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:25.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:24.866+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:25.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:24.866+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:25.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:24 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:24.866+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:25.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:25.276+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:25.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:25.276+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:25.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:25.276+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:25.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:25 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:25.276+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:25.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:25.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:25.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:25.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:25.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:25.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:25.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:25.155+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:26.073 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:25 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:25.820+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:26.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:25 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:25.820+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:26.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:25 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:25.820+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:26.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:25 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:25.820+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:26.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:25.677+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:35.847139+0000 front 2023-10-06T21:36:35.847216+0000 (oldest deadline 2023-10-06T21:37:00.547282+0000) 2023-10-06T21:37:26.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:25.677+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:41.048409+0000 front 2023-10-06T21:36:41.048393+0000 (oldest deadline 2023-10-06T21:37:08.648919+0000) 2023-10-06T21:37:26.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:25.677+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.550319+0000 front 2023-10-06T21:36:51.550126+0000 (oldest deadline 2023-10-06T21:37:15.650186+0000) 2023-10-06T21:37:26.121 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:25 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7[156731]: 2023-10-06T21:37:25.677+0000 7f1620202700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:58.451548+0000 front 2023-10-06T21:36:58.451569+0000 (oldest deadline 2023-10-06T21:37:23.751740+0000) 2023-10-06T21:37:26.321 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:26.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:26.321 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:26.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:26.321 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:26.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:26.321 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:26.057+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:26.483 INFO:journalctl@ceph.osd.7.smithi078.stdout:Oct 06 21:37:26 smithi078 bash[181223]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-7 2023-10-06T21:37:26.483 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:26.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:26.484 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:26.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:26.484 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:26.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:26.484 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:26.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:26.484 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:26.167+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:26.782 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:26.318+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:26.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:26.318+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:26.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:26.318+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:26.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:26 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:26.318+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:27.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:26 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:26.788+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:27.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:26 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:26.788+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:27.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:26 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:26.788+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:27.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:26 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:26.788+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:27.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:26 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:26.788+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:27.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:27.009+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:27.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:27.009+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:27.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:27.009+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:27.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:27.009+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:27.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:27 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:27.118+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:27.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:27 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:27.118+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:27.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:27 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:27.118+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:27.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:27 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:27.118+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:27.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:27 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:27.118+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:27.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-06T21:37:27.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:27.282+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:27.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:27.282+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:27.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:27.282+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:27.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:27 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:27.282+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:27.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:27.827 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-06T21:37:27.827 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-06T21:37:27.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.8 2023-10-06T21:37:28.180 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:27 smithi167 systemd[1]: Stopping Ceph osd.8 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:28.180 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:28.043+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:28.181 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:28.043+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:28.181 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:28.043+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:28.181 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:28.043+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:28.181 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:28.043+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:28.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:27 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:27.825+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:28.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:27 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:27.825+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:28.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:27 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:27.825+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:28.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:27 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:27.825+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:28.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:27 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:27.825+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.178+0000 7f6760e85700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.178+0000 7f6760e85700 -1 osd.8 131 *** Got signal Terminated *** 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.178+0000 7f6760e85700 -1 osd.8 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.295+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.295+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.295+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:28.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:28 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:28.295+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:28.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:28 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:28.161+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:28.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:28 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:28.161+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:28.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:28 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:28.161+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:28.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:28 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:28.161+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:28.622 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:28 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:28.161+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:29.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:28.842+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:29.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:28.842+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:29.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:28.842+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:29.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:28.842+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:29.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:28 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:28.842+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:29.259+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:29.259+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:29.259+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:29.259+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:29.051+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:29.051+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:29.051+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:29.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:29.051+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:29.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:29.051+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:29.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:29 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:29.143+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:29.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:29 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:29.143+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:29.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:29 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:29.143+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:29.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:29 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:29.143+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:29.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:29 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:29.143+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:30.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:29 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:29.876+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:30.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:29 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:29.876+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:30.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:29 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:29.876+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:30.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:29 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:29.876+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:30.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:29 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:29.876+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:30.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:30.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:30.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:30.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:30.062+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:30.062+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:30.062+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:30.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:30.062+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:30.534 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:30 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:30.062+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:30.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:30 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:30.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:30.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:30 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:30.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:30.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:30 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:30.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:30.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:30 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:30.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:30.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:30 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:30.119+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:31.308 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:31.308 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:31.309 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:31.309 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:31.309 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:31.309 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:31.034+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:31.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:30 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:30.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:31.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:30 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:30.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:31.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:30 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:30.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:31.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:30 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:30.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:31.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:30 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:30.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:31.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:31 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:31.147+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:31.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:31.306+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:31.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:31.306+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:31.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:31.306+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:31.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:31 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:31.306+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:32.306 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:32.041+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:32.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:31 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:31.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:32.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:32 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:32.158+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:32.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:32.303+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.605303+0000 front 2023-10-06T21:36:37.605540+0000 (oldest deadline 2023-10-06T21:37:01.105381+0000) 2023-10-06T21:37:32.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:32.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:43.406576+0000 front 2023-10-06T21:36:43.406672+0000 (oldest deadline 2023-10-06T21:37:08.706745+0000) 2023-10-06T21:37:32.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:32.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.607742+0000 front 2023-10-06T21:36:51.607852+0000 (oldest deadline 2023-10-06T21:37:16.907749+0000) 2023-10-06T21:37:32.783 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:32 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8[156584]: 2023-10-06T21:37:32.304+0000 7f675ac76700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:00.909769+0000 front 2023-10-06T21:37:00.909728+0000 (oldest deadline 2023-10-06T21:37:23.209708+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:33.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:32 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:32.893+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:33.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.037+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:33.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:33 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:33.181+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:33.994 INFO:journalctl@ceph.osd.8.smithi167.stdout:Oct 06 21:37:33 smithi167 bash[181070]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-8 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:34.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:33 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:33.991+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:34.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:34.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:34.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:34.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:34.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:34.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:33 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:33.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:34.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:34 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:34.165+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:35.214 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:35 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:35.007+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:35.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-06T21:37:35.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:35.260 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-06T21:37:35.260 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-06T21:37:35.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.9 2023-10-06T21:37:35.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:35.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:35.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:35.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:35.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:35.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:34 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:34.860+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:35.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:35 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:35.156+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:35.784 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 systemd[1]: Stopping Ceph osd.9 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:36.073 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.780+0000 7f1015aeb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.780+0000 7f1015aeb700 -1 osd.9 131 *** Got signal Terminated *** 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.780+0000 7f1015aeb700 -1 osd.9 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:36.074 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:35 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:35.847+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:36.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.012+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:36.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:36.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:36.622 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:36.622 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:36.622 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:36.622 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:36 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:36.139+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:37.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:36 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:36.979+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:37.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:37.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:37.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:37.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:37.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:37.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:36 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:36.872+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:37.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:37 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:37.114+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:38.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:38 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:38.029+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:38.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:38.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:38.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:38.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:38.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:38.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:37 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:37.900+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:38.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:38 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:38.106+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:39.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:39.008+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:39.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:39.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:39.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:39.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:39.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:39.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:38 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:38.910+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:39.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:39 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:39.077+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:40.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:40 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:40.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:40.323 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:27.275026+0000 front 2023-10-06T21:36:27.275415+0000 (oldest deadline 2023-10-06T21:36:54.275963+0000) 2023-10-06T21:37:40.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:34.777295+0000 front 2023-10-06T21:36:34.777271+0000 (oldest deadline 2023-10-06T21:37:00.076938+0000) 2023-10-06T21:37:40.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:45.179838+0000 front 2023-10-06T21:36:45.179890+0000 (oldest deadline 2023-10-06T21:37:11.079954+0000) 2023-10-06T21:37:40.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:51.080475+0000 front 2023-10-06T21:36:51.080461+0000 (oldest deadline 2023-10-06T21:37:17.481080+0000) 2023-10-06T21:37:40.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.782370+0000 front 2023-10-06T21:36:59.782299+0000 (oldest deadline 2023-10-06T21:37:26.182736+0000) 2023-10-06T21:37:40.324 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:39 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9[141961]: 2023-10-06T21:37:39.880+0000 7f10100dd700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:06.183494+0000 front 2023-10-06T21:37:06.183323+0000 (oldest deadline 2023-10-06T21:37:30.883298+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:40.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:40 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:40.035+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:41.282 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:41.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:41.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:41.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:41.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:41.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:41 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.026+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:41.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:41.372 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:41 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:41.063+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:41.977 INFO:journalctl@ceph.osd.9.smithi026.stdout:Oct 06 21:37:41 smithi026 bash[183488]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-9 2023-10-06T21:37:42.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:42.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:42.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:42.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:42.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:42.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:41.999+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:42.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:42.373 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.032+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:43.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:42 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:42.982+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:43.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:43 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:43.040+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:43.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-06T21:37:43.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:43.803 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-06T21:37:43.803 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-06T21:37:43.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.10 2023-10-06T21:37:44.121 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 systemd[1]: Stopping Ceph osd.10 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:44.121 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:44.122 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:43 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:43.969+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:44.532 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:44.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:44.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:44.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:44.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:44.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:44 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:44.047+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:44.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:44 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:44.165+0000 7f682a544700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:44.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:44 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:44.165+0000 7f682a544700 -1 osd.10 131 *** Got signal Terminated *** 2023-10-06T21:37:44.621 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:44 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:44.165+0000 7f682a544700 -1 osd.10 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:45.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.006+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:45.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:45 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:45.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:46.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:45 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:45.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:46.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:46.372 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.004+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:47.285 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:47.285 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:47.285 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:47.285 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:47.286 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:47.286 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:46 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:46.910+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:47.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:46 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:46.970+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:48.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:47 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:47.908+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:48.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:48.372 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:47 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:47.935+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:49.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:48 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:48.957+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.184492+0000 front 2023-10-06T21:36:29.184589+0000 (oldest deadline 2023-10-06T21:36:52.084306+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.885901+0000 front 2023-10-06T21:36:37.885961+0000 (oldest deadline 2023-10-06T21:37:03.186064+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.887041+0000 front 2023-10-06T21:36:44.887146+0000 (oldest deadline 2023-10-06T21:37:09.487586+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:49.988918+0000 front 2023-10-06T21:36:49.988960+0000 (oldest deadline 2023-10-06T21:37:15.288851+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:36:59.890388+0000 front 2023-10-06T21:36:59.890437+0000 (oldest deadline 2023-10-06T21:37:25.190518+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:05.190950+0000 front 2023-10-06T21:37:05.191004+0000 (oldest deadline 2023-10-06T21:37:30.490971+0000) 2023-10-06T21:37:49.371 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:48 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10[165512]: 2023-10-06T21:37:48.939+0000 7f6826339700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-10-06T21:37:13.992164+0000 front 2023-10-06T21:37:13.992285+0000 (oldest deadline 2023-10-06T21:37:38.691888+0000) 2023-10-06T21:37:50.043 INFO:journalctl@ceph.osd.10.smithi078.stdout:Oct 06 21:37:49 smithi078 bash[181504]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-10 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:50.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:49 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:49.975+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:51.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:50 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:50.927+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:51.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-06T21:37:51.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:51.341 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-06T21:37:51.342 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-06T21:37:51.342 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.11 2023-10-06T21:37:51.702 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 systemd[1]: Stopping Ceph osd.11 for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:51.969 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.699+0000 7f58a5577700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:51.970 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.699+0000 7f58a5577700 -1 osd.11 131 *** Got signal Terminated *** 2023-10-06T21:37:51.970 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.699+0000 7f58a5577700 -1 osd.11 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-06T21:37:52.282 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:52.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:51 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:51.967+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:53.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:52 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:52.923+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:54.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:53 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:53.911+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:55.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:55.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:55.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:55.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:55.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:55.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:55.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:54 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:54.942+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-10-06T21:36:29.701265+0000 front 2023-10-06T21:36:29.701213+0000 (oldest deadline 2023-10-06T21:36:54.301705+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6804 osd.1 since back 2023-10-06T21:36:37.103498+0000 front 2023-10-06T21:36:37.103620+0000 (oldest deadline 2023-10-06T21:37:00.003433+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2023-10-06T21:36:44.604832+0000 front 2023-10-06T21:36:44.604865+0000 (oldest deadline 2023-10-06T21:37:06.905037+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-10-06T21:36:52.706847+0000 front 2023-10-06T21:36:52.706819+0000 (oldest deadline 2023-10-06T21:37:15.006649+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6812 osd.4 since back 2023-10-06T21:37:01.407977+0000 front 2023-10-06T21:37:01.408004+0000 (oldest deadline 2023-10-06T21:37:27.307927+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2023-10-06T21:37:07.308331+0000 front 2023-10-06T21:37:07.308296+0000 (oldest deadline 2023-10-06T21:37:30.808480+0000) 2023-10-06T21:37:56.283 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.78:6820 osd.7 since back 2023-10-06T21:37:22.411224+0000 front 2023-10-06T21:37:22.411172+0000 (oldest deadline 2023-10-06T21:37:47.711216+0000) 2023-10-06T21:37:56.284 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:55 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11[164582]: 2023-10-06T21:37:55.933+0000 7f58a136c700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2023-10-06T21:37:32.412223+0000 front 2023-10-06T21:37:32.412258+0000 (oldest deadline 2023-10-06T21:37:55.912226+0000) 2023-10-06T21:37:57.533 INFO:journalctl@ceph.osd.11.smithi167.stdout:Oct 06 21:37:57 smithi167 bash[181349]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-osd-11 2023-10-06T21:37:58.751 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-06T21:37:58.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:37:58.792 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-06T21:37:58.792 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-06T21:37:58.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.a 2023-10-06T21:37:59.324 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:37:58 smithi026 systemd[1]: Stopping Ceph mds.a for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:37:59.823 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:37:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: 2023-10-06T21:37:59.343+0000 7f7b469e4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:37:59.824 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:37:59 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a[149937]: 2023-10-06T21:37:59.343+0000 7f7b469e4700 -1 mds.a *** got signal Terminated *** 2023-10-06T21:38:02.323 INFO:journalctl@ceph.mds.a.smithi026.stdout:Oct 06 21:38:02 smithi026 bash[183754]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-a 2023-10-06T21:38:02.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-06T21:38:02.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:02.473 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-06T21:38:02.473 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-06T21:38:02.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.d 2023-10-06T21:38:02.823 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:38:02 smithi026 systemd[1]: Stopping Ceph mds.d for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:03.323 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:38:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-d[148934]: 2023-10-06T21:38:02.976+0000 7f4a0b2fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:03.324 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:38:02 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-d[148934]: 2023-10-06T21:38:02.976+0000 7f4a0b2fd700 -1 mds.d *** got signal Terminated *** 2023-10-06T21:38:07.324 INFO:journalctl@ceph.mds.d.smithi026.stdout:Oct 06 21:38:06 smithi026 bash[183892]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-d 2023-10-06T21:38:07.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-06T21:38:07.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:07.503 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-06T21:38:07.503 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-06T21:38:07.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.g 2023-10-06T21:38:07.823 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:38:07 smithi026 systemd[1]: Stopping Ceph mds.g for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:08.324 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:38:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-g[149552]: 2023-10-06T21:38:08.001+0000 7f675ce75700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:08.324 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:38:08 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-g[149552]: 2023-10-06T21:38:08.001+0000 7f675ce75700 -1 mds.g *** got signal Terminated *** 2023-10-06T21:38:12.573 INFO:journalctl@ceph.mds.g.smithi026.stdout:Oct 06 21:38:12 smithi026 bash[184038]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-g 2023-10-06T21:38:12.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-06T21:38:12.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:12.853 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-06T21:38:12.853 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-06T21:38:12.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.j 2023-10-06T21:38:13.073 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:38:12 smithi026 systemd[1]: Stopping Ceph mds.j for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:13.823 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:38:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-j[148546]: 2023-10-06T21:38:13.349+0000 7f6695923700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:13.824 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:38:13 smithi026 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-j[148546]: 2023-10-06T21:38:13.349+0000 7f6695923700 -1 mds.j *** got signal Terminated *** 2023-10-06T21:38:15.919 INFO:journalctl@ceph.mds.j.smithi026.stdout:Oct 06 21:38:15 smithi026 bash[184179]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-j 2023-10-06T21:38:16.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-06T21:38:16.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:16.324 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-06T21:38:16.324 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-06T21:38:16.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.b 2023-10-06T21:38:16.621 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:38:16 smithi078 systemd[1]: Stopping Ceph mds.b for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:17.121 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:38:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: 2023-10-06T21:38:16.686+0000 7f78c7aef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:17.121 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:38:16 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b[171871]: 2023-10-06T21:38:16.686+0000 7f78c7aef700 -1 mds.b *** got signal Terminated *** 2023-10-06T21:38:18.370 INFO:journalctl@ceph.mds.b.smithi078.stdout:Oct 06 21:38:18 smithi078 bash[181774]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-b 2023-10-06T21:38:18.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-06T21:38:18.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:18.485 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-06T21:38:18.485 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-06T21:38:18.485 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.e 2023-10-06T21:38:18.864 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:38:18 smithi078 systemd[1]: Stopping Ceph mds.e for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:19.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:38:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: 2023-10-06T21:38:18.860+0000 7fcb97d1a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:19.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:38:18 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e[171192]: 2023-10-06T21:38:18.860+0000 7fcb97d1a700 -1 mds.e *** got signal Terminated *** 2023-10-06T21:38:22.121 INFO:journalctl@ceph.mds.e.smithi078.stdout:Oct 06 21:38:22 smithi078 bash[181922]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-e 2023-10-06T21:38:22.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-06T21:38:22.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:22.250 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-06T21:38:22.250 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-06T21:38:22.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.h 2023-10-06T21:38:22.620 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:38:22 smithi078 systemd[1]: Stopping Ceph mds.h for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:23.121 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:38:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: 2023-10-06T21:38:22.627+0000 7f27c3019700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:23.121 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:38:22 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h[171531]: 2023-10-06T21:38:22.627+0000 7f27c3019700 -1 mds.h *** got signal Terminated *** 2023-10-06T21:38:26.094 INFO:journalctl@ceph.mds.h.smithi078.stdout:Oct 06 21:38:26 smithi078 bash[182064]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-h 2023-10-06T21:38:26.280 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-06T21:38:26.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:26.314 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-06T21:38:26.315 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-06T21:38:26.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.k 2023-10-06T21:38:26.621 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:38:26 smithi078 systemd[1]: Stopping Ceph mds.k for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:27.121 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:38:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-k[170849]: 2023-10-06T21:38:26.694+0000 7f103d68e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:27.121 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:38:26 smithi078 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-k[170849]: 2023-10-06T21:38:26.694+0000 7f103d68e700 -1 mds.k *** got signal Terminated *** 2023-10-06T21:38:28.849 INFO:journalctl@ceph.mds.k.smithi078.stdout:Oct 06 21:38:28 smithi078 bash[182209]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-k 2023-10-06T21:38:29.049 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-06T21:38:29.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:29.081 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-06T21:38:29.082 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-06T21:38:29.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.c 2023-10-06T21:38:29.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:38:29 smithi167 systemd[1]: Stopping Ceph mds.c for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:29.782 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:38:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: 2023-10-06T21:38:29.444+0000 7f34c3e80700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:29.783 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:38:29 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c[170761]: 2023-10-06T21:38:29.444+0000 7f34c3e80700 -1 mds.c *** got signal Terminated *** 2023-10-06T21:38:33.996 INFO:journalctl@ceph.mds.c.smithi167.stdout:Oct 06 21:38:33 smithi167 bash[181612]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-c 2023-10-06T21:38:34.001 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-06T21:38:34.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:34.038 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-06T21:38:34.038 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-06T21:38:34.038 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.f 2023-10-06T21:38:34.283 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:38:34 smithi167 systemd[1]: Stopping Ceph mds.f for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:34.782 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:38:34 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-f[170422]: 2023-10-06T21:38:34.417+0000 7f2a8a90b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:34.783 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:38:34 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-f[170422]: 2023-10-06T21:38:34.418+0000 7f2a8a90b700 -1 mds.f *** got signal Terminated *** 2023-10-06T21:38:38.895 INFO:journalctl@ceph.mds.f.smithi167.stdout:Oct 06 21:38:38 smithi167 bash[181754]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-f 2023-10-06T21:38:38.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-06T21:38:38.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:38.936 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-06T21:38:38.936 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-06T21:38:38.937 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i 2023-10-06T21:38:39.282 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:38:39 smithi167 systemd[1]: Stopping Ceph mds.i for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:39.783 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:38:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: 2023-10-06T21:38:39.313+0000 7f590453a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:39.783 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:38:39 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i[174618]: 2023-10-06T21:38:39.313+0000 7f590453a700 -1 mds.i *** got signal Terminated *** 2023-10-06T21:38:42.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Oct 06 21:38:41 smithi167 bash[181901]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-i 2023-10-06T21:38:42.267 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-06T21:38:42.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:42.301 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-06T21:38:42.301 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-06T21:38:42.302 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.l 2023-10-06T21:38:42.533 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:38:42 smithi167 systemd[1]: Stopping Ceph mds.l for 4dcc50a6-648b-11ee-8db5-212e2dc638e7... 2023-10-06T21:38:43.032 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:38:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-l[170073]: 2023-10-06T21:38:42.675+0000 7f7b63da3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-06T21:38:43.033 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:38:42 smithi167 ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-l[170073]: 2023-10-06T21:38:42.675+0000 7f7b63da3700 -1 mds.l *** got signal Terminated *** 2023-10-06T21:38:46.650 INFO:journalctl@ceph.mds.l.smithi167.stdout:Oct 06 21:38:46 smithi167 bash[182043]: ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7-mds-l 2023-10-06T21:38:46.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4dcc50a6-648b-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-06T21:38:46.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-06T21:38:46.882 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-06T21:38:46.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:38:47.294 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:38:50.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:38:51.057 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:38:52.676 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-06T21:38:52.986 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:38:54.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:38:54.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:38:54.675 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-06T21:38:54.704 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-06T21:38:54.706 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi026/crash 2023-10-06T21:38:54.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:38:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:38:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:38:54.799 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi078/crash 2023-10-06T21:38:54.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:38:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:38:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:38:54.849 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi167/crash 2023-10-06T21:38:54.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash -- . 2023-10-06T21:38:54.893 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-06T21:38:54.894 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2023-10-06T21:38:54.894 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-06T21:38:54.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-06T21:38:54.978 INFO:teuthology.orchestra.run.smithi026.stderr:grep: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-06T21:38:54.982 INFO:tasks.cephadm:Compressing logs... 2023-10-06T21:38:54.983 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:38:54.987 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:38:54.990 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-06T21:38:55.017 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T21:38:55.017 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2023-10-06T21:38:55.017 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-06T21:38:55.018 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-06T21:38:55.018 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-06T21:38:55.018 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-06T21:38:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-06T21:38:55.019 INFO:teuthology.orchestra.run.smithi167.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:38:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-06T21:38:55.019 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-06T21:38:55.025 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.f.log: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.c.log: 92.7% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-06T21:38:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-06T21:38:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2023-10-06T21:38:55.027 INFO:teuthology.orchestra.run.smithi167.stderr: 94.3% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-06T21:38:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-06T21:38:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 /var/log/ceph/cephadm.log: --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-06T21:38:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-06T21:38:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-06T21:38:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-06T21:38:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-06T21:38:55.036 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-06T21:38:55.036 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-06T21:38:55.036 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.i.log: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:38:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 94.4% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-06T21:38:55.040 INFO:teuthology.orchestra.run.smithi078.stderr: 92.6% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-06T21:38:55.047 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-06T21:38:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- : No such file or directory/var/log/ceph/cephadm.log 2023-10-06T21:38:55.051 INFO:teuthology.orchestra.run.smithi026.stderr: 2023-10-06T21:38:55.051 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:38:55.051 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-06T21:38:55.053 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-06T21:38:55.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-06T21:38:55.055 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-06T21:38:55.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-06T21:38:55.057 INFO:teuthology.orchestra.run.smithi078.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:38:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-06T21:38:55.059 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.l.log: 94.3% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-06T21:38:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.3.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-06T21:38:55.060 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-06T21:38:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-06T21:38:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-06T21:38:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-06T21:38:55.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.9.log: /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-06T21:38:55.070 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-06T21:38:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.j.log: 90.9% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:38:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-06T21:38:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-06T21:38:55.419 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-06T21:38:55.441 INFO:teuthology.orchestra.run.smithi167.stderr: 94.1% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-06T21:38:55.586 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-06T21:38:55.840 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-06T21:38:56.320 INFO:teuthology.orchestra.run.smithi078.stderr: 91.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-06T21:38:57.113 INFO:teuthology.orchestra.run.smithi078.stderr: 93.7% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-06T21:38:57.186 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-06T21:38:58.152 INFO:teuthology.orchestra.run.smithi078.stderr: 91.8% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-06T21:39:13.461 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-06T21:39:13.513 INFO:teuthology.orchestra.run.smithi026.stderr: 94.2% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-06T21:39:14.733 INFO:teuthology.orchestra.run.smithi026.stderr: 94.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-06T21:39:15.915 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-06T21:39:19.231 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-06T21:39:19.661 INFO:teuthology.orchestra.run.smithi167.stderr: 93.8% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-06T21:39:19.661 INFO:teuthology.orchestra.run.smithi167.stderr: 2023-10-06T21:39:19.661 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m24.445s 2023-10-06T21:39:19.661 INFO:teuthology.orchestra.run.smithi167.stderr:user 1m25.962s 2023-10-06T21:39:19.661 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m3.543s 2023-10-06T21:39:26.514 INFO:teuthology.orchestra.run.smithi026.stderr: 94.1% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-06T21:39:30.870 INFO:teuthology.orchestra.run.smithi026.stderr: 94.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-06T21:39:38.933 INFO:teuthology.orchestra.run.smithi026.stderr: 97.6% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-06T21:39:38.936 INFO:teuthology.orchestra.run.smithi026.stderr: 2023-10-06T21:39:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m43.913s 2023-10-06T21:39:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:user 2m19.467s 2023-10-06T21:39:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m12.548s 2023-10-06T21:39:44.079 INFO:teuthology.orchestra.run.smithi078.stderr: 94.0% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-06T21:39:52.271 INFO:teuthology.orchestra.run.smithi078.stderr: 93.9% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-06T21:39:57.347 INFO:teuthology.orchestra.run.smithi078.stderr: 93.8% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-06T21:40:06.233 INFO:teuthology.orchestra.run.smithi078.stderr: 93.9% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-06T21:40:12.199 INFO:teuthology.orchestra.run.smithi078.stderr: 97.5% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-06T21:42:04.253 INFO:teuthology.orchestra.run.smithi078.stderr: 98.3% -- replaced with /var/log/ceph/4dcc50a6-648b-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-06T21:42:04.343 INFO:teuthology.orchestra.run.smithi078.stderr: 2023-10-06T21:42:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:real 3m9.336s 2023-10-06T21:42:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:user 3m47.577s 2023-10-06T21:42:04.343 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m11.608s 2023-10-06T21:42:04.343 INFO:tasks.cephadm:Archiving logs... 2023-10-06T21:42:04.344 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi026/log 2023-10-06T21:42:04.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:42:19.456 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi078/log 2023-10-06T21:42:19.457 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:42:40.342 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi167/log 2023-10-06T21:42:40.343 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T21:42:53.049 INFO:tasks.cephadm:Removing cluster... 2023-10-06T21:42:53.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:42:53.452 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:42:54.546 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:42:55.300 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:42:55.947 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4dcc50a6-648b-11ee-8db5-212e2dc638e7 --force 2023-10-06T21:42:56.269 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 4dcc50a6-648b-11ee-8db5-212e2dc638e7 2023-10-06T21:42:56.859 INFO:tasks.cephadm:Removing cephadm ... 2023-10-06T21:42:56.859 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:42:56.909 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:42:56.937 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-06T21:42:56.955 INFO:tasks.cephadm:Teardown complete 2023-10-06T21:42:56.955 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T21:42:56.969 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T21:42:56.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:42:56.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:42:56.979 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T21:42:57.109 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T21:42:57.120 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T21:42:57.120 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:42:57.125 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:42:57.127 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T21:42:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:bash: ntpq: command not found 2023-10-06T21:42:57.143 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 -280us[ -280us] +/- 75ms 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 +148us[ +154us] +/- 55ms 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +76us[ +76us] +/- 74ms 2023-10-06T21:42:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:42:57.159 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:42:57.159 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2023-10-06T21:42:57.160 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 -274us[ -274us] +/- 75ms 2023-10-06T21:42:57.160 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 7 377 98 +180us[ +174us] +/- 54ms 2023-10-06T21:42:57.160 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +108us[ +108us] +/- 74ms 2023-10-06T21:42:57.160 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:42:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-10-06T21:42:57.172 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T21:42:57.172 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-10-06T21:42:57.172 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 -227us[ -227us] +/- 75ms 2023-10-06T21:42:57.172 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 +167us[ +177us] +/- 55ms 2023-10-06T21:42:57.173 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +113us[ +123us] +/- 74ms 2023-10-06T21:42:57.173 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T21:42:57.174 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T21:42:57.184 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T21:42:57.185 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T21:42:57.204 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:42:57.399 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:42:57.511 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T21:42:57.609 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:42:57.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:57.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:42:57.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:57.723 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T21:42:57.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:57.756 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T21:42:57.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T21:42:57.810 INFO:teuthology.task.internal:Duration was 3557.250161 seconds 2023-10-06T21:42:57.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T21:42:57.893 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T21:42:57.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:42:57.898 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:42:57.901 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T21:42:57.947 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:42:57.987 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:42:58.014 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T21:42:58.358 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T21:42:58.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2023-10-06T21:42:58.358 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:42:58.417 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2023-10-06T21:42:58.417 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:42:58.454 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T21:42:58.454 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T21:42:58.476 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T21:42:58.477 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:42:58.481 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:42:58.497 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T21:42:58.580 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T21:42:58.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:42:58.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:42:58.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T21:42:58.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T21:42:58.767 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T21:42:58.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:42:58.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:42:58.879 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T21:42:58.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T21:42:58.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:58.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:58.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:58.973 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2023-10-06T21:42:58.978 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2023-10-06T21:42:58.988 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2023-10-06T21:42:59.024 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:59.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:59.066 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:59.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:59.083 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T21:42:59.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T21:42:59.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T21:42:59.111 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T21:42:59.112 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi026 2023-10-06T21:42:59.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:42:59.198 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi078 2023-10-06T21:42:59.199 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:42:59.238 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643/remote/smithi167 2023-10-06T21:42:59.239 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T21:42:59.281 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T21:42:59.282 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:42:59.286 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:42:59.289 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T21:42:59.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T21:42:59.343 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T21:42:59.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T21:42:59.376 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T21:42:59.376 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:42:59.381 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:42:59.383 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T21:42:59.397 INFO:teuthology.orchestra.run.smithi078.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:42 /home/ubuntu/cephtest 2023-10-06T21:42:59.399 INFO:teuthology.orchestra.run.smithi167.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:42 /home/ubuntu/cephtest 2023-10-06T21:42:59.420 INFO:teuthology.orchestra.run.smithi026.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 21:42 /home/ubuntu/cephtest 2023-10-06T21:42:59.423 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T21:42:59.435 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T21:42:59.498 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T21:42:59.526 DEBUG:teuthology.nuke:shortname: smithi026 2023-10-06T21:42:59.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:42:59.549 DEBUG:teuthology.nuke:shortname: smithi078 2023-10-06T21:42:59.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:42:59.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.094078', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:42:59.573 DEBUG:teuthology.nuke:shortname: smithi167 2023-10-06T21:42:59.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T21:42:59.579 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.095683', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:42:59.594 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 20:35:06.092210', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-06T21:42:59.607 INFO:teuthology.orchestra.console.smithi026:Power off 2023-10-06T21:42:59.607 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:42:59.623 INFO:teuthology.orchestra.console.smithi078:Power off 2023-10-06T21:42:59.623 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:42:59.636 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2023-10-06T21:42:59.636 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:42:59.644 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2023-10-06T21:42:59.645 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:42:59.659 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-10-06T21:42:59.664 INFO:teuthology.orchestra.console.smithi167:Power off 2023-10-06T21:42:59.664 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T21:42:59.675 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-06T21:42:59.687 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-10-06T21:42:59.687 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:42:59.710 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T21:43:03.660 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:03.676 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:03.711 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:03.772 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-10-06T21:43:03.785 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-06T21:43:03.822 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T21:43:07.775 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:07.786 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:07.823 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T21:43:07.885 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2023-10-06T21:43:07.896 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2023-10-06T21:43:07.939 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-10-06T21:43:07.986 INFO:teuthology.orchestra.console.smithi026:Power off completed 2023-10-06T21:43:07.997 INFO:teuthology.orchestra.console.smithi078:Power off completed 2023-10-06T21:43:08.040 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-10-06T21:43:08.145 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/enabled mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 3557.2501606941223 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-10-06T21:43:08.146 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T21:43:08.227 INFO:teuthology.run:FAIL